Comparing Cohorts of Event Sequences

Transcription

1 Comparing Cohorts of Event Sequences A VISUAL ANALYTICS APPROACH presented by Sana Malik with Fan Du, Catherine Plaisant, and Ben Shneiderman May 26, 2016 HCIL 33 rd Annual Symposium, College Park

2 often, analysts compare cohorts within datasets

3 any groups of users, patients, or records often, analysts compare cohorts within datasets

4

5

6 ?

7 FREQUENT PATTERNS

8 ABSENCE OF EVENTS

9 DURATION

10 Data Collection Cohort Selection Statistics

11 Data Collection Cohort Selection Visual Analytics Statistics

12 Data Collection Cohort Selection Visual Analytics Statistics

13 EVENTFLOW Monroe et al. Temporal event sequence simplification. IEEE Transactions on Visualization and Computer Graphics (TVCG 2013).

14 EVENTFLOW Monroe et al. Temporal event sequence simplification. IEEE Transactions on Visualization and Computer Graphics (TVCG 2013).

15 EVENTFLOW? Monroe et al. Temporal event sequence simplification. IEEE Transactions on Visualization and Computer Graphics (TVCG 2013).

16 Data Collection Cohort Selection Visual Analytics Statistics

17 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Exit 38.37, 0.0, 4.11e-123 Emergency -> ICU -> Exit 24.61, 0.0, 2.11e-73 Emergency -> Normal Floor Bed -> Exit -> ICU 5.26, 0.0, 4.12e-15 Emergency -> Normal Floor Bed -> ICU -> Exit 5.26, 0.0, 4.12e-15 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> Exit 7.89, 3.22, 1.20e-06 Aspirin -> Emergency -> ICU -> Intermediate Care -> Exit 4.15, 0.0, 4.22e-12 Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 2.97, 0.0, 7.02e-09 Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.01, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 0.76, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> Exit 0.0, 1.61, 3.37e-05 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 2.37, 2.84e-07 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 3.82, 6.45, 0.00 Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 4.07, 7.17e-12 Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 4.24, 2.48e-12 Emergency -> Exit 0.0, 11.88, 9.00e-34 Emergency -> ICU -> Normal Floor Bed -> Exit 0.0, 16.12, 2.18e-46 Aspirin -> Emergency -> Exit 0.0, 47.79, 2.49e-162 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 0.59, 0.02 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05

18 5.26, 0.0, 4.12e-15 Emergency -> Normal Floor Bed -> ICU -> Exit 5.26, 0.0, 4.12e-15 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> Exit 7.89, 3.22, 1.20e-06 Aspirin -> Emergency -> ICU -> Intermediate Care -> Exit 4.15, 0.0, 4.22e-12 Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 2.97, 0.0, 7.02e-09 Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.01, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 0.76, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> Exit 0.0, 1.61, 3.37e-05 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 2.37, 2.84e-07 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 3.82, 6.45, 0.00 Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 4.07, 7.17e-12 Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 4.24, 2.48e-12 Emergency -> Exit 0.0, 11.88, 9.00e-34 Emergency -> ICU -> Normal Floor Bed -> Exit 0.0, 16.12, 2.18e-46 Aspirin -> Emergency -> Exit 0.0, 47.79, 2.49e-162 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 0.59, 0.02 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.01, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 0.76, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> Exit

19 SAS STATA SAS Business Analytics Software. Vers SAS Institute, Computer software. StataCorp Stata Statistical Software: Release 14. College Station, TX: StataCorp LP.

20 Data Collection Cohort Selection Visual Analytics Statistics

21 Visual Analytics Data Collection Cohort Selection Statistics

22

23 HIGH-VOLUME Hypothesis Testing

24 HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS

25 HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS REAL-WORLD Case Study

26 HIGH-VOLUME Hypothesis Testing

27

28 Emergency Room

29 Normal Floor Bed

30 ICU

31 Discharged

32

33 start and end of record

34

35

36

37

38

39 non-consecutive (contains other events between)

40 1 SHORT SEQUENCE 14 UNIQUE PATTERNS non-consecutive (contains other events between)

41 RECORD COVERAGE Does this sequence occur in more records in one cohort than the other? DURATION On average, does this sequence take longer in one cohort than the other? FREQUENCY On average, does this sequence occur more frequently per record in one cohort than the other?

42 RECORD COVERAGE Does this sequence occur in more records in one cohort than the other? 14 UNIQUE PATTERNS DURATION On average, X 3 does METRICS this sequence take longer in one cohort than the other? 42 On average, does HYPOTHESES this sequence occur more frequently per user in one cohort than the other? FREQUENCY

43 HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS REAL-WORLD Case Study

44 Systematic Exploration OF RESULTS

45

46 Demo

47 HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS REAL-WORLD Case Study

48 REAL-WORLD Case Study

49 MULTI-DIMENSIONAL IN-DEPTH LONG-TERM CASE STUDIES (MILCS) Entry Interview & Training (1 session) Partners Use Tool Partners Provide Feedback (3 months) Researchers Refine Tool Exit Interview (1 session) For Researchers Demonstrate utility, refine tool For Partners Papers, insights, discoveries B. Shneiderman and C. Plaisant. Strategies for evaluating information visualization tools: Multidimensional in-depth long-term case studies. In BELIV 06: Proceedings of the 2006 AVI workshop on BEyond time and errors, pages 1 7. ACM, 2006.

50 CASE STUDY PARTNERS

51 CASE STUDY PARTNERS

52 PARTICIPANTS & DATASET Three analysts at Adobe One experienced user Two novice users Users events on a product website viewing the display ads signing up for promotions or free trials purchasing products Dataset Size 6,999 users 124 events types / 81,563 events

53 GOAL Compare users who purchased a product with using trials versus without using trials to understand ad-related behaviors

54 SYSTEM USE

55 SYSTEM USE

56 SYSTEM USE Event filtering was the most helpful to focus the analysis

57 SYSTEM USE Reduced metric calculation time provided a much better user experience for data analysis

58 RESULTS: FOR PARTNERS Users who had a trial viewed display ads more than the other group & contained more retargeting events. Analysts hypothesized trial users were explorers and non-trial users were experienced users based on event pattern differences

59 Future Work Extensions to other data types (e.g., networks) HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS Interval events REAL-WORLD Case Study Cohort selection

60 Comparing Cohorts of Event Sequences A VISUAL ANALYTICS APPROACH TRY COCO! maliks@cs.umd.edu www presented by Sana Malik Support from Adobe, Oracle, and the University of Maryland s Center for Health-related Informatics & Bioimaging (CHIB).