An Equitable Health Collaborative

Transcription

1 An Equitable Health Collaborative 1

2 Measurement and Variation Dan Nelson, M.D. 2

3 Part 1 3

4 Measurement and Variation Dan Nelson, M.D. 4

5 Part 2 5

6 Improvement How will we know that a change is an improvement? 6

7 Triple Aim Experience Health Cost 7

8 Triple Aim Patient Meaningful Clinically Meaningful Economically Meaningful 8

9 Triple Aim Patient Meaningful QOL Functional Assessment Patient Satisfaction Clinically Meaningful HGA1c BP Med Reconciliation Economically Meaningful # ER Visits Readmission rate LOS 9

10 Measurement 10

11 Measurement Research, Improvement & Accountability Type Audience Purpose Character R Scientific community New knowledge Complex, slow, precise I Providers of care Understand processes, confirm results or efforts Simple, rapid, motivating A Customers (patients, purchasers) Provide basis for choice, reassure Comparative, summarized

12 You Can t Fatten a Cow by Weighing it Palestinian Proverb 12

13 Measurement and Data Measure To test changes and to learn Not to judge Data: Understand, control, improve processes & systems Do not improve processes, people do. 13

14 Goals of Measurement Focuses: Improvement efforts Facilitates: Objective evaluation of progress Motivates: Provides feedback to the team Eliminates: Wishful thinking Accelerates: Improvement, doesn t slow it down 14

15 Characteristics of a Measure Provides an answer to: How will we know that a change is an improvement? 15

16 Characteristics of a Measure Directly relates to the aim Clearly identifies the patient population Reflects progress toward achieving the aim Represents small, frequent samples 16

17 Characteristics of a Measure Population Aim Data Often expressed as a percentage Denominator: sample of cases observed Numerator: number of case observed that meet specified criteria 17

18 Types of Data Variable (Continuous) Length, Time, Temperature, Money Count (Classification or Attribution) # Errors, Yes/No, Integers (1, 2, 3, ) 18

19 Types of Measures Outcome Intermediate Process Proxy Balancing 19

20 Outcome Measures Clearly state how the result of the aim will be assessed Outcome data demonstrate whether an outcome or a result has been impacted Example: Percentage of patients with blindness, leg amputations, MI 20

21 Intermediate Measures Predict outcome Data demonstrates that if we obtain x then y will happen Example: A higher percentage of patients with a HbA1C <8 will result in less diabetic retinopathy, amputations, or death 21

22 Process Measures Assess the changes that are needed to support improvement of the outcomes These data demonstrate whether a process has been impacted (e.g., # of pts tested, # of pts who receive diabetic education) 22

23 Proxy Measures Indirect measures that coincide with or approximate the outcome These data demonstrate indirectly that an aim has been achieved (a decrease in EMD visits in a population of patients with CHF, means we are controlling progression of their disease) 23

24 Balancing Measures Assess potential adverse reactions These data demonstrate whether a change on a process impacted another process 24

25 Balancing Measures Outcome (quality or time) Transactional (volume, # of pts) Productivity (cycle time, efficiency) Financial (charges, staff hours, materials) Patient Satisfaction (surveys, complaints) Staff Satisfaction (MDs, RNs, etc ) 25

26 Aim- Decrease Infection Rate Outcome- Decreased rate of infections Intermediate- Increased rate of hand washing Process- Staff Education (% attended training) Proxy- Refill rate of soap or alcohol-based sanitizers Balancing- Cost of soap; staff satisfaction, Incidence of chapped hands 26

27 Numerator Number of cases from the sample that meet the measurement criteria. Less than or equal to denominator Example for diabetes # of patients whose HbA1C < 8 27

28 Denominator Reflects the total number of tests for a measure. Greater than or equal to the numerator. Diabetes example Total # of patients who had a HbA1C completed. 28

29 Measure Numerator Denominator # of patients whose HbA1C < 8 Total # of patients who had a HbA1C completed. 29

30 Rate Use if percentage is too small (e.g., 0.2%) Multiply percentage to arrive at whole number Example: Rate of codes per 1000 discharges (e.g,.002 x 1000 = 2 per 1000 discharges) The measure and data display must specify that the data are in the form of a rate 30

31 Operational Definition A description, in quantifiable term, of what to measure and the steps to measure it consistently and reliably. 1. Gives communicable meaning to a concept 2. Is clear and unambiguous 3. Usually numerical 4. Specifies measurement method & equipment 31

32 32

33 On Time Arrival? The arrival time of a flight will be the time the wheels touch down on the runway Time will be determined by a time piece use for navigation for the flight. An arrival will be on time if the arrival time is not more than 15 after the scheduled arrival time. 33

34 Tips About Measurement Seek usefulness, not perfection Don t wait for IS Use small, frequent sample sizes Use qualitative & quantitative data 34

35 Common Pitfalls of Measurement No clear aim, uses, & users No commitment to a common aim by those who will be using the measures Starting too big Measuring too many things 35

36 Evaluating Measures Answers: How will we know that a change is an improvement? Derived from the aim Defined patient population Trending reflects progress Data are available Able to measure frequently 36

37 Other Types of Measures 37

38 Composite Measure Includes component measures (4-8) [a/z x b/z x c/z] or [a/z + b/z + c/z]/z Amount of credit given dependent on number of component measures met Example: Diabetes composite measure 90%x90%x90%x90%x90%= 59% [90%+90%+90%+90%+90%]/5= 90% 38

39 All-or-None Measure Includes component measures (4-8) Credit given ONLY if meet all component measures Higher standard Fosters system perspective Examples: MN Community Measurement VAP bundle 39

40 HgA1c Lipid HgA1c Lipid HgA1c Lipid 40

41 Data Collection 41

42 Data collection itself is a process that can always be improved 42

43 Key Consideration: Measuring for Improvement Small sample sizes Few measures, easy to collect Frequent sampling 43

44 Key Consideration: Measuring for Improvement Users involved in the selection of measures Develop a data collection plan Data collection tools reflect the plan 44

45 Once you have Defined your Improvement Project Defined your AIM(S) Defined your MEASURES NOW you have to collect the data 45

46 Data Collection Plan 1. Define the patient population 2. How to identify that patient population 3. How to collect the data 4. Sample size and sample frequency 5. Who collects the data 6. Where will the data be presented after collection and analysis? 46

47 Defining the Patient Population Which patients in the sample? - Age - Diagnosis - Time frame of visit 47

48 How will you identify this patient population? Examples IS systems Claims data Registry Lab logs Concurrently 48

49 How will you collect the data? Examples Chart audits Retrospective Concurrent Patient collected data: - Surveys - Telephone follow-up Visit planning forms 49

50 Sample size and frequency Sampling intervals (clearly defined): - One time sample (baseline) - Ongoing (ie: at time of visit) Weekly Monthly Quarterly 50

51 Sample size and frequency Sample sizes: Baseline: 50 to 100 charts (dependent on frequency of diagnosis as well) Ongoing: per month, could be 5 to 10 per week, could be more if automated systems or registries 51

52 WHO collects the data? Examples: People involved in the process QI team - all involved Quality resource staff Centralized group of data collectors (Epic) 52

53 Where will data be presented? Examples QI committee Board Sponsor Med/Exec committees Department staff meetings Clinic sites staff meetings 53

54 Data Collection Tools Reflect the data collection plan Keep it simple and easy to understand Make it easy to summarize the data on the same tool for easy for data entry 54

55 Data Collection Sheet Time Patient Spends in Clinic Date: xx/xx/xxxx Pt 01 Pt 02 Pt 03 Pt 04 Pt Check-In Pt Roomed Nurse Enters Nurse Leaves MD Enters MD Leaves Patient Leaves MD Name: Comments: Nurse Name: Pt. ID Date: Site: AM or PM 55

56 Data Collection Sheet Time Patient Spends in Clinic Date: xx/xx/xxxx Pt 01 Pt 02 Pt 03 Pt 04 Pt Check-In 9:05 9:15 Pt Roomed 9:12 9:20 Nurse Enters 9:17 9:27 Nurse Leaves 9:25 9:35 MD Enters 9:45 10:10 MD Leaves 10:05 10:30 Patient Leaves 10:10 10:30 MD Name: Dr Nelson Comments: Nurse Name: Smith Pt. ID 01 Date: 04/21/10 Site: HSC AM or PM 56

57 PITFALLS to data collection Attempting to be ABSOLUTELY complete: - Volumes of data collected are overwhelming - Data are not pertinent - Time required is not reasonable - Analyzing data will be difficult 57

58 Summary You need a data collection plan 58

59 Variation and Measurement Dan Nelson, M.D. 59

60 CycleTime (min) Improvement in Cycle Time Avg Before Change 35 Avg After Change 60

61 Tracking Data Over Time Unit #1 Change Made Unit #2 Change Made Change Made Unit #3 All have an average of 70 before and 35 after change 61

62 Percent Mortality Average CABD Mortality Before and After the Implementation of a New Protocol % WOW A Significant Drop From 5% to 4% 4.0% 3.8 Time 1 Time 2 Conclusion: The protocol was a success A 20% drop in the average mortality 62

63 Percent Mortality Average CABD Mortality Before and After the Implementation of a New Protocol 9.0 Protocol Implemented Here Now what do you conclude about the Impact of the protocol? 63

64 If you don t understand the variation in your data, you will be tempted to Deny the data (It doesn t fit my view of reality) See trends where there are no trends Try to explain natural variation as special events Blame and give credit to people for things over which they have no control Distort the process that produced the data Kill the messenger! 64

65 Number Understanding Variation Every data set contains random variation Some data sets may contain signals Before you can detect a signal within any data set, you must filter out the noise (random or common cause variation) Time 65

66 Number Misinterpreting Variation Interpreting random/common cause variation as a meaningful departure from the past (seeing trends where there are no trends) Time 66

67 Number Misinterpreting Variation Not recognizing when a change has occurred in a process (failing to detect a signal when it is present) Time 67

68 Number Misinterpreting Variation Blame or credit people for things over which they have no control Time 68

69 Common Cause Variation An inherent part of every process Is random and due to regular, natural or ordinary causes Affects all outcomes of a process 69

70 Common Cause Variation Affects all outcomes of a process Results in a stable process because the variation is predictable Is also known as random or unassignable causes of variation 70

71 Special Cause Variation Is due to irregular or unnatural causes that are not inherent in a process Affects some, but not necessarily all, outcomes of a process 71

72 Special Cause Variation Special causes result in the process being unstable or out of control Is also known as non-random or assignable causes of variation 72

73 Unit of Measure Example of a Run Chart Time 73

74 BMI Tools to Understand Variation Run Chart New chart Run chart Control Chart New chart (1) c chart Temporary: UCL = 49.69, Mean = 32.57, LCL = UCL Mean LCL 1 2 Powered by: CHARTrunner PQ Systems 14 incorporated Jan Feb Powered by: CHARTrunner Mar Apr Jan Feb Mar Apr Jan Feb Mar Apr Jan PQ Systems incorporated Feb Scatter Plot Frequency Plot Pareto Chart 80 BMI Values vs Age- Endo Scattergram y = x Correlation Coefficient = 0.08 Coefficient of Determination = 0.01 t-statistic = Adult Medicine BMI Values Histogram Count BMI- Missing Data Adult Count Ht Missing % Both Missing % Wt Missing % AGE Percent 10% 20% 30% 40% 50% 60% 74

75 Document Project Next Steps Be prepared to present in Power Point (template will be provided) your flowchart and PDSA documents at the next session to share with the other teams and learn Team Member June 17 th Save presentation on EBAN Shared Drive Have Fun! Team Continuous Check in with Coach as needed to get questions answered Team Leader Continuous

76 Questions Contact your coach PDSA questions Dan Nelson, MD

77 An Equitable Health Collaborative 77