Methodological Issues

Transcription

1 Methodological Issues Presentation by Ian Brownwood for the meeting of Health Promotion, Prevention and Primary Care Subgroup 22 October, 2009, Paris

2 The Indicators Asthma* Chronic Obstructive Pulmonary Disease* Congestive Heart Failure* Angina Hypertension* Diabetes Short -term complications* Long -term complications Lower -extremity amputation* Uncontrolled * To be published in the 2009 edition of OECD Health at a Glance

3 Their Nature Based on data from routine hospital administrative information systems. Conditions are considered amenable or sensitive to care provided in primary care. Lower values are intended to be reflective of effective primary care systems. Does not imply that indicator values of zero can be achieved.

4 Their Definition AHRQ definitions except: Aged 15+ instead of 18+ years Exclusion of day cases. Mapping between ICD9 and ICD10. Procedure coding Based on PDX codes, except for: Asthma (i.e. cystic fibrosis and anomalies of the respiratory system) Lower-extremity amputation (i.e. trauma) COPD (i.e. qualification of bronchitis)

5 Validity and Comparability Key issues considered: 1. Potential confounding factors 2. Data coverage of the hospital sector 3. Reporting of meaningful composites 4. Indicator instability in small populations 5. Indicator-specific issues: Technical specification of indicators Cross walk of ICD 9 and ICD 10 codes Country variations in coding practice/capacity

6 Confounding Factors Potential confounding factors Demand Prevalence Supply Hospital Expenditures, Staff, Beds, Bed days and Admissions Potential explanatory factors GP per 100,000 Preliminary examination during HCQI data collection for

7 WE DID FIND..

8 Demand

9 Demand

10 Supply

11 Supply

12 Explanatory

13 Explanatory

14 BUT THEN WE ALSO FOUND

15 Demand

16 Supply

17 Explanatory

18 Conclusions Some evidence of relations but not strong or consistent. Data limitations: Invalidated country reported prevalence rates. Small sample size Temporal issues for diabetes long-term complications. Some of the key findings reflected in 2009 edition of Health at a Glance.

19 Key Issues 1. Potential confounding factors 2. Data coverage of the hospital sector 3. Reporting of meaningful composites 4. Indicator instability in small populations 5. Indicator-specific issues: Technical specification of indicators Cross walk of ICD 9 and ICD 10 codes Country variations in coding practice/capacity

20 Hospital Coverage Denominator =Population not Admissions (i.e. not same unit of measurement). Sensitive to coverage of hospital system. Not all countries provided the additional data (e.g. UK could be 10% understated). Adjustment of rates may result in slight overestimation in some cases (i.e. 40% of countries indicated they thought the public rate would be higher).

21 Adjusted Rates

22 Conclusions Without adjustment potential exists for underestimation of rates in some countries. Incomplete additional data for Unadjusted rates reported in 2009 edition of Health at a Glance. Reconsider application of adjusted rates based on enhanced data in

23 Key Issues 1. Potential confounding factors 2. Data coverage of the hospital sector 3. Reporting of meaningful composites 4. Indicator instability in small populations 5. Indicator-specific issues: Technical specification of indicators Cross walk of ICD 9 and ICD 10 codes Country variations in coding practice/capacity

24 Composite What is the objective? 1. To explain the variation in the individual indicators through the use of one or more composite indicators? 2. To indicate the performance of the whole system through the aggregation of indicators that relate to parts of the system? Some countries currently report composite indicators (e.g. Canada, Australia).

25 AHRQ Findings Indicators are positively correlated Positive factor loadings on the 1st factor 1st factor explains 94% of variation Indication for separate diabetes composite (high factor loading on the 2 nd factor)

26

27 HCQI Findings Not all indicators are positively correlated (e.g. angina and diabetic amputations) Factor loading for amputation is negative 1 st factor only explains 31% of variation No clear indication for a 2 nd factor Preliminary HCQI analysis does not provide clear support for use of composite.

28 Correlation Asthma COPD CHF Angina Hyper Diab ST Diab LT Diab UC DiabLEA Asthma COPD CHF Angina Hypertension Diabetes ST Diabets LT Diabetes UC Diabetes LEA Rotated factor loadings (pattern matrix) and unique variances Variable Factor1 Factor2 Uniqueness Asthma COPD CHF Angina Hypertension Diabetes ST Diabetes LT Diabetes UC Diabetes LEA Proportion

29 Key Issues 1. Potential confounding factors 2. Data coverage of the hospital sector 3. Reporting of meaningful composites 4. Indicator instability in small populations 5. Indicator-specific issues: Technical specification of indicators Cross walk of ICD 9 and ICD 10 codes Country variations in coding practice/capacity

30 Indicator Stability Inherent instability in annual indicator values (high random error) associated with low numerator values indicates use of multi- year averaging or aggregation of numerator cases. Particular issue for smaller states (e.g. Iceland and Luxembourg). Use of the 3-year rolling average would facilitate comparison with other countries.

31 Key Issues 1. Potential confounding factors 2. Data coverage of the hospital sector 3. Reporting of meaningful composites 4. Indicator instability in small populations 5. Indicator-specific issues: Technical specification of indicators Cross walk of ICD 9 and ICD 10 codes Country variations in coding practice/capacity

32

33 Asthma Differential diagnosis of asthma/copd. Code coverage issue between ICD 9 and ICD 10: Coding guidelines indicate that ICD equates to ICD 10 J448. Supplementary data indicates that chronic obstructive asthma (4932) accounts for 10-50% of asthma numerator cases (n=4). Plan to remove J448 from COPD and add to Asthma codes to align with ICD 9 code coverage for future data collections.

34 Asthma and COPD Other Specified COPD (ICD 10 J448) Asthma Chronic Obstructive Asthma (ICD ) COPD (Chronic Bronchitis) COPD (Pulmonary Emphysema)

35

36 COPD Alternative calculation suggested by Denmark: Aged 30+ years with J44 as PDX, and J44 as SDX and one of the following PDX: J96 (Respiratory Failure not elsewhere classified) J13-18 (Pneumonia) Reflect common diagnosis of patients with COPD (i.e. PDX = Pneumonia).

37 Potential Issues Use of SDX is potentially problematic. Create inconsistency in the numerator and denominator across indicators impact on ability to aggregate indicators.

38

39 Angina Calculation based on principal diagnosis. Supplementary data was collected during HCQI data collection for to explore variations in coding practices: Cases identified in SDX (any PDX). Cases identified in SDX (PDX of chest pain).

40 PDX and SDX

41 Explanation Secondary diagnosis coding Limited data availability for this analysis: Italy had the lowest number of SDX codes and highest proportion of PDX cases. However, this relationship did not hold for other countries. Other potential factors at play? E.g. Saver et al. (2009) attribute a sharp decline in angina admissions in the 1990 s to more aggressive diagnosis of coronary atherosclerosis.

42 Secondary Diagnosis

43 Discharge Trends Saver et al. in Medical Care 2009:47, pp

44 Angina DX on Admission and Discharge DX Saver et al. in Medical Care 2009:47, pp

45

46 Hypertension Benign hypertension included in ICD 10 code list but not ICD 9 code list (i.e. 4011) Questions over validity clinical grounds for PDX (The Netherlands, NZ).

47

48

49

50

51 Lower Extremity Amputation Inclusion of toe amputation Interpretation in relation to quality of care Performed in hospital and ambulatory service settings and on a day case and inpatient basis. Use of secondary diagnoses Up to 50% cases identified in SDX codes Availability of secondary diagnosis codes? Trauma code exclusions Adequacy of code list

52 Toe Amputation Reflects tertiary prevention Can reflect good quality of care as it prevents further complications (OECD 2004) Failed primary and secondary prevention in primary care system retain in code set? Only multi-day stays reflected in data. If ambulatory care = day cases retain in code set?

53 Secondary Diagnosis

54 Secondary Diagnosis Consider further data collection to locate where cases are identified in SDX codes, similar to PSI in Options: 1. Limit calculation to: countries able to report above a lower threshold number of SDX codes, and/or an upper threshold number of SDX codes 2. Apply adjustment methodology, similar to PSI 3. Limit the calculation to PDX only.

55 Trauma Exclusions Currently includes traumatic amputation of toe, foot and leg (ICD 9 codes ) There may be justification for extending the exclusion code list. The Netherlands suggest inclusion of: Fracture of specified bones of lower limbs (820 to 828) Late effect of muscle & skin injury (905,906) Crushing injury of lower limb & other sites (928, 929) Burns of lower limbs & other sites (945, 946)

56

57 Uncontrolled Diabetes Potentially problematic for ICD 10 countries. 60% (11) countries reporting data in could identify uncontrolled cases: Just<50% by using ICD 9 Just >50% by using ICD 10 markers. Austria, Canada, Finland, Korea, UK, Norway, Poland and Sweden could not identify uncontrolled cases. Canada establishing a marker for 2009+

58 Suggested Approach to Key Issues 1. Continue to examine potential confounding and explanatory factors and report key findings with the indicators in OECD publications. 2. Collect further data on hospital coverage from all countries in and review approach. 3. Further analysis required before proceeding with the use of composite indicators. 4. Present 3-year averages for Iceland and Luxembourg in the future to address indicator instability. 5. Consider indicator-specific issues on a case by case basis next slide.

59 Indicator-Specific Issues Indicator Suggested Approach Asthma Revise ICD 10 code list (add J448) Chronic Obstructive Pulmonary Disease For discussion Congestive Heart Failure No action Angina* For discussion Hypertension Revise ICD 9 code list (add 4011) Diabetes : Short term complications No action Diabetes: Long term complications* For discussion Diabetes: Lower -extremity amputation For discussion Diabetes: Uncontrolled* For discussion * Not currently considered for publication in 2009.

60 The Subgroup is invited to: comment on these methodological issues endorse plans for further analysis and refinement of the indicators in preparation for the next round of data collection in