Methodological Issues - PDF Free Download

Methodological Issues Presentation by Ian Brownwood for the meeting of Health Promotion, Prevention and Primary Care Subgroup 22 October, 2009, Paris

The Indicators Asthma* Chronic Obstructive Pulmonary Disease* Congestive Heart Failure* Angina Hypertension* Diabetes Short -term complications* Long -term complications Lower -extremity amputation* Uncontrolled * To be published in the 2009 edition of OECD Health at a Glance

Their Nature Based on data from routine hospital administrative information systems. Conditions are considered amenable or sensitive to care provided in primary care. Lower values are intended to be reflective of effective primary care systems. Does not imply that indicator values of zero can be achieved.

Their Definition AHRQ definitions except: Aged 15+ instead of 18+ years Exclusion of day cases. Mapping between ICD9 and ICD10. Procedure coding Based on PDX codes, except for: Asthma (i.e. cystic fibrosis and anomalies of the respiratory system) Lower-extremity amputation (i.e. trauma) COPD (i.e. qualification of bronchitis)

Validity and Comparability Key issues considered: 1. Potential confounding factors 2. Data coverage of the hospital sector 3. Reporting of meaningful composites 4. Indicator instability in small populations 5. Indicator-specific issues: Technical specification of indicators Cross walk of ICD 9 and ICD 10 codes Country variations in coding practice/capacity

Confounding Factors Potential confounding factors Demand Prevalence Supply Hospital Expenditures, Staff, Beds, Bed days and Admissions Potential explanatory factors GP per 100,000 Preliminary examination during HCQI data collection for 2008-09.

WE DID FIND..

Demand

Supply

Explanatory

BUT THEN WE ALSO FOUND

Demand

Supply

Explanatory

Conclusions Some evidence of relations but not strong or consistent. Data limitations: Invalidated country reported prevalence rates. Small sample size Temporal issues for diabetes long-term complications. Some of the key findings reflected in 2009 edition of Health at a Glance.

Key Issues 1. Potential confounding factors 2. Data coverage of the hospital sector 3. Reporting of meaningful composites 4. Indicator instability in small populations 5. Indicator-specific issues: Technical specification of indicators Cross walk of ICD 9 and ICD 10 codes Country variations in coding practice/capacity

Hospital Coverage Denominator =Population not Admissions (i.e. not same unit of measurement). Sensitive to coverage of hospital system. Not all countries provided the additional data (e.g. UK could be 10% understated). Adjustment of rates may result in slight overestimation in some cases (i.e. 40% of countries indicated they thought the public rate would be higher).

Adjusted Rates

Conclusions Without adjustment potential exists for underestimation of rates in some countries. Incomplete additional data for 2009-10. Unadjusted rates reported in 2009 edition of Health at a Glance. Reconsider application of adjusted rates based on enhanced data in 2010-11.

Composite What is the objective? 1. To explain the variation in the individual indicators through the use of one or more composite indicators? 2. To indicate the performance of the whole system through the aggregation of indicators that relate to parts of the system? Some countries currently report composite indicators (e.g. Canada, Australia).

AHRQ Findings Indicators are positively correlated Positive factor loadings on the 1st factor 1st factor explains 94% of variation Indication for separate diabetes composite (high factor loading on the 2 nd factor)

HCQI Findings Not all indicators are positively correlated (e.g. angina and diabetic amputations) Factor loading for amputation is negative 1 st factor only explains 31% of variation No clear indication for a 2 nd factor Preliminary HCQI analysis does not provide clear support for use of composite.

Correlation -------------+------------------------------------------------------------------------------------ Asthma COPD CHF Angina Hyper Diab ST Diab LT Diab UC DiabLEA Asthma 1.0000 COPD 0.1812 1.0000 CHF 0.1559 0.0152 1.0000 Angina -0.1658 0.4965 0.1733 1.0000 Hypertension 0.0627 0.2765 0.4532 0.3830 1.0000 Diabetes ST 0.5346 0.2123 0.2886-0.2476-0.0903 1.0000 Diabets LT 0.1367 0.1675 0.0892-0.0779 0.5676 0.1674 1.0000 Diabetes UC 0.1403 0.2929 0.1552 0.2508 0.8065 0.0226 0.5757 1.0000 Diabetes LEA 0.0692-0.2891 0.3810-0.3284-0.1404 0.2650 0.2201-0.3302 1.0000 Rotated factor loadings (pattern matrix) and unique variances ------------------------------------------------- Variable Factor1 Factor2 Uniqueness -------------+--------------------+-------------- Asthma 0.1936 0.5950 0.6085 COPD 0.5645-0.1366 0.6627 CHF 0.3596 0.4773 0.6429 Angina 0.5136-0.5182 0.4677 Hypertension 0.9054 0.0088 0.1802 Diabetes ST 0.0829 0.7547 0.4235 Diabetes LT 0.6062 0.3689 0.4965 Diabetes UC 0.8721-0.0298 0.2385 Diabetes LEA -0.2611 0.6924 0.4524 ------------------------------------------------- Proportion 0.3080 0.2284

Indicator Stability Inherent instability in annual indicator values (high random error) associated with low numerator values indicates use of multi- year averaging or aggregation of numerator cases. Particular issue for smaller states (e.g. Iceland and Luxembourg). Use of the 3-year rolling average would facilitate comparison with other countries.

Asthma Differential diagnosis of asthma/copd. Code coverage issue between ICD 9 and ICD 10: Coding guidelines indicate that ICD 9 4932 equates to ICD 10 J448. Supplementary data indicates that chronic obstructive asthma (4932) accounts for 10-50% of asthma numerator cases (n=4). Plan to remove J448 from COPD and add to Asthma codes to align with ICD 9 code coverage for future data collections.

Asthma and COPD Other Specified COPD (ICD 10 J448) Asthma Chronic Obstructive Asthma (ICD 9 4932) COPD (Chronic Bronchitis) COPD (Pulmonary Emphysema)

COPD Alternative calculation suggested by Denmark: Aged 30+ years with J44 as PDX, and J44 as SDX and one of the following PDX: J96 (Respiratory Failure not elsewhere classified) J13-18 (Pneumonia) Reflect common diagnosis of patients with COPD (i.e. PDX = Pneumonia).

Potential Issues Use of SDX is potentially problematic. Create inconsistency in the numerator and denominator across indicators impact on ability to aggregate indicators.

Angina Calculation based on principal diagnosis. Supplementary data was collected during HCQI data collection for 2008-09 to explore variations in coding practices: Cases identified in SDX (any PDX). Cases identified in SDX (PDX of chest pain).

PDX and SDX

Explanation Secondary diagnosis coding Limited data availability for this analysis: Italy had the lowest number of SDX codes and highest proportion of PDX cases. However, this relationship did not hold for other countries. Other potential factors at play? E.g. Saver et al. (2009) attribute a sharp decline in angina admissions in the 1990 s to more aggressive diagnosis of coronary atherosclerosis.

Secondary Diagnosis

Discharge Trends Saver et al. in Medical Care 2009:47, pp.1106-1110

Angina DX on Admission and Discharge DX Saver et al. in Medical Care 2009:47, pp.1106-1110

Hypertension Benign hypertension included in ICD 10 code list but not ICD 9 code list (i.e. 4011) Questions over validity clinical grounds for PDX (The Netherlands, NZ).

Lower Extremity Amputation Inclusion of toe amputation Interpretation in relation to quality of care Performed in hospital and ambulatory service settings and on a day case and inpatient basis. Use of secondary diagnoses Up to 50% cases identified in SDX codes Availability of secondary diagnosis codes? Trauma code exclusions Adequacy of code list

Toe Amputation Reflects tertiary prevention Can reflect good quality of care as it prevents further complications (OECD 2004) Failed primary and secondary prevention in primary care system retain in code set? Only multi-day stays reflected in data. If ambulatory care = day cases retain in code set?

Secondary Diagnosis

Secondary Diagnosis Consider further data collection to locate where cases are identified in SDX codes, similar to PSI in 2008-09. Options: 1. Limit calculation to: countries able to report above a lower threshold number of SDX codes, and/or an upper threshold number of SDX codes 2. Apply adjustment methodology, similar to PSI 3. Limit the calculation to PDX only.

Trauma Exclusions Currently includes traumatic amputation of toe, foot and leg (ICD 9 codes 895-897) There may be justification for extending the exclusion code list. The Netherlands suggest inclusion of: Fracture of specified bones of lower limbs (820 to 828) Late effect of muscle & skin injury (905,906) Crushing injury of lower limb & other sites (928, 929) Burns of lower limbs & other sites (945, 946)

Uncontrolled Diabetes Potentially problematic for ICD 10 countries. 60% (11) countries reporting data in 2008-09 could identify uncontrolled cases: Just<50% by using ICD 9 Just >50% by using ICD 10 markers. Austria, Canada, Finland, Korea, UK, Norway, Poland and Sweden could not identify uncontrolled cases. Canada establishing a marker for 2009+

Suggested Approach to Key Issues 1. Continue to examine potential confounding and explanatory factors and report key findings with the indicators in OECD publications. 2. Collect further data on hospital coverage from all countries in 2010-11 and review approach. 3. Further analysis required before proceeding with the use of composite indicators. 4. Present 3-year averages for Iceland and Luxembourg in the future to address indicator instability. 5. Consider indicator-specific issues on a case by case basis next slide.

Indicator-Specific Issues Indicator Suggested Approach Asthma Revise ICD 10 code list (add J448) Chronic Obstructive Pulmonary Disease For discussion Congestive Heart Failure No action Angina* For discussion Hypertension Revise ICD 9 code list (add 4011) Diabetes : Short term complications No action Diabetes: Long term complications* For discussion Diabetes: Lower -extremity amputation For discussion Diabetes: Uncontrolled* For discussion * Not currently considered for publication in 2009.

The Subgroup is invited to: comment on these methodological issues endorse plans for further analysis and refinement of the indicators in preparation for the next round of data collection in 2010-11.