Analysing clinical datasets: Challenges and pitfalls. Amanda Mocroft University College London

Transcription

1 Analysing clinical datasets: Challenges and pitfalls Amanda Mocroft University College London

2 Introduction Clinical cohort data one of most abundant sources of data Presents a real-life picture of wide range of outcomes Cost effective if use routinely available data Analysis and interpretation of analyses can be complex Many different biases to consider

3 Outline of talk What is a clinical cohort? Data quality and management Validity of endpoints in cohort studies Multi-cohort collaborations / data from different sources Common issues causing bias in cohort studies Summary

4 Outline of talk What is a clinical cohort? Data quality and management Validity of endpoints in cohort studies Multi-cohort collaborations / data from different sources Common issues causing bias in cohort studies Summary

5 What is a clinical cohort? Group of persons followed up over calendar time Data collected on range of variables o clinical, laboratory, genetic, socioeconomic, demographic etc Regular or interval follow-up Information collected from patient records, forms, databases, questionnaires etc Computerised data allows easy analysis

6 Key inclusion criteria Examples Royal Free Hospital CC HIV+ at RFH EuroSIDA HIV+, aged > 16 pre-booked O/P appt at selected clinics Follow-up 3-6 monthly At clinic 3-6 monthly, information collated centrally every 6 months Open cohort Yes Periodically opens for recruitment for a set time Data collection Extracted from patient notes 9-12 months, manually entered, downloaded QA Generalisability Key data checks from statistician Large UK (London) centre of excellence, 80% MSM Extracted from clinic via computerised download or REDCAP All clinical events, random 10% Heterogeneous European clinics, largely university hospitals

7 Outline of talk What is a clinical cohort? Data quality and management Validity of endpoints in cohort studies Multi-cohort collaborations / data from different sources Common issues causing bias in cohort studies Summary

8 Data quality and management Data quality key to good clinical cohort Standardised way of collecting and recording data, eg HICDEP * Merging data sets from different sources o lab and clinical data from different systems o data from different cohorts Completeness of data Quality assurance Documentation *

9 Outline of talk What is a clinical cohort? Data quality and management Validity of endpoints in cohort studies Multi-cohort collaborations / data from different sources Common issues causing bias in cohort studies Summary

10 Validity and variety of endpoints in cohort studies Examples of endpoints Advantages Disadvantages Clinical Laboratory Combined Death, CVD, AIDS CD4 <200, VL < 50, CKD, genotypic resistance Clinical relevance, easily determined, can be linked to data registries Less common, central validation required?, delays in reporting High power, routine data Lab variation, variation in measurement frequency, clinical relevance AIDS or death, CD4<200/AIDS, raised BP/antihypertensives Increases power Weights different components equally

11 Example - death as an outcome Has patient died? Yes No

12 Example - death as an outcome Has patient died? Yes No Registry records Reported by clinic Reported by other Clinic records Registry records LTFU *

13 Example - death as an outcome Has patient died? Yes Registry records Reported by clinic Reported by other No Clinic records Registry records LTFU Cause of death

14 Example - death as an outcome Has patient died? Yes Registry records Reported by clinic Reported by other No Clinic records Registry records LTFU Cause of death CoDe* Adapted CoDe/computer algorithm ICD10 Clinic/cohort specific *

15 Outline of talk What is a clinical cohort? Data quality and management Validity of endpoints in cohort studies Multi-cohort collaborations / data from different sources Common issues causing bias in cohort studies Summary

16 Multi-cohort collaborations / data from different sources Increasingly data comes from different sources o Combining data across cohorts o Within single cohort, data from different sources Advantages o increased power o can consider very specific questions o use data from different specialties Disadvantages o heterogeneity in data quality across cohorts/studies o completeness of data o difficulty merging data from different sources/formats

17 Example - COHERE COHERE : Collaboration of Observational HIV Epidemiological Research in Europe including 245,000 persons from 36 HIV cohorts % cohorts HCV serostatus CD8 lab data Link with death registries Cause of death 22 (61%) yes, 3 (8%) occasionally 25 (69%) yes, 1 (3%) occasionally 6 (17%) yes, 7 (19%) occasionally % of deaths CoDe via D:A:D 1 1 Modified CoDe via ART-CC 2 31 CoDe procedure at cohort level 23 No CoDe procedure applied 36 Unknown CID 2010;50:1387

18 Outline of talk What is a clinical cohort? Data quality and management Validity of endpoints in cohort studies Multi-cohort collaborations / data from different sources Common issues causing bias in cohort studies Summary

19 Common issues causing significant bias in cohort studies Missing data Loss to follow-up Confounding by indication Frequency and variability of laboratory measurements

20 Missing data Can vary widely depending on data collected Beware of systematic differences o CD4 count at presentation o Completeness of information on death/ltfu Excluding persons influences results Consider flow-chart for inclusion of patients for generalisability and interpreting findings

21 Missing data Example D:A:D CKD risk score* 49,717 in D:A:D 36,180 >1 egfr > ,954 included Key exclusions 2,645 <3 egfrs 308 no baseline CD4 / VL 15,172 prior TDF, LPV/r, AZV, AZV/r, other PI/r 101 <3 months FU Incidence CKD 6.2 ( )/1000 PYFU *Mocroft et al, PLoS Med 2015

22 Missing data Example D:A:D risk score* 49,717 in D:A:D 36,180 >1 egfr > ,954 included Key exclusions 2,645 <3 egfrs 308 no baseline CD4 / VL 15,172 prior TDF, LPV/r, AZV, AZV/r, other PI/r 101 <3 months FU Incidence CKD 6.2 ( )/1000 PYFU *Mocroft et al, PLoS Med 2015 Incidence CKD 12.7 ( )/1000 PYFU

23 Missing data Late presentation for HIV in COHERE CD4 in X months after diagnosis % late presenters month 6 months 12 months 1st CD4 Missing CD4=LP Missing CD4=not LP N included in analyses *Mocroft et al, PLoS Med 2013; late presentation CD4<350/mm 3 or AIDS

24 Missing data Late presentation for HIV in COHERE Eastern Europe CD4 in X months after diagnosis % late presenters month 6 months 12 months 1st CD4 Missing CD4=LP Missing CD4=not LP N included in analyses *Mocroft et al, PLoS Med 2013; late presentation CD4<350/mm 3 or AIDS

25 Missing data Can vary widely depending on data collected Beware of systematic differences Excluding persons influences results Consider flow-chart for inclusion of patients for generalisability and interpreting findings Solutions include multiple imputation, complete case analysis, categories for missing variables, sensitivity analyses

26 Missing data Can vary widely depending on data collected Beware of systematic differences o CD4 count at presentation o AIM FOR BETTER Completeness of information on death/ltfu DATA Consider flow-chart for inclusion of patients for generalisability and interpreting findings Solutions include multiple imputation, complete case analysis, categories for missing variables, sensitivity analyses

27 Loss to follow-up Varies widely across cohorts Wide range of definitions used Can significantly impact outcomes Study Cohort Definition LTFU Rates, notes Mocroft et al, HIV Med 2008 Lanoy et al, HIV Med 2009 Mberi et al, BMC Health Serv Res 2015 Fox, JAIDS 2015 EuroSIDA No CD4/VL in 12 months after median last visit date 3.7/100 PYFU FHDH No FU visit in % LTFU; 21.4% died, 28.1% RtC (churn) South Africa Sub Saharan Africa; meta analysis No contact >180 days after planned visit Varied 10.3% first year of cart, 40.5%>1 st year 65-70% retained on Tx, 43% not retained died

28 Mortality in EuroSIDA after CD4<350/mm 3 Impact on mortality if those LTFU have died (1) % died Actual data 3265 deaths incidence 3.6 ( )/100 PYFU Months of follow-up LTFU defined as not died and no visit after 1/8/2013 (median date last visit)

29 Mortality in EuroSIDA after CD4<350/mm 3 Impact on mortality if those LTFU have died (2) Actual data : 3.6 ( )/100 PYFU 25 % died Random 10% LTFU dead 3577 deaths; incidence 3.9 ( )/100 PYFU Months of follow-up LTFU defined as not died and no visit after 1/8/2013 (median date last visit)

30 Mortality in EuroSIDA after CD4<350/mm 3 Impact on mortality if those LTFU have died (3) % died Actual data : 3.6 ( )/100 PYFU Random 10% LTFU dead 3.9 ( )/100 PYFU Random 50% IDUs/off ART/CD4<50 died, 10% others 4341 deaths; incidence 4.8 ( )/100 PYFU Months of follow-up LTFU defined as not died and no visit after 1/8/2013 (median date last visit)

31 Confounding by indication Randomised controlled trials gold standard for antiretroviral comparisons Confounding by indication significant bias in clinical cohorts One group of people systematically chosen to receive one antiretroviral, based on characteristics o Calendar time o HIV exposure group o Underlying disease o Risk of adverse events

32 Confounding by indication D:A:D : Use of abacavir and CKD Crude incidence CKD (/1000 PYFY) and 95% CI ,5 Cumulative time on ABC >6 Years CKD CKD : Confirmed (>3 months apart) egfr < 60 (CG)

33 Crude incidence CKD (/1000 PYFY) and 95% CI ,5 Confounding by indication D:A:D : Use of abacavir and CKD Cumulative time on ABC > >6 Years Cumulative time on ABC Exc. never started and off ARV CKD CKD : Confirmed (>3 months apart) egfr < 60 (CG)

34 Crude incidence CKD (/1000 PYFY) and 95% CI ,5 Confounding by indication D:A:D : Use of abacavir and CKD Cumulative time on ABC People switching away from eg TDF as egfr declines; takes some months for egfr to recover; ABC not nephrotoxic but commonly used in those with low or decreasing egfr > >6 Years Cumulative time on ABC Exc. never started and off ARV CKD CKD : Confirmed (>3 months apart) egfr < 60 (CG)

35 Frequency/variability in lab data Data measured only in selected group, eg ALT/AST and liver disease ISSUE ALT/AST not routinely measured in all Most likely to be measured in those with clinical symptoms, HBV/HCV coinfection, starting certain ARVs SOLUTIONS Change inclusion criterion, eg those with >1 ALT/year Sensitivity analyses, eg in coinfected, on/off cart, measurement within 1 month of CD4/VL

36 Frequency/variability in lab data Variability in laboratory data : CKD in D:A:D Decreasing effect of lab variation and measurement frequency Inclusion Endpoint N CKD (%) Incidence /1000 PYFU (95% CI) Baseline egfr > 60 1 st egfr < (11.2) 17.9 ( ) Baseline egfr > 60 2 consecutive egfr < 60 Baseline egfr > 60 Confirmed baseline egfr > 60 Confirmed (>3 months apart) egfr < 60 Confirmed (>3 months apart) egfr < (5.4) 8.1 ( ) (4.8) 7.1 ( ) (4.3) 6.7 ( )

37 Outline of talk What is a clinical cohort? Data quality and management Validity of endpoints in cohort studies Multi-cohort collaborations / data from different sources Common issues causing bias in cohort studies Summary

38 Summary Clinical cohorts provide wealth of valuable evidence Invaluable in HIV research Report data using STROBE 1 guidelines Careful collection of data, analysis and interpretation of data essential for placing results in context More complex statistical methods can help, but will never remove bias Adjust for known confounders if measured Unknown and/or unmeasured confounders 1 BMJ Oct 20;335(7624):806-8