Statistical Considerations: Study Designs and Challenges in the Development and Validation of Cancer Biomarkers

Transcription

1 MD-TIP Workshop at UVA Campus, 2011 Statistical Considerations: Study Designs and Challenges in the Development and Validation of Cancer Biomarkers Meijuan Li, PhD Acting Team Leader Diagnostic Devices Branch Division of Biostatistics, OSB/CDRH/FDA Silver Spring MD

2 Evaluation of Cancer Biomarkers FDA regulation and guidance on diagnostic devices for obtaining cancer biomarker measurements Intended uses of biomarkers (e.g., diagnosis, screening, monitoring, risk assessment, treatment selection) Study designs and methods for evaluating biomarkers according to their intended use Common statistical review issues arising from deficiencies in study design and study conduct Imputation of missing biomarker results (intent-todiagnose analysis) Bridging from a clinical trial assay to a market ready test

3 Intended Uses of Biomarkers Diagnosis, the identification of the presence of cancer Screening, enabling intervention at an earlier and potentially more curable stage than under usual clinical diagnostic conditions Monitoring, monitoring of cancer response during therapy, with potential for adjusting level of intervention (e.g. drug dose) Risk prediction, leading to preventive interventions for those at sufficient risk Treatment selection, predicts safety, efficacy of a specific therapy, thereby providing guidance in selecting it for patients or tailoring its dose.

4 Study Design Training Validation Description Exploratory Confirmatory Goal Identify markers performance for further study Unbiased estimation of marker performance Sample Size Small Moderate to large Subjects Design Representative of IU population Representative of IU population Avoid confounding with technical variations; align all statistical goals with clinical objectives based on IU/IFU; align the statistical design with study goals and the study populations

5 Analytical Validation of Cancer Biomarkers Precision (repeatability, reproducibility: closeness of repeated results, e.g. repeatability standard deviation) Sensitivity, limit of detection (limit of quantification) Specificity (interference, cross-reactivity) Sample type / matrix Performance around the cut-off Carryover, cross-hybridization, contamination Linearity

6 Clinical Validation: Performance Measures for Qualitative Biomarkers Sensitivity and specificity Likelihood ratio of positive test and likelihood ratio of negative test Positive predictive value and negative predictive value and prevalence Positive and negative percent agreement Receiver Operating Characteristic (ROC) plots and the area under the ROC (AUC) for ordinal or quantitative test with a cutoff The utility function

7 Clinical Validation: Performance Measures for Quantitative Biomarkers Trueness (closeness to correct result, on average, e.g. estimated bias with respect to a reference result) Slope and intercept from a linear regression (many different kinds), scatter plots Bias (mean difference) between the new biomarker test and the reference method 95% limits of agreement, Bland-Altman difference plots

8 A Biomarker Test Better than Chance Test Cancer Absent Present Total Total True positive rate (sensitivity) = 81% (42/52) False positive rate (1 specificity) = 24% (20/85) Positive predictive value = 68% (42/62) Negative predictive value = 87% (65/75 65/75)

9 Agendia Mammaprint Gene Signature for Time to Breast Cancer (N=302) Predictive value Years of Follow-Up Risk Group Low High ( ) 0.22 ( ) 0.10 ( ) 0.29 ( )

10 Proportion alive at 10 years Added Value Over Clinical Risk Groups Gene Clinical Signature N Survival Probability* Low Risk Low Risk (0.74 to 0.95) Low Risk High Risk (0.45 to 0.84) High Risk Low Risk (0.77 to 0.95) High Risk High Risk (0.61 to 0.76) * Buyse et al JNCI 2006

11 Challenges: Cancer Biomarker Performance Studies They are observational study; patients are not randomized; there are potential confounding factors Many sources of bias can be introduced causing a biased or spurious association Many cancer biomarkers are not a specific cancer marker, and its levels in a patient are affected by many other factors

12 Sources of Bias in Biomarker Performance Studies Selection bias convenience sampling of available specimens Spectrum bias advanced stage of disease vs. healthy patients enrichment with cases outside of IU population Verification bias disease status not verified in all subjects by the reference standard Imperfect Reference Standard Bias

13 Sources of Bias in Biomarker Performance Studies Ordering bias order in which results are taken by test, comparator, and reference standard is not randomized order in which disease and non-disease subjects are tested is not randomized. for predictive tests, test result is taken AFTER onset of target condition! Missing biomarker results Test interpretation, integrity, and context bias Device users / operators not masked to true disease status. Access to other clinical information not consistent with clinical practice.