Use of Archived Tissues in the Development and Validation of Prognostic & Predictive Biomarkers

Transcription

1 Use of Archived Tissues in the Development and Validation of Prognostic & Predictive Biomarkers Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute

2 Different Kinds of Biomarkers Prognostic biomarkers Measured before treatment to indicate long-term outcome for patients untreated or receiving standard treatment May reflect both disease aggressiveness and effect of standard treatment Used to determine who needs more intensive treatment Predictive biomarkers Measured before treatment to identify who will benefit from a particular treatment Other Early detection, pharmacodynamic, surrogate endpoint

3 Prognostic Biomarkers in Node Negative ER Positive Breast Cancer Intended use is to identify patients who are likely to be cured by surgery/radiotherapy and hormonal therapy and therefore are unlikely to benefit from adjuvant chemotherapy Oncotype Dx recurrence score

4 Prognostic Factors in Oncology Most prognostic factors are not used because they are not therapeutically relevant; i.e. have no intended use Most prognostic factor studies are not conducted with an intended use clearly in mind They use a convenience sample of patients for whom tissue is available. Generally the patients are either inappropriate or too heterogeneous to support therapeutically relevant conclusions There is rarely a validation study separate from the developmental study that addresses medical utility

5

6

7

8 Key steps in development of biomarkers Identify specific intended use of the biomarker Perform developmental study using samples appropriate for the intended use Develop an analytically validated assay Perform a focused validation study that addresses the medical utility of a specific biomarker or biomarker score/classifier

9 Before I forget It s not about false discovery; it s about prediction accuracy It s not about stability of genes in the model, it s about prediction accuracy It s actually not about prediction accuracy, it s about actionability; i.e. enabling better treatment decisions to be made

10 Validation = Fitness for Intended Use

11 Analytical validation Types of Validation Accuracy in measurement of analyte Robustness and reproducibility Clinical validation Correlation of score/classifier with clinical state or outcome Medical utility Actionable Use results for patient benefit

12 Clinical Utility Benefits patient by improving treatment decisions Depends on context of use of the biomarker Treatment options and practice guidelines Other prognostic factors

13 Developmental vs Validation Studies Developmental studies screen candidate markers to develop biomarker scores or classifiers Select and combine Train classifiers, optimize tuning parameters, set cut-off values for classification Developmental studies often use cross-validation or split-sample validation to provide a preliminary estimate of the accuracy of the marker/classifier for predicting a clinical outcome Developmental studies often focus on clinicalvalidation (i.e. prediction accuracy), not medical utility

14 Developmental vs Validation Studies Validation studies use a previously developed, completely specified biomarker classifier Validation studies should use analytically validated assays and focus on medical utility, not predictive accuracy

15 Optimal Design for Validation of Predictive Biomarker Develop Predictor of Response to New Rx Predicted Responsive To New Rx Predicted Nonresponsive to New Rx New RX Control New RX Control

16 Marker Strategy Design

17 Marker Strategy Design Generally very inefficient because some (many) patients in both randomization groups receive the same treatment Often poorly informative Not measuring marker in control group means that merits of complex marker treatment strategies cannot be dissected Generally provides no information not provided by the stratification design

18 The Optimal Design for a Validation Study Generally Does Not Involve Randomization Between Testing and Non-Testing It involves either observing patients on low intensity treatment and measuring how outcomes correlate with biomarker values or Randomizing patients between treatments and observing how treatment effects correlate with biomarker values

19 In some cases a trial with optimal structure for evaluating a new biomarker will have been previously performed and will have pre-treatment tumor specimens archived Under certain conditions, a focused analysis based on specimens from the previously conducted clinical trial can provide highly reliable evidence for the medical utility of a prognostic or predictive biomaker In some cases, it may be the only way of obtaining high level evidence

20 Prospective-Retrospective Study

21 Guidelines Proposed by Simon, Paik, Hayes Prospective-retrospective design 1. Adequate archived tissue from an appropriately designed phase III clinical trial must be available on a sufficiently large number of patients that the appropriate biomarker analyses have adequate statistical power and that the patients included in the evaluation are clearly representative of the patients in the trial. 2. The test should be analytically and pre-analytically validated for use with archived tissue. Testing should be perform blinded to the clinical data. 3. The analysis plan for the biomarker evaluation should be completely specified in writing prior to the performance of the biomarker assays on archived tissue and should be focused on evaluation of a single completely defined classifier. 4. The results should be validated using specimens from a similar, but separate study involving archived tissues.

22 Conclusions of Simon, Paik, Hayes Claims of medical utility for prognostic and predictive biomarkers based on analysis of archived tissues can have either a high or low level of evidence depending on several key factors. These factors include the analytical validation of the assay, the nature of the study from which the specimens were archived, the number and condition of the specimens, and the development prior to assaying tissue of a focused written plan for analysis of a completely specified biomarker classifier. Studies using archived tissues from prospective clinical trials, when conducted under ideal conditions and independently confirmed can provide the highest level of evidence. Traditional analyses of prognostic or predictive factors, using non analytically validated assays on a convenience sample of tissues and conducted in an exploratory and unfocused manner provide a very low level of evidence for clinical utility.

23

24 Primary Question Recommendations Developmental studies should be focused on an intended use. Clinical samples should be selected appropriate for that use and the analysis focused on that use

25 Primary Question Recommendations A separate focused validation study of the medical utility of a pre-defined classifier/score should be conducted For predictive markers, the study should be based on a randomized clinical trial For prognostic markers, a randomized trial is not always needed Prospective-retrospective designs can provide level I evidence under appropriate conditions

26 Selected Secondary Questions 16. When is a test ready for a clinical trial involving treatment choice For predictive markers, the marker strategy design is rarely necessary. The stratification design is more informative, more efficient and does not require using the test to determine treatment For prognostic markers, using a test to withhold therapy for patients predicted to be at low risk of recurrence should generally only be done after a carefully performed prospective-retrospective analysis for that intended use

27 17. Why do signatures not progress successfully through clinical trials Many signatures have clinical validity but no medical utility. Many are predictive of outcome but have no important intended use The most common intended use of a prognostic marker is withholding chemotherapy Few developmental studies identify a subset with such good prognosis that a clinical trial is warranted Such clinical trials are difficult to conduct and expensive.

28 1. Key steps for prognostic markers Developmental study with archived samples and analysis focused on intended use Often patients with stage I disease who did not receive chemotherapy Split-sample or cross-validation for assessing prediction accuracy Validation study Prospective trial in which treatment is withheld from patients predicted low-risk or Prospective-retrospective validation using archived specimens from clinical trial in which treatment was withheld; outcomes are correlated with test result

29 1. Key steps for predictive markers Developmental study with archived samples and analysis focused on predicting response or treatment benefit Phase II trial or prior phase III trial Split-sample or cross-validation for assessing prediction accuracy Prospective randomized phase 3 trial comparing new treatment to control measuring marker at time of randomization but not used for treatment assignment Prospective-retrospective analysis of previous rct s

30 7. Omics data differs among labs That s what analytic validation is all about Proper analytic validation requires extensive standardization and industrial style quality control. It is time consuming and expensive. Proper analytic validation is often best done by diagnostics companies Validation clinical studies should be conducted using analytically validated assays

31 10. Independent test set Split sample and cross-validation are internal validations. If properly performed, either are suitable for developmental studies. Neither constitute external validation Demonstrating predictive accuracy on an independent test set does not necessarily demonstrate medical utility; i.e. actionability of the test result for altering treatment decisions that benefit the patient.