From single studies to an EBM based assessment some central issues

Size: px
Start display at page:

Download "From single studies to an EBM based assessment some central issues"

Transcription

1 From single studies to an EBM based assessment some central issues Doug Altman Centre for Statistics in Medicine, Oxford, UK

2 Prognosis Prognosis commonly relates to the probability or risk of an individual developing a particular state of health (an outcome ) over a specific time period, based on his/her clinical and non-clinical profile 2

3 Predicted vs observed survival in 468 terminally ill hospice patients Christakis & Lamont BMJ

4 What sorts of prognostic research? Natural history of disease Investigation of prognostic importance of a particular marker Investigation of whether a marker predicts which patients are more likely to respond to a specific treatment ( predictive marker ) Simultaneous investigation of several factors to try to determine which are of prognostic importance Development of a prognostic model using clinical information and laboratory measurements Development of a signature using high dimensional data, e.g. DNA microarray 4

5 What sorts of prognostic research? Natural history of disease Investigation of prognostic importance of a particular marker Investigation of whether a marker predicts which patients are more likely to respond to a specific treatment ( predictive marker ) Simultaneous investigation of several factors to try to determine which are of prognostic importance Development of a prognostic model using clinical information and laboratory measurements Development of a signature using high dimensional data, e.g. DNA microarray ,000 5

6 What sorts of prognostic research? Natural history of disease Investigation of prognostic importance of a particular marker Investigation of whether a marker predicts which patients are more likely to respond to a specific treatment ( predictive marker ) Simultaneous investigation of several factors to try to determine which are of prognostic importance Development of a prognostic model using clinical information and laboratory measurements Development of a signature using high dimensional data, e.g. DNA microarray ,000 Single and multiple studies 6

7 Biomarkers (prognostic markers) in cancer Many markers have some prognostic potential As yet few markers have been demonstrated to be clinically useful oestrogen receptor (ER) status for breast cancer c-erbb-2 (Her-2) for breast cancer prostate specific antigen (PSA) for prostate cancer Fewer still have been shown to predict who will benefit from a particular treatment ER and tamoxifen for breast cancer Her-2 and herceptin for breast cancer 7

8 Prognostic importance of a single specific variable If this was easy by now we would know the prognostic importance of numerous markers that have been investigated for many cancers and other diseases We don t! 8

9 Frequent methodological shortcomings of published prognostic studies Design and data Poorly defined or unrepresentative cohort Imprecise measurements Unknown quality of tissue samples (when relevant) Missing data Unknown completeness of follow up Some important predictors may be unavailable Sample too small Analysis Unclear which variables have been adjusted for (and how) Data-dependent (biased) analysis e.g. data-derived cutpoint 9

10 Study Design Design should reflect standards used in randomised trials as far as possible Clearly defined inclusion and exclusion criteria Ideally a homogenous group (e.g. newly diagnosed) Measurements should be reproducible and performed blinded to clinical data and outcome Ideally, patient treatment should be either standardized or assigned by randomization Primary and secondary hypotheses should be clearly stated Follow up should be long enough and complete for (nearly) all patients 10

11 In practice Many (most?) studies are retrospective Few are protocol driven Studies are often small with unclear aims Many are exploratory but interpreted as confirmatory Many fail to address a key question: How much does the new factor add to the predictive ability of recognized prognostic factors? 11

12 p53 as a prognostic marker in bladder cancer 168 published studies with over 10,000 patients Interpretation: After 10 years of research, evidence is not sufficient to conclude whether changes in P53 act as markers of outcome in patients with bladder cancer. [Malats et al, Lancet Oncology 2005] 12

13 Publication bias Evidence is accumulating of publication bias in prognostic studies 13

14 TP53 as a prognostic factor for head and neck squamous cell cancer [Kyzas et al, JNCI 2005] Studies Random effects (patients) risk ratio (95% CI) Published and indexed 18 (1364) 1.27 (1.06 to 1.53) Published, not indexed 13 (1028) 1.13 (0.81 to 1.59) Retrieved 11 (996) 0.97 (0.72 to 1.29) 14

15 Bcl-2 Martin et al, BJC

16 What can be done better? Ideally need prospectively designed studies, possibly using high quality tissue banks More standardisation of laboratory methods Better methodological and data quality Replication (external validation) More cautious interpretation Better reporting quality 16

17 Aim of a prognostic model Primary goal should be identification of the model that (best?) predicts outcome for the relevant population (e.g. all stroke patients) needs to predict well enough to be clinically useful Statistical methods primarily focus on producing a model that (best?) fits the data from the available sample of patients A crucial difference! 17

18 Prognostic models Three major steps in multivariable prognostic research: Developing a prognostic model, Validating its performance in new individuals Studying its clinical impact 18

19 Examples Nottingham prognostic index (breast cancer mortality) Index = 0.2 x tumour size (cm) + stage (coded 1 to 3) + grade (coded 1 to 3) MELD (liver transplant) log creatinine log bilirubin log INR (with adjustments for diagnosis) The predicted probability of an event is 1/[1 + exp( risk score)] 19

20 Decisions to make (assuming suitable data are available) Select a set of clinically relevant candidate predictors for possible inclusion in the model Evaluate data quality and decide strategy for missing data Choose a strategy for selecting the important variables in the final model Full model, variable selection? Decide how to model continuous variables Select measure(s) of model performance or predictive accuracy 20

21 Assumptions Sample is representative of all patients for whom the resulting model will be used No important predictors are missing Predictors are measured without error Any missing data are missing at random (explanatory variables or loss to follow up) Effect of each predictor is additive on the modelling scale (no interaction) Effect of continuous predictors is modelled correctly 21

22 How well can we predict? Goal should be prediction for future data not maximal fit to the primary data Should distinguish between failure to predict through poor model and through inherent noise Even if we model the data as well as possible we may still not be able to predict well we may distinguish groups with varying outcome we may not be able to make useful predictions for individuals 22

23 Validating a model A prognostic model is valuable only with evidence that it performs well for patients not used to develop the model preferably using data from elsewhere. Unvalidated models should not be used in clinical practice When validating a prognostic model, calibration and discrimination should be evaluated A model may not perform well in practice because of deficiencies in the methods used to derive the model or because the new sample is too different from the development sample 23

24 Design of a validation study Hierarchy of increasingly stringent strategies: Internal : procedures restricted to a single data set data splitting (random?) bootstrap/cross-validation Temporal : evaluation on a second data set from the same centre(s) internal but prospective External : evaluation on data from one or more other centres, perhaps by different investigators prospective or retrospective 24

25 Predicted and observed mortality by EuroSCORE risk level for Australian patients undergoing CABG EuroSCORE Patients (deaths) Observed mortality (95% CI) Predicted mortality (95% CI) 0-2 (low risk) 1955 (8) 0.41% (0.18 to 0.80) 1.03% (0.99 to 1.06) 3-5 (medium risk) 1996 (17) 0.85% (0.50 to 1.36) 3.90% (3.87 to 3.94) 6+ (high risk) 1641 (87) 5.30% (4.27 to 6.50) 8.52% (8.39 to 8.65) Total 5592 (112) 2.00% (1.65 to 2.40) 4.25% (4.16 to 4.34) 25

26 Application When a prognostic model performs less well in a new population, can modify the model using the new data, rather than developing a complete new model Impact studies quantify the benefit of using a prognostic model on physicians behaviour, patient outcome or cost-effectiveness of care 26

27 The main limitations of microarray prognostic signatures are known: instability of gene lists, overoptimistic performance indicators and inadequate validation. Microarray-based prognostic tests are irremediably moving to the clinics, but their clinical utility might never be formally established. 27

28 Systematic reviews Systematic reviews and (perhaps) meta-analysis are preferred to get the most reliable evidence Single markers Prognostic models Microarray studies 28

29 Systematic review of prognostic models in breast cancer 61 published models Poor methods and reporting were common Apart from one very small study, only the Nottingham index had been externally validated by independent researchers Williams et al, HTA

30 Review of multivariate analyses of survival from melanoma [Volmer 1989] In spite of 54 studies using multivariate techniques, there remain uncertainties about which prognostic factors to use in melanoma and how well we can predict the course of this disease. To some degree we must blame the methods of these studies... They seldom published the coefficients of their models so that others could validate the results, and they almost never validated their models with their own test data. 30

31 PLoS Med

32 Final comments For all types of prognostic study, major problems arise from poor quality studies There is (as yet) no consensus on many aspects of how to design and analyse single or multiple prognostic studies Risk prediction scores are easy to produce, hard to validate, harder still to implement in clinical practice and evidence of impact on decision making or prognosis is nearly always lacking Unreasonable expectations of prognostic models are common 32

33 33

Introduction to REMARK: Reporting tumour marker prognostic studies

Introduction to REMARK: Reporting tumour marker prognostic studies Introduction to REMARK: Reporting tumour marker prognostic studies Doug Altman The EQUATOR Network Centre for Statistics in Medicine, Oxford, UK C 1 S M Reliable research Reliability is determined by proper

More information

Summary of main challenges and future directions

Summary of main challenges and future directions Summary of main challenges and future directions Martin Schumacher Institute of Medical Biometry and Medical Informatics, University Medical Center, Freiburg Workshop October 2008 - F1 Outline Some historical

More information

The importance of good reporting of medical research. Doug Altman. Centre for Statistics in Medicine University of Oxford

The importance of good reporting of medical research. Doug Altman. Centre for Statistics in Medicine University of Oxford The importance of good reporting of medical research Doug Altman Centre for Statistics in Medicine University of Oxford 1 Why reporting matters CONSORT and other reporting guidelines EQUATOR Network Other

More information

Use of Archived Tissues in the Development and Validation of Prognostic & Predictive Biomarkers

Use of Archived Tissues in the Development and Validation of Prognostic & Predictive Biomarkers Use of Archived Tissues in the Development and Validation of Prognostic & Predictive Biomarkers Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute http://brb.nci.nih.gov Different

More information

Lecture Outline. Biost 590: Statistical Consulting. Stages of Scientific Studies. Scientific Method

Lecture Outline. Biost 590: Statistical Consulting. Stages of Scientific Studies. Scientific Method Biost 590: Statistical Consulting Statistical Classification of Scientific Studies; Approach to Consulting Lecture Outline Statistical Classification of Scientific Studies Statistical Tasks Approach to

More information

Development, validation and application of risk prediction models

Development, validation and application of risk prediction models Development, validation and application of risk prediction models G. Colditz, E. Liu, M. Olsen, & others (Ying Liu, TA) 3/28/2012 Risk Prediction Models 1 Goals Through examples, class discussion, and

More information

Protocol Development: The Guiding Light of Any Clinical Study

Protocol Development: The Guiding Light of Any Clinical Study Protocol Development: The Guiding Light of Any Clinical Study Susan G. Fisher, Ph.D. Chair, Department of Clinical Sciences 1 Introduction Importance/ relevance/ gaps in knowledge Specific purpose of the

More information

Biost 590: Statistical Consulting

Biost 590: Statistical Consulting Biost 590: Statistical Consulting Statistical Classification of Scientific Questions October 3, 2008 Scott S. Emerson, M.D., Ph.D. Professor of Biostatistics, University of Washington 2000, Scott S. Emerson,

More information

Predictive Assays in Radiation Therapy

Predictive Assays in Radiation Therapy Outline Predictive Assays in Radiation Therapy Radiation Biology Introduction Early predictive assays Recent trends in predictive assays Examples for specific tumors Summary Lecture 4-23-2014 Introduction

More information

Lecture Outline Biost 517 Applied Biostatistics I

Lecture Outline Biost 517 Applied Biostatistics I Lecture Outline Biost 517 Applied Biostatistics I Scott S. Emerson, M.D., Ph.D. Professor of Biostatistics University of Washington Lecture 2: Statistical Classification of Scientific Questions Types of

More information

Roadmap for Developing and Validating Therapeutically Relevant Genomic Classifiers. Richard Simon, J Clin Oncol 23:

Roadmap for Developing and Validating Therapeutically Relevant Genomic Classifiers. Richard Simon, J Clin Oncol 23: Roadmap for Developing and Validating Therapeutically Relevant Genomic Classifiers. Richard Simon, J Clin Oncol 23:7332-7341 Presented by Deming Mi 7/25/2006 Major reasons for few prognostic factors to

More information

CHAMP: CHecklist for the Appraisal of Moderators and Predictors

CHAMP: CHecklist for the Appraisal of Moderators and Predictors CHAMP - Page 1 of 13 CHAMP: CHecklist for the Appraisal of Moderators and Predictors About the checklist In this document, a CHecklist for the Appraisal of Moderators and Predictors (CHAMP) is presented.

More information

Systematic reviews of prediction modeling studies: planning, critical appraisal and data collection

Systematic reviews of prediction modeling studies: planning, critical appraisal and data collection Systematic reviews of prediction modeling studies: planning, critical appraisal and data collection Karel GM Moons, Lotty Hooft, Hans Reitsma, Thomas Debray Dutch Cochrane Center Julius Center for Health

More information

GLOSSARY OF GENERAL TERMS

GLOSSARY OF GENERAL TERMS GLOSSARY OF GENERAL TERMS Absolute risk reduction Absolute risk reduction (ARR) is the difference between the event rate in the control group (CER) and the event rate in the treated group (EER). ARR =

More information

CONSORT 2010 checklist of information to include when reporting a randomised trial*

CONSORT 2010 checklist of information to include when reporting a randomised trial* CONSORT 2010 checklist of information to include when reporting a randomised trial* Section/Topic Title and abstract Introduction Background and objectives Item No Checklist item 1a Identification as a

More information

Prognostic studies and the need for guidance Peggy Sekula, on behalf of TG5 of STRATOS initiative

Prognostic studies and the need for guidance Peggy Sekula, on behalf of TG5 of STRATOS initiative Prognostic studies and the need for guidance Peggy Sekula, on behalf of TG5 of STRATOS initiative Institute of Genetic Epidemiology Medical Center University of Freiburg, Germany STRATOS TG5: Study Design

More information

Role of evidence from observational studies in the process of health care decision making

Role of evidence from observational studies in the process of health care decision making Role of evidence from observational studies in the process of health care decision making Jan van der Meulen Health Services Research Unit London School of Hygiene and Tropical Medicine Clinical Effectiveness

More information

Systematic reviews of prognostic studies: a meta-analytical approach

Systematic reviews of prognostic studies: a meta-analytical approach Systematic reviews of prognostic studies: a meta-analytical approach Thomas PA Debray, Karel GM Moons for the Cochrane Prognosis Review Methods Group (Co-convenors: Doug Altman, Katrina Williams, Jill

More information

STUDY DESIGNS WHICH ONE IS BEST?

STUDY DESIGNS WHICH ONE IS BEST? STUDY DESIGNS WHICH ONE IS BEST? David Nunan, PhD Research Fellow and Tutor Nuffield Department of Primary Care Health Sciences and Oxford Centre for Evidence Based Medicine University of Oxford Exercise

More information

Overview of Multivariable Prediction Modelling. Methodological Conduct & Reporting: Introducing TRIPOD guidelines

Overview of Multivariable Prediction Modelling. Methodological Conduct & Reporting: Introducing TRIPOD guidelines Overview of Multivariable Prediction Modelling Methodological Conduct & Reporting: Introducing TRIPOD guidelines Gary Collins Centre for Statistics in Medicine www.csm-oxford.org.uk University of Oxford

More information

Prognostic markers in cancer: the evolution of evidence from single studies to meta-analysis, and beyond

Prognostic markers in cancer: the evolution of evidence from single studies to meta-analysis, and beyond British Journal of Cancer (2009) 100, 1219 1229 All rights reserved 0007 0920/09 $32.00 www.bjcancer.com Review from single studies to meta-analysis, and beyond RD Riley*,1, W Sauerbrei 2 and DG Altman

More information

Systematic reviews of prognostic studies 3 meta-analytical approaches in systematic reviews of prognostic studies

Systematic reviews of prognostic studies 3 meta-analytical approaches in systematic reviews of prognostic studies Systematic reviews of prognostic studies 3 meta-analytical approaches in systematic reviews of prognostic studies Thomas PA Debray, Karel GM Moons for the Cochrane Prognosis Review Methods Group Conflict

More information

Janet E. Dancey NCIC CTG NEW INVESTIGATOR CLINICAL TRIALS COURSE. August 9-12, 2011 Donald Gordon Centre, Queen s University, Kingston, Ontario

Janet E. Dancey NCIC CTG NEW INVESTIGATOR CLINICAL TRIALS COURSE. August 9-12, 2011 Donald Gordon Centre, Queen s University, Kingston, Ontario Janet E. Dancey NCIC CTG NEW INVESTIGATOR CLINICAL TRIALS COURSE August 9-12, 2011 Donald Gordon Centre, Queen s University, Kingston, Ontario Session: Correlative Studies in Phase III Trials Title: Design

More information

Selective reporting, quality of reporting and statistical significance chasing in prognostic marker studies

Selective reporting, quality of reporting and statistical significance chasing in prognostic marker studies Selective reporting, quality of reporting and statistical significance chasing in prognostic marker studies Panayiotis A. Kyzas 1,2 1: Research Fellow, Department of Hygiene and Epidemiology, University

More information

doi: /

doi: / doi: 0./00 Usefulness of Palliative prognostic Index for advanced cancer patient in home care setting Journal: Manuscript ID: Draft Manuscript Type: Medical Manuscripts Keyword: Advanced Cancer patient,

More information

Lecture 5 Conducting Interviews and Focus Groups

Lecture 5 Conducting Interviews and Focus Groups Lecture 5 Conducting Interviews and Focus Groups Talking to participants enables in-depth information about the experiences of health and illness; and of factors that influence health and illness behaviour

More information

Screening for novel oncology biomarker panels using both DNA and protein microarrays. John Anson, PhD VP Biomarker Discovery

Screening for novel oncology biomarker panels using both DNA and protein microarrays. John Anson, PhD VP Biomarker Discovery Screening for novel oncology biomarker panels using both DNA and protein microarrays John Anson, PhD VP Biomarker Discovery Outline of presentation Introduction to OGT and our approach to biomarker studies

More information

Systematic Reviews. Simon Gates 8 March 2007

Systematic Reviews. Simon Gates 8 March 2007 Systematic Reviews Simon Gates 8 March 2007 Contents Reviewing of research Why we need reviews Traditional narrative reviews Systematic reviews Components of systematic reviews Conclusions Key reference

More information

Summary. 20 May 2014 EMA/CHMP/SAWP/298348/2014 Procedure No.: EMEA/H/SAB/037/1/Q/2013/SME Product Development Scientific Support Department

Summary. 20 May 2014 EMA/CHMP/SAWP/298348/2014 Procedure No.: EMEA/H/SAB/037/1/Q/2013/SME Product Development Scientific Support Department 20 May 2014 EMA/CHMP/SAWP/298348/2014 Procedure No.: EMEA/H/SAB/037/1/Q/2013/SME Product Development Scientific Support Department evaluating patients with Autosomal Dominant Polycystic Kidney Disease

More information

EVIDENCE AND RECOMMENDATION GRADING IN GUIDELINES. A short history. Cluzeau Senior Advisor NICE International. G-I-N, Lisbon 2 November 2009

EVIDENCE AND RECOMMENDATION GRADING IN GUIDELINES. A short history. Cluzeau Senior Advisor NICE International. G-I-N, Lisbon 2 November 2009 EVIDENCE AND RECOMMENDATION GRADING IN GUIDELINES A short history Françoise Cluzeau Senior Advisor NICE International G-I-N, Lisbon 2 November 2009 Long ago.. Before grading Evidence? Plato (3 rd Century

More information

MODEL SELECTION STRATEGIES. Tony Panzarella

MODEL SELECTION STRATEGIES. Tony Panzarella MODEL SELECTION STRATEGIES Tony Panzarella Lab Course March 20, 2014 2 Preamble Although focus will be on time-to-event data the same principles apply to other outcome data Lab Course March 20, 2014 3

More information

Screening for Disease

Screening for Disease Screening for Disease An Ounce of Prevention is Worth a Pound of Cure. Actually, an ounce of prevention is better than a pound of cure, but if prevention hasn t been effective, perhaps early identification

More information

Problem solving therapy

Problem solving therapy Introduction People with severe mental illnesses such as schizophrenia may show impairments in problem-solving ability. Remediation interventions such as problem solving skills training can help people

More information

Statistical reports Regression, 2010

Statistical reports Regression, 2010 Statistical reports Regression, 2010 Niels Richard Hansen June 10, 2010 This document gives some guidelines on how to write a report on a statistical analysis. The document is organized into sections that

More information

Good Old clinical markers have similar power in breast cancer prognosis as microarray gene expression profilers q

Good Old clinical markers have similar power in breast cancer prognosis as microarray gene expression profilers q European Journal of Cancer 40 (2004) 1837 1841 European Journal of Cancer www.ejconline.com Good Old clinical markers have similar power in breast cancer prognosis as microarray gene expression profilers

More information

Recent developments for combining evidence within evidence streams: bias-adjusted meta-analysis

Recent developments for combining evidence within evidence streams: bias-adjusted meta-analysis EFSA/EBTC Colloquium, 25 October 2017 Recent developments for combining evidence within evidence streams: bias-adjusted meta-analysis Julian Higgins University of Bristol 1 Introduction to concepts Standard

More information

Are the likely benefits worth the potential harms and costs? From McMaster EBCP Workshop/Duke University Medical Center

Are the likely benefits worth the potential harms and costs? From McMaster EBCP Workshop/Duke University Medical Center CRITICAL REVIEW FORM FOR THERAPY STUDY Did experimental and control groups begin the study with a similar prognosis? Were patients randomized? Was randomization concealed? Were patients analyzed in the

More information

Meta-analysis of external validation studies

Meta-analysis of external validation studies Meta-analysis of external validation studies Thomas Debray, PhD Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, The Netherlands Cochrane Netherlands, Utrecht, The

More information

Appendix G: Methodology checklist: the QUADAS tool for studies of diagnostic test accuracy 1

Appendix G: Methodology checklist: the QUADAS tool for studies of diagnostic test accuracy 1 Appendix G: Methodology checklist: the QUADAS tool for studies of diagnostic test accuracy 1 Study identification Including author, title, reference, year of publication Guideline topic: Checklist completed

More information

Guideline on the use of minimal residual disease as a clinical endpoint in multiple myeloma studies

Guideline on the use of minimal residual disease as a clinical endpoint in multiple myeloma studies 1 2 3 26 July 2018 EMA/CHMP/459559/2018 Committee for Medicinal Products for Human Use (CHMP) 4 5 6 Guideline on the use of minimal residual disease as a clinical endpoint in multiple Draft Draft agreed

More information

The Royal College of Pathologists Journal article evaluation questions

The Royal College of Pathologists Journal article evaluation questions The Royal College of Pathologists Journal article evaluation questions Previous exam questions Dorrian CA, Toole, BJ, Alvarez-Madrazo S, Kelly A, Connell JMC, Wallace AM. A screening procedure for primary

More information

Traumatic brain injury

Traumatic brain injury Introduction It is well established that traumatic brain injury increases the risk for a wide range of neuropsychiatric disturbances, however there is little consensus on whether it is a risk factor for

More information

Understanding your biomarker: what this marker can do for you

Understanding your biomarker: what this marker can do for you Understanding your biomarker: what this marker can do for you Dr. John Bartlett/Dr. Harriet Feilotter As is your Pathology, so is your Medicine. Sir William Osler. 1849-1919. Disclosure statement John

More information

Why published medical research may not be good for your health

Why published medical research may not be good for your health Why published medical research may not be good for your health Doug Altman EQUATOR Network and Centre for Statistics in Medicine University of Oxford 2 Reliable evidence Clinical practice and public health

More information

Epidemiology: Overview of Key Concepts and Study Design. Polly Marchbanks

Epidemiology: Overview of Key Concepts and Study Design. Polly Marchbanks Epidemiology: Overview of Key Concepts and Study Design Polly Marchbanks Lecture Outline (1) Key epidemiologic concepts - Definition - What epi is not - What epi is - Process of epi research Lecture Outline

More information

C. Data & methodology: validity of approach, quality of data, quality of presentation Excellent.

C. Data & methodology: validity of approach, quality of data, quality of presentation Excellent. Reviewers' comments: Reviewer #1 (Immunotherapy expert) (Remarks to the Author): A. Summary of the key results This study elegantly demonstrates the sequential effects of bevacizumab followed by bevacizumab

More information

Cochrane Pregnancy and Childbirth Group Methodological Guidelines

Cochrane Pregnancy and Childbirth Group Methodological Guidelines Cochrane Pregnancy and Childbirth Group Methodological Guidelines [Prepared by Simon Gates: July 2009, updated July 2012] These guidelines are intended to aid quality and consistency across the reviews

More information

Making a psychometric. Dr Benjamin Cowan- Lecture 9

Making a psychometric. Dr Benjamin Cowan- Lecture 9 Making a psychometric Dr Benjamin Cowan- Lecture 9 What this lecture will cover What is a questionnaire? Development of questionnaires Item development Scale options Scale reliability & validity Factor

More information

Evolution of Pathology

Evolution of Pathology 1 Traditional pathology Molecular pathology 2 Evolution of Pathology Gross Pathology Cellular Pathology Morphologic Pathology Molecular/Predictive Pathology Antonio Benivieni (1443-1502): First autopsy

More information

Assessment of performance and decision curve analysis

Assessment of performance and decision curve analysis Assessment of performance and decision curve analysis Ewout Steyerberg, Andrew Vickers Dept of Public Health, Erasmus MC, Rotterdam, the Netherlands Dept of Epidemiology and Biostatistics, Memorial Sloan-Kettering

More information

95% 2.5% 2.5% +2SD 95% of data will 95% be within of data will 1.96 be within standard deviations 1.96 of sample mean

95% 2.5% 2.5% +2SD 95% of data will 95% be within of data will 1.96 be within standard deviations 1.96 of sample mean Efficient Clinical Trials John H. Powers, MD Senior Medical Scientist SAIC in support of Clinical Collaborative Research Branch NIAID/NIH Introduction Definitions what is a small clinical trial? Review

More information

Garbage in - garbage out? Impact of poor reporting on the development of systematic reviews

Garbage in - garbage out? Impact of poor reporting on the development of systematic reviews Garbage in - garbage out? Impact of poor reporting on the development of systematic reviews ACT now: EQUATOR Scientific Symposium Freiburg, 11 October 2012 Erik von Elm, MD MSc FMH Cochrane Switzerland

More information

Results. NeuRA Motor dysfunction April 2016

Results. NeuRA Motor dysfunction April 2016 Introduction Subtle deviations in various developmental trajectories during childhood and adolescence may foreshadow the later development of schizophrenia. Studies exploring these deviations (antecedents)

More information

Breast cancer classification: beyond the intrinsic molecular subtypes

Breast cancer classification: beyond the intrinsic molecular subtypes Breast cancer classification: beyond the intrinsic molecular subtypes Britta Weigelt, PhD Signal Transduction Laboratory CRUK London Research Institute Summary Breast cancer heterogeneity Molecular classification

More information

Critical Review Form Clinical Decision Analysis

Critical Review Form Clinical Decision Analysis Critical Review Form Clinical Decision Analysis An Interdisciplinary Initiative to Reduce Radiation Exposure: Evaluation of Appendicitis in a Pediatric Emergency Department with Clinical Assessment Supported

More information

Empirical evidence on sources of bias in randomised controlled trials: methods of and results from the BRANDO study

Empirical evidence on sources of bias in randomised controlled trials: methods of and results from the BRANDO study Empirical evidence on sources of bias in randomised controlled trials: methods of and results from the BRANDO study Jonathan Sterne, University of Bristol, UK Acknowledgements: Tony Ades, Bodil Als-Nielsen,

More information

Lecture Outline Biost 517 Applied Biostatistics I. Statistical Goals of Studies Role of Statistical Inference

Lecture Outline Biost 517 Applied Biostatistics I. Statistical Goals of Studies Role of Statistical Inference Lecture Outline Biost 517 Applied Biostatistics I Scott S. Emerson, M.D., Ph.D. Professor of Biostatistics University of Washington Statistical Inference Role of Statistical Inference Hierarchy of Experimental

More information

Further data analysis topics

Further data analysis topics Further data analysis topics Jonathan Cook Centre for Statistics in Medicine, NDORMS, University of Oxford EQUATOR OUCAGS training course 24th October 2015 Outline Ideal study Further topics Multiplicity

More information

Guidelines and Clinical Utility of Molecular Diagnostics

Guidelines and Clinical Utility of Molecular Diagnostics Guidelines and Clinical Utility of Molecular Diagnostics Al B. Benson III, MD, FACP Professor of Medicine Associate Director for Clinical Investigations Robert H. Lurie Comprehensive Cancer Center of Northwestern

More information

INTRODUCTION TO EPIDEMIOLOGICAL STUDY DESIGNS PHUNLERD PIYARAJ, MD., MHS., PHD.

INTRODUCTION TO EPIDEMIOLOGICAL STUDY DESIGNS PHUNLERD PIYARAJ, MD., MHS., PHD. INTRODUCTION TO EPIDEMIOLOGICAL STUDY DESIGNS PHUNLERD PIYARAJ, MD., MHS., PHD. 1 OBJECTIVES By the end of this section, you will be able to: Provide a definition of epidemiology Describe the major types

More information

Risk-Assessment Instruments for Pain Populations

Risk-Assessment Instruments for Pain Populations Risk-Assessment Instruments for Pain Populations The Screener and Opioid Assessment for Patients with Pain (SOAPP) The SOAPP is a 14-item, self-report measure that is designed to assess the appropriateness

More information

Clinical Policy Title: Breast cancer index genetic testing

Clinical Policy Title: Breast cancer index genetic testing Clinical Policy Title: Breast cancer index genetic testing Clinical Policy Number: 02.01.22 Effective Date: January 1, 2017 Initial Review Date: October 19, 2016 Most Recent Review Date: October 19, 2016

More information

Issues in popular designs for observational studies

Issues in popular designs for observational studies Issues in popular designs for observational studies Doug Altman University of Oxford on behalf of STRATOS TG5: Study Design CSM celebration, 29 March 2017 TG5 Members Chairs: Mitchell Gail, Suzanne Cadarette

More information

Reliability, validity, and all that jazz

Reliability, validity, and all that jazz Reliability, validity, and all that jazz Dylan Wiliam King s College London Published in Education 3-13, 29 (3) pp. 17-21 (2001) Introduction No measuring instrument is perfect. If we use a thermometer

More information

Temporal trends in the incidence of molecular subtypes of breast cancer. Jonine D. Figueroa, Ph.D., M.P.H.

Temporal trends in the incidence of molecular subtypes of breast cancer. Jonine D. Figueroa, Ph.D., M.P.H. Temporal trends in the incidence of molecular subtypes of breast cancer Jonine D. Figueroa, Ph.D., M.P.H. Epidemiology: Health data science Study of the distribution and determinants of health and disease

More information

How Does Analysis of Competing Hypotheses (ACH) Improve Intelligence Analysis?

How Does Analysis of Competing Hypotheses (ACH) Improve Intelligence Analysis? How Does Analysis of Competing Hypotheses (ACH) Improve Intelligence Analysis? Richards J. Heuer, Jr. Version 1.2, October 16, 2005 This document is from a collection of works by Richards J. Heuer, Jr.

More information

Statistical validation of biomarkers and surogate endpoints

Statistical validation of biomarkers and surogate endpoints Statistical validation of biomarkers and surogate endpoints Marc Buyse, ScD IDDI, Louvain-la-Neuve & Hasselt University, Belgium marc.buyse@iddi.com OUTLINE 1. Setting the scene: definitions and types

More information

The Three Ages of Systemic Adjuvant Therapy for EBC

The Three Ages of Systemic Adjuvant Therapy for EBC The Three Ages of Systemic Adjuvant Therapy for EBC 1896-2018 Nicholas Wilcken Westmead Hospital Petersham Institute University of Sydney Questions to be answered (concentrating on chemotherapy) Why should

More information

Defining and Measuring Recent infection

Defining and Measuring Recent infection Defining and Measuring Recent infection Application to Incidence Estimation Alex Welte Alex Welte (SACEMA) Recent Infection November 2013 1 / 29 Introduction What is Recent Infection? Introduction of a

More information

Discussion Meeting for MCP-Mod Qualification Opinion Request. Novartis 10 July 2013 EMA, London, UK

Discussion Meeting for MCP-Mod Qualification Opinion Request. Novartis 10 July 2013 EMA, London, UK Discussion Meeting for MCP-Mod Qualification Opinion Request Novartis 10 July 2013 EMA, London, UK Attendees Face to face: Dr. Frank Bretz Global Statistical Methodology Head, Novartis Dr. Björn Bornkamp

More information

EQUATOR Network: promises and results of reporting guidelines

EQUATOR Network: promises and results of reporting guidelines EQUATOR Network: promises and results of reporting guidelines Doug Altman The EQUATOR Network Centre for Statistics in Medicine, Oxford, UK Key principles of research publications A published research

More information

What is indirect comparison?

What is indirect comparison? ...? series New title Statistics Supported by sanofi-aventis What is indirect comparison? Fujian Song BMed MMed PhD Reader in Research Synthesis, Faculty of Health, University of East Anglia Indirect comparison

More information

Results. NeuRA Treatments for internalised stigma December 2017

Results. NeuRA Treatments for internalised stigma December 2017 Introduction Internalised stigma occurs within an individual, such that a person s attitude may reinforce a negative self-perception of mental disorders, resulting in reduced sense of selfworth, anticipation

More information

Publishing Your Study: Tips for Young Investigators. Learning Objectives 7/9/2013. Eric B. Bass, MD, MPH

Publishing Your Study: Tips for Young Investigators. Learning Objectives 7/9/2013. Eric B. Bass, MD, MPH Publishing Your Study: Tips for Young Investigators Eric B. Bass, MD, MPH Learning Objectives To apply a logical approach to organizing & presenting your work in a manuscript To recognize the importance

More information

Research in Real-World Settings: PCORI s Model for Comparative Clinical Effectiveness Research

Research in Real-World Settings: PCORI s Model for Comparative Clinical Effectiveness Research Research in Real-World Settings: PCORI s Model for Comparative Clinical Effectiveness Research David Hickam, MD, MPH Director, Clinical Effectiveness & Decision Science Program April 10, 2017 Welcome!

More information

Prognosis. VCU School of Medicine M1 Population Medicine Class

Prognosis. VCU School of Medicine M1 Population Medicine Class Prognosis VCU School of Medicine M1 Population Medicine Class Gonzalo Bearman MD, MPH Associate Professor of Medicine, Epidemiology and Community Health Associate Hospital Epidemiologist Virginia Commonwealth

More information

Addressing error in laboratory biomarker studies

Addressing error in laboratory biomarker studies Addressing error in laboratory biomarker studies Elizabeth Selvin, PhD, MPH Associate Professor of Epidemiology and Medicine Co-Director, Biomarkers and Diagnostic Testing Translational Research Community

More information

Summary HTA. HTA-Report Summary

Summary HTA. HTA-Report Summary Summary HTA HTA-Report Summary Prognostic value, clinical effectiveness and cost-effectiveness of high sensitivity C-reactive protein as a marker in primary prevention of major cardiac events Schnell-Inderst

More information

You can t fix by analysis what you bungled by design. Fancy analysis can t fix a poorly designed study.

You can t fix by analysis what you bungled by design. Fancy analysis can t fix a poorly designed study. You can t fix by analysis what you bungled by design. Light, Singer and Willett Or, not as catchy but perhaps more accurate: Fancy analysis can t fix a poorly designed study. Producing Data The Role of

More information

SkillBuilder Shortcut: Levels of Evidence

SkillBuilder Shortcut: Levels of Evidence SkillBuilder Shortcut: Levels of Evidence This shortcut sheet was developed by Research Advocacy Network to assist advocates in understanding Levels of Evidence and how these concepts apply to clinical

More information

Epidemiology 2200b Lecture 3 (continued again) Review of sources of bias in Case-control, cohort and ecologic(al) studies

Epidemiology 2200b Lecture 3 (continued again) Review of sources of bias in Case-control, cohort and ecologic(al) studies Epidemiology 2200b Lecture 3 (continued again) Review of sources of bias in Case-control, cohort and ecologic(al) studies Before we start one- and two-sided tests When to we perform a one-sided hypothesis

More information

Research Questions and Survey Development

Research Questions and Survey Development Research Questions and Survey Development R. Eric Heidel, PhD Associate Professor of Biostatistics Department of Surgery University of Tennessee Graduate School of Medicine Research Questions 1 Research

More information

Let s look a minute at the evidence supporting current cancer screening recommendations.

Let s look a minute at the evidence supporting current cancer screening recommendations. I m Dr. Therese Bevers, Medical Director of the Cancer Prevention Center and Professor of Clinical Cancer Prevention at The University of Texas MD Anderson Cancer Center. Today s lecture is on screening

More information

CRITICAL APPRAISAL WORKSHEET 1

CRITICAL APPRAISAL WORKSHEET 1 For Residents CRITICAL APPRAISAL WORKSHEET 1 AN ARTICLE ON THERAPY This worksheet should be submitted along with the completed Submission Form to your supervisor. It is based on the articles on therapy

More information

Systematic Reviews. Madhukar Pai, MD, PhD Associate Professor Department of Epidemiology & Biostatistics McGill University, Montreal, Canada

Systematic Reviews. Madhukar Pai, MD, PhD Associate Professor Department of Epidemiology & Biostatistics McGill University, Montreal, Canada Systematic Reviews Madhukar Pai, MD, PhD Associate Professor Department of Epidemiology & Biostatistics McGill University, Montreal, Canada Email: madhukar.pai@mcgill.ca Information explosion a key barrier

More information

Reporting and Methods in Clinical Prediction Research: A Systematic Review

Reporting and Methods in Clinical Prediction Research: A Systematic Review Reporting and Methods in Clinical Prediction Research: A Systematic Review Walter Bouwmeester 1., Nicolaas P. A. Zuithoff 1., Susan Mallett 2, Mirjam I. Geerlings 1, Yvonne Vergouwe 1,3, Ewout W. Steyerberg

More information

providers in these settings face in adhering to guidelines for evaluating patients for TB. We identified diagnostic guideline adherence and

providers in these settings face in adhering to guidelines for evaluating patients for TB. We identified diagnostic guideline adherence and Summary 87 Summary The studies presented in this thesis were focused on the quality of diagnostic care provided to suspected tuberculosis (TB) patients and centre on four principle investigations. The

More information

Rationale For & Design of TAILORx. Joseph A. Sparano, MD Albert Einstein College of Medicine Montefiore-Einstein Cancer Center Bronx, New York

Rationale For & Design of TAILORx. Joseph A. Sparano, MD Albert Einstein College of Medicine Montefiore-Einstein Cancer Center Bronx, New York Rationale For & Design of TAILORx Joseph A. Sparano, MD Albert Einstein College of Medicine Montefiore-Einstein Cancer Center Bronx, New York Declining Breast Cancer Mortality & Event Rates in Adjuvant

More information

Types of Data. Systematic Reviews: Data Synthesis Professor Jodie Dodd 4/12/2014. Acknowledgements: Emily Bain Australasian Cochrane Centre

Types of Data. Systematic Reviews: Data Synthesis Professor Jodie Dodd 4/12/2014. Acknowledgements: Emily Bain Australasian Cochrane Centre Early Nutrition Workshop, December 2014 Systematic Reviews: Data Synthesis Professor Jodie Dodd 1 Types of Data Acknowledgements: Emily Bain Australasian Cochrane Centre 2 1 What are dichotomous outcomes?

More information

Results. NeuRA Treatments for dual diagnosis August 2016

Results. NeuRA Treatments for dual diagnosis August 2016 Introduction Many treatments have been targeted to improving symptom severity for people suffering schizophrenia in combination with substance use problems. Studies of dual diagnosis often investigate

More information

Safeguarding public health CHMP's view on multiplicity; through assessment, advice and guidelines

Safeguarding public health CHMP's view on multiplicity; through assessment, advice and guidelines Safeguarding public health CHMP's view on multiplicity; through assessment, advice and guidelines Rob Hemmings Statistics Unit Manager, MHRA CHMP member Chair, CHMP Scientific Advice Working Party Biostatistics

More information

Epidemiologic Study Designs. (RCTs)

Epidemiologic Study Designs. (RCTs) Epidemiologic Study Designs Epidemiologic Study Designs Experimental (RCTs) Observational Analytical Descriptive Case-Control Cohort + cross-sectional & ecologic Epidemiologic Study Designs Descriptive

More information

Critical Appraisal. Einas Al-Eisa, MSc, PhD King Saud University

Critical Appraisal. Einas Al-Eisa, MSc, PhD King Saud University Critical Appraisal Einas Al-Eisa, MSc, PhD King Saud University Critical Appraisal = assessment of evidence by systematically reviewing its: Relevance Validity Applicability of its results to specific

More information

Dissemination experiences from CONSORT and other reporting guidelines

Dissemination experiences from CONSORT and other reporting guidelines Dissemination experiences from CONSORT and other reporting guidelines Doug Altman Centre for Statistics in Medicine, Oxford EQUATOR Network [Altman and Goodman, JAMA 1994] 2 Passive

More information

Downloaded from:

Downloaded from: Hemingway, H; Croft, P; Perel, P; Hayden, JA; Abrams, K; Timmis, A; Briggs, A; Udumyan, R; Moons, KG; Steyerberg, EW; Roberts, I; Schroter, S; Altman, DG; Riley, RD; PROGRESS Group (2013) Prognosis research

More information

The Oncotype DX Assay A Genomic Approach to Breast Cancer

The Oncotype DX Assay A Genomic Approach to Breast Cancer The Oncotype DX Assay A Genomic Approach to Breast Cancer Pathology: 20 th and 21 st Century Size Age Phenotype Nodal status Protein/Gene Genomic Profiling Prognostic & Predictive Markers Used in Breast

More information

Results. NeuRA Hypnosis June 2016

Results. NeuRA Hypnosis June 2016 Introduction may be experienced as an altered state of consciousness or as a state of relaxation. There is no agreed framework for administering hypnosis, but the procedure often involves induction (such

More information

Gene expression profiling predicts clinical outcome of prostate cancer. Gennadi V. Glinsky, Anna B. Glinskii, Andrew J. Stephenson, Robert M.

Gene expression profiling predicts clinical outcome of prostate cancer. Gennadi V. Glinsky, Anna B. Glinskii, Andrew J. Stephenson, Robert M. SUPPLEMENTARY DATA Gene expression profiling predicts clinical outcome of prostate cancer Gennadi V. Glinsky, Anna B. Glinskii, Andrew J. Stephenson, Robert M. Hoffman, William L. Gerald Table of Contents

More information

Current Status and Future Development of Tools for Prognosis and Prediction - USA

Current Status and Future Development of Tools for Prognosis and Prediction - USA Current Status and Future Development of Tools for Prognosis and Prediction - USA John H. Ward, MD Professor of Medicine University of Utah School of Medicine Huntsman Cancer Institute October18, 2009

More information

Journal Club Critical Appraisal Worksheets. Dr David Walbridge

Journal Club Critical Appraisal Worksheets. Dr David Walbridge Journal Club Critical Appraisal Worksheets Dr David Walbridge The Four Part Question The purpose of a structured EBM question is: To be directly relevant to the clinical situation or problem To express

More information