The Practical Application of Statistics: Reading papers and interpreting clinical studies...

Transcription

1 The Practical Application of Statistics: Reading papers and interpreting clinical studies... Richard Szydlo Imperial College / Hammersmith Hospital, London

2 Karlskirche

3

4

5

6

7

8

9

10 Evaluating new treatments New treatments - pharmaceutical agents, devices, surgical procedures - are being developed every day. How do we know if there is any benefit to patients of these new treatments? Randomised clinical trials (RCT s) provide the best means for evaluating the benefits and risks of a new treatment in comparison with current therapy.

11 Types of studies Retrospective Clinical trials Non-randomised Randomised

12 Retrospective studies In a comparison of patients undergoing a new treatment with a historical cohort receiving the old treatment, there may be trends in health care or other factors related to the outcome. In addition, the quality of data available for historical controls may differ from the quality of data collected for patients receiving the new treatment because of different diagnostic criteria, measurement standards, completeness of records, etc. Thus, lack of standard criteria could yield spurious results.

13 Retrospective studies Database studies tend to be inexpensive and allow evaluation of large numbers of patients, but potential bias in treatment selection, outcome assessment, and other factors make them a lower standard of evidence. Although these types of studies improve on case studies or case series because they offer comparison groups, problems with selection of patients for treatment and lack of comparability of treatment groups complicate the analysis. The randomised, controlled clinical trial is designed to reduce such biases.

14

15

16

17 In a phase-ii trial designed for patients who were unresponsive to interferon, treatment with imatinib resulted in a 41% complete cytogenetic response, but a survival advantage for these patients has not been established. Because it would not now be possible to initiate prospective randomised trials of treatment with imatinib for such patients, we compared the outcome for patients who were unresponsive to interferon, and were then treated with imatinib, with that for patients in a large trial of conventional treatment for chronic myeloid leukamia.

18

19

20

21

22

23 Sign of the Zodiac as a predictor of survival for recipients of an allogeneic stem cell transplant (SCT) for Chronic Myeloid Leukaemia (CML) R. M. Szydlo, I.Gabriel, E. Olavarria and J. F. Apperley Dept of Haematology, Imperial College, Hammersmith Hospital, London, UK

24 Zodiac (star) sign has been shown to be a statistically significant factor in the determination of a variety of diseases, conditions and phenomena: schizophrenia, GI haemorrhage, humerus fracture, fertility and suicidal disposition. To investigate the relevance of this factor in the context of a SCT, we examined its influence in a population of patients with CML. This disease has well-established prognostic factors encompassed in the EBMT/Gratwohl scoring system, and is an excellent model for teasing out potential new factors.

25 Probability of Survival % Aries (N=44) Aquarius (N=35) 20 0 P= Years post SCT

26 Probability of Survival % Aries, Taurus, Gemini, Leo, Scorpio, Capricorn (N=317) Aquarius, Cancer, Virgo, Sagittarius, Pisces, Libra (N=309) P= Years post SCT

27 Multivariate analysis of survival Variable N Relative Risk 95%CI P Gratwohl Score Zodiac Sign Group A Group B

28 Multivariate analysis of survival Variable N Relative Risk 95%CI P Gratwohl Score p=0.59 p=0.002 p<0.001 Zodiac Sign Group A Group B P=0.009

29 In this study, we have demonstrated that providing adequate care is taken, a statistically significant relationship between patient Zodiac sign and survival post SCT for CML can be observed.

30 We can thus postulate that any pseudoscientific parameter can be manipulated to give a seemingly important result, however implausible. Critical awareness therefore, must be maintained, both with the presentation of statistically significant and non-significant results.

31 Sign of the Zodiac as a predictor of survival for recipients of an allogeneic stem cell transplant for chronic myeloid leukaemia (CML): an artificial association. Szydlo RM, Gabriel I, Olavarria E, Apperley J. Transplant Proc Oct;42(8):

32

33

34

35

36

37

38

39 Patients at risk?

40

41

42 They analysed response in 62 patients, with no adjustment for any other variables.

43

44 Odds ratio and Hazard ratio The odds ratio is the ratio of the odds of an event occurring in one group to the odds of it occurring in another group The hazard ratio in survival analysis is the effect of an explanatory variable on the hazard or risk of an event

45 Granulocyte colony-stimulating factor generates epigenetic and genetic alterations in lymphocytes of normal volunteer donors of stem cells [Nagler,A. et al. Exp.Haem (2004) 32;122-30]. The frequency of cells displaying aneuploidy for chromosome 17 in the 18 samples of the G-CSF group ranged from 15 to 26% (mean and standard deviation value of 19.3±2.7%;)

46 PROTOCOL TITLE: MULTI CENTRE CONTROLLED STUDY ON THE IMPACT OF STEM CELL DONATION EITHER AFTER MOBILISATION WITH GRANULOCYTE COLONY STIMULATING FACTOR OR BONE MARROW HARVEST ON UNRELATED BONE MARROW DONORS. Sponsor s Protocol Number: BRD/06/143 EudraCT Number: SHORT TITLE: Study on the Impact of Stem Cell Donation and Bone Marrow Harvesting on Unrelated Donors.

47 The study took 5 years to recruit enough patients, and found no instances of clonal damage. Although this would appear to be a negative result, in the broad scheme of things it is actually very re-assuring for donors. The study also highlights a number of issues in the original paper by Nagler, which erroneously brought to the attention of the scientific community a detrimental effect for donors. Being a donor is safe, but the publishing of misleading results based on poor analytical methods can have repercussions for the greater scientific community, and thus the responsibilities of peer review must be taken very seriously.

48 The Research Idea Having an idea / research topic... What is the purpose? Audit Paper MD PhD Is it original?

49 Can the clinical / research question be Be realistic... answered?

50 A study published in BJH: An association between human leucocyte antigen alleles and acute and chronic GVHD after SCT

51 No. Age at HLA-DR HLA-DQ N sct cells Diagnosis Conditioning AGvHD C GvHD Rel Died 1 21 DR 5/7 DQ? 1 BM CML Ph+ BUCY2 0 0 relaps 2 36 DR 1/4 DQ 3/5 1 BM CML Ph+ BUCY2 0 0 NA YES 3 59 DR 3/13 DQ 2/6 2 PB CML Ph+ BUCY2 0 limited DR 3/16 DQ 2/5 1 PB NHL BUCY4 0 0 NA YES 5 39 DR 11/11 DQ 3/3 1 BM AML E-BUCY2 0 0 NA YES 6 23 DR 15/11 DQ 3/6 1 BM SAA CY DR 15/15 DQ 5/5 3 PB AML E-BUCY2 0 0 NA YES 8 41 DR 15/11 DQ 3/6 1 BM CML BUCY2 0 extensive NA YES 9 25 DR 15/3 DQ 2/6 2 PB SAA CY4 0 extensive 0 YES DR 3/13 DQ 2/6 1 BM AML BUCY2 0 extensive DR 1/11 DQ 5/3 2 PB CML BUCY2 4 0 NA YES DR 15/3 DQ 6/2 1 BM CML Ph+ E-BUCY2 0 0 relaps DR 15/15 DQ 6/3 1 BM CML Ph+ BUCY DR 7/13 DQ 6/2 1 BM AML BUCY

52 Study check list (1) Is there scientific merit to your study? Do a Pubmed / Google scholar search Have you enough patients / samples? Is there enough time / resources available?

53 Study check list (2) Who will collate / analyse the data? What statistical tests will you use? Talk to experienced researchers... Do not be afraid to ask! Have a good think...

54

55 How not to collaborate with a biostatistician