Mathematical Microbiologists: Why we have to return to our square roots to uncover uncertainty (of measurement) in Quantitative PCR (qpcr)

Transcription

1 Mathematical Microbiologists: Why we have to return to our square roots to uncover uncertainty (of measurement) in Quantitative PCR (qpcr) J. Ian Stuart George Zahariadis Marina Salvadori

2 Objectives 1. Identify why Uncertainty of Measurement is required for qpcr and why it is important clinically 2. Demonstrate why you may be understating your uncertainty if you use a black box approach 3. Present appropriate methods for UOM calculation in qpcr 4. For discussion: A) The need for universal standards, B) Why clinical samples maybe better for UOM determination

3 Uses of viral load qpcr As surrogates for viral resistance For diagnosis of infection/disease As a trigger to start pre emptive therapy in order to prevent development of disease For treatment monitoring or safety monitoring Clearly a qpcr result has clinically important implications

4 What is our certainty in qpcr? Experience has taught us there can be a lot variability in measurements of any analyte This can occur both within a laboratory and between laboratories Clinicians use laboratory results as the foundation to implement treatment How can a laboratory handle this uncertainty when reporting a measured value?

5 Next 4 slides are a summary of a study by Dr. J Preiksaitis on CMV proficiency panels tested at international laboratories that highlight points regarding uncertainty of measurement(1) CMV DNA Copies/ml (log10) Clinical sample CMV viral panel sample Expected result based on stock quantified by reference laboratories Positive but not quantifiable (assigned lowest detectable value) CMV Sample Number

6 Summary of CMV Qualitative Results (constructed samples) 35 panels/33 labs Sample No EM based expected result copies/ml (log10) Reference lab expected result copies/ml (log10) Negative (%) Number of panels Positive-NQ (%) Positive-Q (%) (97) 0 1 (3) (94) 0 1 (3) (74) 6 (17) 3 (9) (11) 4 (11) 27 (77) (3) 34 (97) (6) 33 (94) (100) (100) (100) One test was invalid Pos-NQ: positive but not quantifiable Pos-Q: positive with quantifiable results

7 Summary of CMV Quantitative results (constructed samples) 35 panels/33 labs Sample No EM based expected result copies/ml (log10) Reference lab expected result copies/ml (log10) Number positive GM ±SD copies/ml (log10) Median (range) copies/ml (log10) ± (0-2.78) ± (0-4.32) ± ( ) ± ( ) ± ( ) ± ( ) ± ( ) Geometric mean; negative results were excluded

8 CMV Qualitative and Quantitative results (clinical samples) Clinical Sample Number #10 #05 #01 Qualitative Result Negative (%) 13 (37) 0 0 Pos-NQ (%) 9 (26) 1 (3) 0 Pos-Q (%) 13 (37) 34 (97) 35 (100) Quantitative Result copies/ml (log10) GM±SD 2.78 ± ± ± 0.47 Median (range) 2.24 (0-4.18) 3.87 ( ) 3.99 ( ) GM=Geometric mean; negative results were excluded

9 Ground Rules The methods in the following presentation are based on fictionalized data for CMV quantitative PCR. While some of the methods presented are sound, the results and error limits presented ARE NOT APPLICABLE TO ANY PATIENT POPULATION. Please consult your own laboratory for appropriate guidance on the significance of a result in quantitative PCR. Please reserve questions until the end of the presentation.

10 No! This is not Driver s Ed_ despite the figures to the right!

11 Some equations

12 The Problem with Inverse Regression

13 Another way to view this: From Miller 2006 (2)

14 The Wrong Way to do UOM

15

16 About ½ log difference over the entire range

17 Proper ways to Calculate UOM 1) The Box Method (note that the box is no longer black) 2) Taylor (Delta) Method 3) Bootstrapping 4) According to Fieller s Theorem

18 A graphical Approach

19

20

21

22 Problems with the box: Assumes normal distribution and homoscedastic data Only gives an approximation of UOM May underreport uncertainty Lacks mathematical proofs Advantages: Easy to use: Linest function in Excel Can be done manually for quick evaluations

23 About 0.7 to 0.95 log difference throughout the range

24 And you thought that you were done with calculus after your undergraduate courses!

25

26

27 About ½ log difference through the entire range. Note Convergence of the error bars around the mean Ct value (17)

28 Advantages: Mathematically more intuitive than other methods Easier to compute Disadvantages: Only gives an approximation of UOM Falls apart when the denominator of the ratio has a mean that approaches zero Assumes normal distributions and homoscedastic data

29

30 Fieller s method: Developed for The Biological Standardization of Insulin The Delta method only considers positive ratios: In Fieller s method, a negative ratio is considered and permits a quadratic solution for variance: Proof of Fieller s theorem is widely available and is beyond the scope of this discussion.? Could Fieller s conviction that a Ratio is a Ratio have been borrowed by a certain Canadian Politician who stated No, a proof is a proof. What kind of a proof? It's a proof. A proof is a proof, and when you have a good...?

31 About 1 log difference over the entire range

32 Advantages: Conservative estimates of error Quadratic formula permits easy computation One of the best recognized methods for evaluating error in ratios Disadvantages: Assumes normal distributions and homoscedastic data

33 THE MANHATTAN PROJECT? Now how can that apply to uncertainty of measurement for qpcr?

34 Bootstrapping (a special case of Monte Carlo Sampling) Resample the original sample with replacement k times k usually equals 1000 or greater Quote (Fox 2008): The population is to the sample as as the sample is to the bootstrap sample. So: 1)The bootstrap observations are analogous to the original observations 2)The bootstrap mean is analogous to the sample mean which is analogous to the population mean 3)The distribution of the bootstrap sample means is analogous to the distribution of means (n) drawn from the original sample

35 How to bootstrap

36 From (Ref. 6)

37 From (Ref. 7)

38

39

40

41 Advantages of Bootstrap: Relatively easy to perform Data does not have to be normally distributed or homoscedastic Mathematically simple Disadvantages of Bootstrap: May be computationally intensive (i.e plus loop commands creating a matrix of plus elements)

42 About 0.7 to 1 log difference over the entire range

43

44

45 For Discussion 1) The Need for standardized universal samples (such as those currently available for HIV, Hepatitis C and CMV): Healthcare portability Standardized reporting Inter laboratory comparisons 2) Standardized samples vs. patient samples for UOM determination: Precision vs. Accuracy

46 References 1) Preiksaitis, J. Hayden, R., Tong, Y. et al. Assessing the Impact of the 1rst WHO International Standard for Human Cytomegalovirus (HCMV) DNA on Result Harmonization. World Congress of Transplantation conference ) Miller, J. N. Uncertainties in concentrations estimated from calibrations experiments. calibrationexperiments technical brief 22_tcm pdf ) Parker, P., Vining, G., Szarka III, J., Johnson, N. The Prediction Properties of Inverse and Reverse Regression for the Simple Linear Calibration Problem ) Beyene, J. and Moineddin, R. Methods for confidence interval estimation of a ratio parameter with application to location quotients. BMC Medical Research Methodology : ) Fox, F. and Weisberg, S. Bootstrapping Regression Models in R. An Appendix to An R Companion to Applied Regression, Second Edition. Bootstrapping.pdf ) resampling/ 7)