Two Important Statistics 70% 40% 0 Quality Control & YOU David Plaut Fall, 2012 davidplaut@yahoo.com 1 11
Why do physicians order laboratory tests? 2 2 Why do Physicians Order Laboratory Tests? 1. Patients cannot always give an accurate history. 2. Signs (patient looks flushed) are not specific for a particular disease. 3. Symptoms ( It hurts! ) are not specific. 3 3 21
Achieving Quality Results Quality Goals --> Quality Monitoring (QM) --> Quality Control (QC) 4 4 The Three Sources of Errors 1. Pre-analytical ~ 60 70% 2.Analytical ~ 5-10 3. Post-analytical ~ 25-35 5 31
Types of Errors The four types of analytical errors Gross Random Random Systematic Anyone would detect Inherent, background, normal Not normal; something s wrong Bias, shift, trend 6 6 Sources of Error? Bad instrument Bad reagents Bad control Bad calibration Bad sample 7 7 41
Types of Errors 1 Random Same patient sample run 10 times 1 51.0 2 50.3 3 48.9 4 54.2 5 49.2 6 50.0 7 52.7 8 48.5 9 49.9 10 53.3 Why are the values not identical? 8 Types of Errors 1 Random A normal, bell, Gaussian, curve mean Frequency (-) (+) Value. 9 9 51
The Gaussian Curve Probability or Frequency mean Areas under the curve -1 -- 1 68% -2 2 95% -3 3 99.7% Optional: Click here to view animation. 10 10 Types of Errors 2 Increased Random Errors mean Frequency (-) (+) Value 11 11 61
Types of Errors 2 Increased Random Occur on both sides of the mean and can be of varying size 12.00 10.00 8.00 6.00 4.00 2.00 0.00 0 5 10 15 20 25 Tend to be caused by changes in the instrument 12 12 Types of Errors 3 The second type of analytical error: Systematic Tend to be on one side (same side) of the mean Tend to be caused by changes made by people changing reagents calibration doing something to the instrument 13 13 71
Types of Errors 3 Systematic analytical error (aka bias) Old mean The true mean is truly not known in analytical hematolgy For us it is either the package insert or the group mean. Let s agree that our mean is reasonably close to the true mean. 14 14 Types of Errors 2 The second type of analytical error: Systematic Tend to be on one side (same side) of the mean Tend to be caused by changes made by people changing reagents calibration doing something to the instrument 15 15 81
Types of Errors 2 Systematic analytical error (aka bias) True mean The true mean is truly not known in analytical hematology. For us it is either the package insert or the group mean. Let s agree that our mean is reasonably close to the true mean. 16 16 The Primary Goal of our QC Program To detect errors significant rapidly 17 91
To Detect the Two Kinds of Errors: Two Statistics The Mean The CV 18 HOW MANY DATA POINTS TO ESTABLISH A MEAN? Hgb Cumulative "Crit" Cumulative Data Mean CV Mean CV 1 12.5 42 2 11.5 39 3 11.7 41 4 11.8 11.9 3.8 40 40.3 3.2 5 11.7 11.9 41 40.4 6 11.7 11.8 38 40.0 7 12.5 11.9 39 39.9 8 12.0 11.9 3.2 39 39.8 3.2 9 12.5 12.0 40 39.8 10 12.1 12.0 3.2 39 39.7 3.0 20 11.9 11.9 3.3 39 40.0 2.8 30 11.4 12.0 3.6 39 39.9 2.7 Mean 12.0 39.9 SD 0.4 1.1 CV % 3.6 2.7 19 101
Establishing YOUR SD From the historical data use the cumulative CV (CV c ) in the formula: SD(New) = CV c * mean (New) Westgard, J. et al. Clin Chem. 27: 493, 1981. Plaut, D. various publications 20 20 Establishing YOUR CV CV= mean* SD 21 21 111
How are 3 CV% and SDI related? 3 r = 1.000 2 1 0 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 1 2 3 22 How are 3 CV% and SDI related? 1.50 1.00 0.50 0.00 0.40 0.30 0.20 0.10 0.00 0.50 0.10 0.20 1.00 1.50 2.00 2.50 3.00 Series1 1 8 0.28 2.53 2 8.4 0.25 2.22 3 8.8 0.21 1.91 4 9.2 0.18 1.59 5 9.6 0.14 1.28 6 10 0.11 0.96 7 10.4 0.07 0.65 8 10.8 0.04 0.33 9 11.2 0.00 0.02 10 11.6 0.03 0.30 11 12 0.07 0.61 12 12.4 0.10 0.93 23 121
The Primary Goal of our QC Program To detect errors significant rapidly 24 What is a Significant Error? or When MUST I? 25 131
What is a Significant Error? For our discussion 12.5 is correct value. Name a test that could have this value PT, BUN, Hemoglobin. Which of these values is wrong? or When do I freak? 12.5 12.3 12.0 11.5 11.0 10.5 10.0 26 When do I freak? Mean YOU CLIA Your Lab Group Limit & Mean Mean Error budget CLIA LIMIT 27 27 141
When do I freak? my 3 CV range error budget Freak O mean x My mean Freak? Freak + 28 Selecting the Rules: An Example From CAP My Group Acceptable ~ Error Value Mean SD Range Budget CVs I 54 54 2.6 40-68 (5) 2 II 461 462 11.9 341-578 (9) 6 III 194 201 5.3 150-252 (10) 7 IV 109 109 4.2 81-137 (7) 4 V 319 320 7.9 243-406 (10) 7 29 151
Platelet QC CAP YOU 40 45 50 55 60 65 70 30 Platelet QC CAP YOU 300 350 400 450 500 550 600 31 161
An Example 300 runs/yr with 3 controls/run limits at 3 CV% and 1.0% Failures Good data Bad data In 298 good runs TA (297reported) 2 TR Out 32 3CV 2CV 1 3 CV% Systematic? Or Increased random error? x... o...... Mean ----------------------------- x x 2CV 3CV x o x o o x x o............... o o A value could exceed 3 S for many reasons -- random control material short sample mix up in control systematic reagent calibration thus 1 3 CV difficult to troubleshoot without more information. 1 2 3 4 5 6 7 33 33 171
Document! 8-25-12. Changed Lot 123.MMP. Expires 11.2012 signed by DSP. 34 34 Instrument Self Checks Pressure Vacuum Background Microprocessor Mechanical parts Temperature.are all monitored continuously and will give an error message if a measurement is out of specs 35 181
Delta Checks Absolute Delta Hemoglobin 11/10/11 12.1 11/11/11 9.2? WBC 05/09/12 9.0 06/22/12 4.1? Johnson and Stelmach Clin. Lab News September, 2007 36 Patient Moving Average A QC tool developed by Dr. Brian Bull in 1974. This algorithm is based on the inherent stability of the red blood cell indices in a general hospital populations. Utilizes patient data, recording the MCV, MCH and MCHC of each patient sample. Monitors stability over time of: Instrumentation Reagents Technique Patient population 37 191
Average of Normals 5 Patient Runs of 20 1 89 84 91 87 93 2 98 80 89 85 92 17 100 97 96 84 85 18 92 90 85 91 91 19 92 93 91 92 91 20 89 86 86 84 94 Control Patients Mean 90 89 90 90 90 SD 6 5 5 5 4 3% 88 86 87 87 88 +3% 93 92 92 93 93 38 Patient Average 98 96 94 92 90 88 86 0 5 10 15 20 25 39 201
Advantages of the Patient Moving Average Inexpensive, simple to use Utilizes unfixed patient blood Incorporated into current Hematology analyzers Real time QC 40 Installing X Series: Xm Settings Xm must be turned ON Set Xm batch size Set Sampler Stop Conditions (QC Errors) Assign QC Limits Enter lab established Xm Limits Assign Variable Target for Xm Calculate lab s Historical limits 41 211
Patient Moving Averages X-bar-M Effective in checking hour-by-hour changes in instrument and reagents Parameters remain stable once characteristics of population established and the population is stable Does not include: ID number zero Commercial QC data Calibration data Background counts that exceeds linearity range (@) those are unreliable data ( *) Data reported with analysis error displayed as **** or ---- 42 QC Troubleshooting Tool Concern? Commercial QC - OK Xm - OK Commercial QC - OK Xm - Not OK Commercial QC - Not OK Xm - OK Commercial QC - Not OK Xm - Not OK No Problem Population Shift or Reagent Control problem? Systematic Error 221
Time for more questions & discussion 44 Tinkerbell 45 231
Thank You davidplaut@yahoo.com 46 241