RELIABILITY OF OPERATORS DURING THE VISUAL INSPECTION OF PRODUCED PARENTERAL DRUGS

Transcription

1 RELIABILITY OF OPERATORS DURING THE VISUAL INSPECTION OF PRODUCED PARENTERAL DRUGS F. Sadeghipour, A. Bugmann, V. Herrera, P. Bonnabry Geneva, 22 March th Congress of the European Association of Hospital Pharmacists Geneva Palexpo, Switzerland

2 Conflict of Interest Nothing to disclose!

3 Background The visual inspection of parenteral drugs is mandatory (PhEur( PhEur). Most hospitals do not have automatic devices to perform this control Results are therefore strongly dependent on the performance of human operators.

4 Background The reliability of human controls was demonstrated in other fields to be not higher than 85% No data regarding specifically the visual inspection exist in the literature. Venn diagram of two inspectors results Knapp J.Z. PDA J. of Pharm. Sc. & Tech. 54 (3), 2000, pp (6), 1999, pp (4), 1998, pp

5 Objectives To develop training and validation kits in order to permit a : Structured training of visual inspectors Standard validation of all the operators To evaluate the performance of each inspector and to determine his reliability in the detection of Non-Conform vials and ampoules produced in the Pharmacy

6 Methods A visual inspection training kit 30 positive vials A visual inspection validation kit was developed : 90 negative + 10 positive vials 15 different operators used the validation kit.

7 Methods The performance of operators during the validation was estimated by the calculation of the following parameters from their results grid: True Positive (TP) : 10 positive (Non-Conform) vials of the validation kit True Negative (TN) : 90 Negative (Conform) vials of the validation kit False Positive (FP) : Conform vials identified as Non-Conform by the visual inspector False Negative (FN) : Non-Conform vials identified as Conform by the visual inspector

8 Methods The performance of operators during the validation was estimated by the calculation of the following parameters from their results grid: Sensitivity (Se) : Detection of Non-Conformity Specificity (Sp): Se the ONLY detection of Conform vials (0 to 1 for all the values) = TP TP + FN TN Sp = TN + FP

9 Methods The performance of operators during the validation was estimated by the calculation of the following parameters from their results grid: Positive predictive value (PPV) : Probability that a detected non-conformity is true PPV Negative predictive value (NPV) : Probability that a conformity is true (0 to 1 for all the values) NPV = = TP TP + FP TN TN + FN

10 Methods The performance of operators during the validation was estimated by the calculation of the following parameters from their results grid: The Accuracy Score (AS) : Sum of the 4 values x 100 (0 to 400) AS = ( Se + Sp + PPV + NPV ) x 100

11 Results The results showed a high variability between the operators for the different calculated values : Parameter Mean value Minimum Maximum RSD Sensitivity % Specificity % PPV % NPV % AS %

12 Discussion Similar experiences in medical field performed on the mammography interpretation of radiologists also showed important variability in obtained results. Results were surprising : For a same mammography, radiologists have proposed several diagnoses : Especially in one case, two radiologists were not agree with the localization of the cancer (left breast or right breast). Radiologists with lower experience showed less satisfying results than radiologists who practice mammography interpretation everyday.

13 Conclusions Our results confirmed : A limited reliability of human controls, which is probably reinforced by the particularities of the studied task : Repetitive activity Necessity of a high-attention Particle size approaching the limits of vision performance

14 Conclusions Our results confirmed that: Even if the experimented operators have better results than beginners, a systematic evaluation approach determines their real reliability (Sensitivity, Specificity, PPV, NPV AS) Consequently : The initial and continuous training are essential elements of the quality assurance of visual inspection.

15 Aknowledgement Special thanks to Ms Alexandra Bugmann who carried out this work for her Master in the Pharmacy of the University hospitals of Geneva This presentation could be downloaded at