and accuracy D. Demilly (1) B. de Goyon (2) SNES GNIS-SOC (1) (2) GNIS - GEVES

Transcription

1 Germination interlaboratory test variability and accuracy D. Demilly (1) B. de Goyon (2) (1) SNES (2) GNIS-SOC

2 Statistical analysis use in France for germination ring tests or referee tests Statistical analysis developed by Mrs J.J. Daudin and Ph. Trécourt From the department of Mathematics of the INA-PG Computer program developed by Mr de Goyon (SOC GNIS).

3 SUMMARY I) Analysis of the Variability II) Analysis of the Accuracy Principles of the Barycentric Presentation Examples of the using of the Barycentric Presentation

4 VARIABILITY ANALYSIS WITHIN A LABORATORY

5 The objective of the variability analysis is to measure the fidelity of the used methods, by studying the variability of the results obtained within a given laboratory. To make this analysis about one or more types of germs, and in order to make the calculation easier, we use an angular transformation

6 2 N ARCSINUS P N = seeds Numbers by replication P = Value ( % ) We know that, if the variability observed is only the variability of seeds sampling, the expected value of the variance of the random variable obtained with the above transformation must be equal to 1 ( Angular Transformation )

7 REJECTION REGION ACCEPTANCE REGION REJECTION REGION 0, ,116 Lower Limit Upper Limit CHI 1 2 ( ) k r 1 EXPECTED CHI α 2 α k ( r 1) k = Samples Numbers r = Replications numbers Degrees of Freedom for Chi Squarred Distribution = k x ( r -1 )

8 Total Seeds: 400 Replications ( r) : 4 Samples (k) : 1 Seeds Number by replication : Rep1 Rep2 Rep3 Rep4 MEAN Upper Lower 81% 87% 88% 89% 86% 89% 81% 22,4 24,04 24,34 24,65 0, ,013 Variance 1,013 Standart deviation 1,007 3,116 EXPECTED VALUE = 1 Angular transformation TEST CONCLUSION 5 % Lower Upper Limit Limit 0,072 1,013 3,116 NS TEST CONCLUSION 1 % Lower Upper Limit Limit 0,024 1,013 4,279 NS

9 Total Seeds: 400 Replications ( r) : 4 Samples (k) : 1 Seeds Number by replication : Rep1 Rep2 Rep3 Rep4 MEAN Upper Lower 85% 79% 95% 90% 87% 95% 79% 23,49 21,90 26,91 24,88 0,072 1 Variance 4,521 Standart deviation 2,126 3,116 EXPECTED 4,521 VALUE = 1 TEST CONCLUSION 5 % Lower Upper Limit Limit 0,072 3,116 4,521 heterogeneous TEST CONCLUSION 1 % Lower Upper Limit Limit 0,024 4,279 4,521 heterogeneous

10 Total Seeds: 400 Replications ( r) : 4 Samples (k) : 1 Seeds Number by replication : Rep1 Rep2 Rep3 Rep4 MEAN Upper Lower 89% 87% 88% 88% 88% 89% 87% 24,55 24,04 24,34 24,34 0,045 0,072 1 Variance 0,045 Standart deviation 0,212 3,116 EXPECTED VALUE = 1 TEST CONCLUSION 5 % Lower Upper Limit Limit 0,045 0,072 3,116 too Homogeneous TEST CONCLUSION 1 % Lower Upper Limit Limit 0,024 0,045 4,279 NS

11 The variability analysis can be computed for several types of germs, on the different replications ( r ) of only one sample or on the different replications of all the samples ( k ).

12 Conclusion Results are generally within the accepted limits Heterogeneous results are exceptional as laboratories check there results against ISTA compatibility table Too homogeneous results could happen when analysts harmonize their results between replicates

13 ANALYSIS OF THE ACCURACY

14 The analysis of the Accuracy must show if the results given by a laboratory are as close as possible to the reference results. The reference results are - the mean of the all laboratories, - the results of one laboratory, - the mean result from several laboratories chosen as reference.

15 PRINCIPLES OF THE BARYCENTRIC PRESENTATION

16 This analysis is carried out in two steps. First Analyse sample by sample the results of each laboratory to find specific deviations. Second Analysis of all the samples to find the general tendency of each laboratory, and thus to appreciate accuracy.

17 THE BARYCENTRIC PRESENTATION by SAMPLE

18 0 100 UNGERMINATED SEEDS ABNORMAL NORMAL

19 25 % UNGERMINATED SEEDS 40 % ABNORMAL Lab 1 40 % Lab 2 25 % % 50 % NORMAL 0

20 25 % UNGERMINATED SEEDS 40 % ABNORMAL Lab 1 40 % MEAN Lab 2 25 % % 50 % NORMAL 0

21 Computation of a confidence interval ( ) ± N0 P U P P 0 1 α 2 1 N1 P0 N0 : MEAN OF THE REFERENCE : TOTAL SEEDS NUMBER FOR THE REFERENCE N1 : TOTAL SEEDS NUMBER FOR EACH LABORATORY U1 α : NORMAL LAW VALUE 2

22 A Band of Confidence for Normal Ungerminated Seeds Band of Confidence for Abnormal AREA OF CONFIDENCE FOR NORMAL ABNORMAL UNGERMINATED SEEDS M Abnormal B Normal C Band of Confidence for Ungerminated Seeds

23 0% 100% Ungerminated Seeds Abnormal 24 % 100% 0% 0% 100% Anormal 76 %

24 0 % A 24 % ABNORMAL UNGERMINATED SEEDS B 24 % 76 % NORMAL 0 % C 100 %

25 THE BARYCENTRIC PRESENTATION FOR SEVERAL SAMPLES A global accuracy of the laboratories

26 For a laboratory and a given sample Normal + Abnormal + Ungerminated = 100 Normal + Abnormal + Ungerminated = Normal ref + Abnormal ref + Ungerminated ref (Normal - Normal ref )+ ( Abnormal- Abnormal ref ) +(Ungerminated - Ungerminated ref ) = =0 E1 E2 E3 For a laboratory and all samples Sum (E1)+sum (E2)+Sum(E3)=0

27 -1,33% 4,61% * 1 0 % ABNORMAL UNGERMINATED SEEDS * 41 * % * 8-17 * 7-34 * 11 * 0 * 32 5,13% * 5-3,28% 0 % 3,18% NORMAL * 73

28 Examples of use of this Statistical analysis in France

29 Example on Pea

30 Analyse sample by sample

31 0% 40% 3,43% * 72 * 67* 51 UNGERMINATED SEEDS ABNORMAL 27,86% Pisum ring test 29 licensed laboratories * 14 * 1 * 120 * 130 * 58* 124* 40* 114 * 41* 2* 27* 34 * 61 * 46* 0 * 123 * 35* 29 * 33* 79* 4 * 13* 12 * 7 * 22 * 81 Comparison with general mean Sample P2 confidence level 1/ outliers 40% 0% 60% 68,71% 100% NORMAL

32 0% 44% 2,00% INATED SEEDS ABNORMAL 31,50% * 80 * 58* 22 * 3* 0* 61 * 120* 114* 1* 35* 47* 76* 67* 62* 86 * 115* 4* 13* 2* 40* 34 * 46* 27* 121* 14* 123* 78 * 119* 28* 126* 81 * 95* 72* 55* 124* 106 * 79* 10* 60 * 64* 125* 29* 7 Pisum ring test 43 laboratories Comparison with general mean NORMAL 66,50% 0% 100% Confidence level 1/ outlier Sample N P1

33 0% 39% 1,75% UNGERMINATED SEEDS ABNORMAL 26,50% * 125 * 120* 47 * 115 * 34* 55* 78* 40* 86 * 7* 76 * 124* 3* 58* 1* 35 * 106* 0* 61* 27* 2* 72 * 67* 14* 114* 46* 4* 22* 62* 79 * 80 * 13* 126* 123* 121* 60 * 81* 119* 64 * 10 * 29 * 28* 95 Pisum ring test 43 laboratories Comparison with general mean 39% 0% 61% 71,75% 100% NORMAL Confidence level 1/ outliers Sample N P2

34 0% 33% 1,25% UNGERMINATED SEEDS ABNORMAL 21,75% * 78 * 14 * 114* 13 * 0* 3* 61* 76* 4* 95 * 86 * 1* 119* 60* 126* 106* 72* 46* 62 * 7* 58* 40* 115 * 34 * 80 * 67* 35* 124* 29 * 28* 10* 47* 2 * 121 * 125* 27* 64 * 22* 120* 81* 55 33% 0% * 79 * % 100% 77,00% NORMAL Pisum ring test 43 laboratories Comparison with general mean Confidence level 1/ outliers Sample N P3

35 Analyse of all samples

36 -1,33% 3,33% 0 % UNGERMINATED SEEDS ABNORMAL 0 % * 3* 78 * 58* 76 * 86* 0* 61* 115 * 1* 47* 114* 120 * 40 * 34* 14 * 4* 35 * 13* 62 * 67* 46 * 22* 106 * 72 * 2* 125* 126 * 7* 124* 119 * 27 * 55 * 81 * 28* 80 * 123 * 10 * 79* 64 * 60 * 121* 95 9,25% -7,25% * 29-2,00% 0 % 8,58% NORMAL

37 Example on Tomato : results according different references

38 Tomato-Sample T5 5% level reference global mean: 7 outliers

39 Sample T5 1/1000 level reference global mean: 3 outliers

40 Sample T5 1/1000 level ref. global mean except 95 : 2 outlier

41 Sample T5 1/1000 level reference global lab «0» : 7 outliers

42 Tomato global mean 1% - 7 outliers -2,32% 9,14% 0 % ABNORMAL UNGERMINATED SEEDS * 95 * 1 * 93 0 % * 7 * 9 * 3 * 4 * 8 * 11 9,38% -2,56% * 0-6,82% 0 % 4,88% NORMAL

43 Tomato reference «lab 0» 1% - 9 outliers!!! -3,25% 11,70% 0 % ABNORMAL UNGERMINATED SEEDS * 95 * 1 * 93 * 7 * 9 8,45% 0,00% -8,45% 0 % 0% 3,25% * 3 * 4 * 8 * 11 * 0

44 Tomato global mean except 93 & 95 1% - 6 outliers 0 % -1,34% 9,69% ABNORMAL UNGERMINATED SEEDS * 95 * 1 * 93 * 7 * 9 10,36% -2,01% * 0-8,36% 0 % 3,34% NORMAL 0 % * 3 * 8 * 4 * 11

45 Conclusion 1 Representation : Usefulness of the barycentric representation for the technician to visualise the 3 values on one graph

46 Conclusion 2 Computer program : Usefulness of a program that allow both To edit standard tables and representations, to print reports of the referee or ring tests. To make choice of reference and confidence level.

47 Conclusion 3 Statistical evaluation based on the binomial distribution and not the variability of the data.