NF VALIDATION Validation of alternative analytical methods Application in food microbiology. Summary report

Transcription

1 ACCREDITATION N PORTEE DISPONIBLE SUR Oxoid Ltd, part of Thermo Fisher Scientific Wade Road, Basingstoke, Hampshire, RG24 8PW, UK NF VALIDATION Validation of alternative analytical methods Application in food microbiology Summary report EN ISO validation study of the BAX System PCR Assay Salmonella spp. Qualitative method This document includes 60 pages, with 5 appendixes. Only copies including the totality of this document are authorised. Competences of the laboratory are certified by COFRAC accreditation for the analyses marked with symbol. Version 0 December 4, 2014 ADRIA DEVELOPPEMENT Creac h Gwen - F QUIMPER Cedex - Tél. (33) Fax (33) adria.developpement@adria.tm.fr - Site web : ASSOCIATION LOI DE N SIRET N EXISTENCE N TVA FR

2 1 INTRODUCTION Dates of the initial validation, extension and renewal studies Alternative method Reference method 6 2 VALIDATION STUDIES RESULTS Methods comparison study Relative accuracy, relative specificity and relative sensitivity Relative detection level Inclusivity / exclusivity Inter-laboratory study Study organization Control of experimental parameters Results analyses Results interpretation Interpretation Practicability Conclusion 25 Appendix 1 Flow diagrams of the alternative method 26 Appendix 2 NF EN ISO 6579: 2002 reference method: Microbiology of food and animal feeding stuffs Horizontal method for the detection of Salmonella spp. 29 Appendix 3 Relative accuracy: raw data in French (Study realized by Institut Pasteur de Lille) 30 Appendix 4 Relative accuracy: raw data (Study realized by ADRIA Développement) 46 Appendix 5 Inclusivity / exclusivity: raw data (in French) 51 ADRIA Développement 2/60 December 4, 2014

3 Before comment Quality assurance documents related to this study can be consulted upon request from OXOID, part of Thermo Fisher Scientific. The technical protocol and the result interpretation were realized according to the EN ISO and the AFNOR technical rules. o Company: Oxoid Ltd, part of Thermo Fisher Scientific Wade Road, Basingstoke, Hampshire, RG24 8PW, UK o Expert Laboratory: ADRIA Développement ZA Creac h Gwen F QUIMPER Cedex o Studied method: BAX System PCR Assay Salmonella spp. o Validation standard: EN ISO (October 2003): Food microbiology Protocol for the validation of alternative methods o Standard method : EN ISO 6579 (2002): Horizontal method for the detection of Salmonella spp. o Scope: Food and feed products, environmental samples (excluding samples from primary production) o Certification organism: AFNOR Certification Analysis performed according to the COFRAC accreditation ADRIA Développement 3/60 December 4, 2014

4 1 INTRODUCTION 1.1 Dates of the initial validation, extension and renewal studies BAX System PCR Assay Salmonella spp performances were assessed on November 28, 2002 (certificate number QUA 18/03 11/02). Extension studies and renewal studies were performed: March 2004 Extension for two specific protocols dedicated to raw meat and dairy products (milk products excluded) Study realised by IPL May 2006 Extension for a second automate (BAX Q7) Study realised by IPL October 2006 Renewal according to the ISO standard Study realised by IPL June 2008 November 2008 May 2009 September 2010 March 2011 March 2012 November 2014 Extension for a new protocol dedicated to raw meat (seasoned or not, using MP broth) Study realised by ADRIA Développement Extension for a new protocol for raw meat samples with a short incubation time (9 h at 42 C) Study realised by ADRIA Développement Extension for a new version of the software Renewal study and extension for a new version of the kit Extension to version 2.8 of the BAX software Extension to version 2.9 of the BAX software Renewal according to the ISO standard Study realised by ADRIA Développement 1.2 Alternative method Principle The BAX system is based on the gene amplification of a Salmonella spp. specific nucleic sequence by PCR technology. The reagents necessary for the PCR reaction and for the internal control are included in the same PCR tube. ADRIA Développement 4/60 December 4, 2014

5 The BAX Q7 system is a system composed by a thermocycler and an optical module detecting the fluorescence. The software program analyses the level of fluorescence and provides results, i.e. positive or negative. Protocols (See Appendix 1) The protocols are described below: - Protocols: All products, excluding raw meat and raw poultry Pre-enrichment: 16 h - 20 h at 37 C in BPW (d 1/10) Subculture: 3 h 4 h in BHI (10 µl BPW/500 µl BHI) Raw poultry meat (non seasoned) Pre-enrichment: 16 h - 20 h at 37 C in BPW (d 1/10) Dairy products (except milk powders) Pre-enrichment: 20 h - 24 h at 42 C in BPW supplemented with novobiocin (20 mg/l) (d 1/10) Raw meat Pre-enrichment: 24 h at 42 C in MP broth Raw beef meat Pre-enrichment: 9 h 24 h at 42 C in MP broth - DNA extraction lysis: * addition of 5 l enrichment broth to 200 l lysis reagent * 30 minutes at 55 C * 10 minutes at 95 C - Amplification: * transfer 50 l of the lysate in a PCR tube * run the PCR in the automate - Detection The fluorescence is measured directly by the BAX system, which provides positive or negative results. - confirmation of positive results 1) according to the tests described in the reference method after or streaking the last enrichment broth onto selective agar plates, 2) by applying a subculture in RVS broth (24 h at 41.5 C) prior to streaking onto Brilliance TM Salmonella for raw meats. Typical colonies are then confirmed by latex test. ADRIA Développement 5/60 December 4, 2014

6 1.3 Reference method The reference method is the EN ISO 6579 (2002): Horizontal method for the detection of Salmonella spp (See Appendix 2). 2 VALIDATION STUDIES RESULTS 2.1 Methods comparison study Relative accuracy, relative specificity and relative sensitivity Accuracy is the closeness of agreement between a test result and the accepted reference value. Relative specificity is defined as the degree to which a method is affected (or not) by the other components present in a multi-component sample; that is, it is the ability of the method to measure exactly a given analyte, or its amount, within the sample without interference from non-target components such as matrix effect or background noise. Relative sensitivity is defined as the ability of the alternative method to detect two different amounts of analyte measured by the reference method within a given matrix over the whole measurement range; that is, it is the minimal quantity variation (increase of the analyte concentration x) which gives a significant variation of the measured signal (response y) Number and nature of samples In 2002, 2004 and 2006, the studies were realised by IPL (See raw data in Appendix 3). In 2008 (June and November), they were performed by ADRIA Développement (See raw data Appendix 4). The repartition per tested category and per study is provided Table 1. Analysis performed according to the COFRAC accreditation ADRIA Développement 6/60 December 4, 2014

7 Table 1 Repartition per tested category and per study Category Protocol Types Meat products BPW, h Raw meat products Raw beef meats Dairy products Fishery products and vegetables Various products Feed stuff Environmental samples MP broth, 24 h MP broth, 9 h- 24 h BPW, h + novobiacin BPW, 16 h - 20 h BPW, 16 h - 20 h BPW, 16 h - 20 h BPW, 16 h - 20 h TOTAL (June) 2008 (Nov.) (June) 2008 (Nov.) Raw meat Poultry Delicatessen Total Poultry Beef Others Total Fresh Frozen Seasoned Total Raw milk cheeses Pasteurised milk cheeses and ice creams Milks and milk powders Total Fish Raw vegetables and spices Positive samples Negative samples Total Ready-to-eat vegetables Total Egg products Pastry, chocolate Ready-to-cook or readyto-heat meals Total "Tourteaux" Flours, pellets Meat Total Process water Swabs, wipes Dusts, wastes Total Artificial contamination of samples Artificial contaminations were done by using injured bacterial suspensions. The injured treatment and the efficiency were determined according to EN ISO and AFNOR technical rules. The percentage of naturally and artificially contaminated samples per study is presented below (Table 2). ADRIA Développement 7/60 December 4, 2014

8 Table 2 - Percentage of naturally and artificially contaminated samples per study Year of study Number of positive Percentage of Percentage of Number of Positive results observed artificially naturally contaminated results among the contaminated contaminated samples observed contaminated samples samples samples % 50.0 % 2008 (June) % 64.0 % 2008 (November) % 8.8 % All validation studies % 46.3 % Test results The results are reported below. Table 3 Paired results of the reference and alternative methods All products General protocol (enrichment broth: BPW or BPW + novobiocin at 37 C) Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 186 Negative deviation (A-/R+) ND = 6 (PPND = 0) Reference method negative (R-) Positive deviation (R-/A+) PD = 1 Negative agreement (A-/R-) NA = 232 (PPNA = 1) PA: positive agreement PD: positive deviation NA: negative agreement ND: negation deviation PPNA: positive presumptive negative agreement ADRIA Développement 8/60 December 4, 2014

9 Table 4 Paired results of the reference and alternative methods Meat products Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 37 Negative deviation (A-/R+) ND = 2 Reference method negative (R-) Positive deviation (R-/A+) PD = 0 Negative agreement (A-/R-) NA = 42 Table 5 Paired results of the reference and alternative methods Dairy products Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 31 Negative deviation (A-/R+) ND = 0 Reference method negative (R-) Positive deviation (R-/A+) PD = 1 Negative agreement (A-/R-) NA = 33 Table 6 Paired results of the reference and alternative methods Fishery products and vegetables Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 28 Negative deviation (A-/R+) ND = 2 Reference method negative (R-) Positive deviation (R-/A+) PD = 0 Negative agreement (A-/R-) NA = 45 Table 7 Paired results of the reference and alternative methods Various products Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 32 Negative deviation (A-/R+) ND = 0 Reference method negative (R-) Positive deviation (R-/A+) PD = 0 Negative agreement (A-/R-) NA = 47 ADRIA Développement 9/60 December 4, 2014

10 Table 8 Paired results of the reference and alternative methods Feed stuff Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 29 Negative deviation (A-/R+) ND = 1 Reference method negative (R-) Positive deviation (R-/A+) PD = 0 Negative agreement (A-/R-) NA = 35 Table 9 Paired results of the reference and alternative methods Environmental samples Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 29 Negative deviation (A-/R+) ND = 1 Reference method negative (R-) Positive deviation (R-/A+) PD = 0 Negative agreement (A-/R-) NA = 30 Table 10 Paired results of the reference and alternative methods Specific protocols (enrichment broth: MP 24 h at 42 C) Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 21 Negative deviation (A-/R+) ND = 7 (PPND = 0) Reference method negative (R-) Positive deviation (R-/A+) PD = 5 Negative agreement (A-/R-) NA = 32 (PPNA = 0) ADRIA Développement 10/60 December 4, 2014

11 Table 11 Paired results of the reference and alternative methods Raw beef meats (enrichment broth: MP 9 24 h at 42 C) Incubation Responses Reference method Reference method time positive (R+) negative (R-) Alternative Positive agreement (A+/R+) Positive deviation (R-/A+) 9 h method positive (A+) Alternative PA = 25 Negative deviation (A-/R+) PD = 4 Negative agreement (A-/R-) method negative (A-) ND = 4 (PPND = 0) NA = 31 (PPNA = 0) Alternative Positive agreement (A+/R+) Positive deviation (R-/A+) 24 h method positive (A+) Alternative PA = 28 Negative deviation (A-/R+) PD = 5 Negative agreement (A-/R-) method negative (A-) ND = 1 (PPND = 0) NA = 30 (PPNA = 0) Table 12 Calculation of relative accuracy (AC), relative sensitivity (SE) and relative specificity (SP) Category PA NA ND PD N Relative accuracy AC (%) [100x(PA+NA])/N] N+ PA + ND Relative sensitivity SE (%) [100xPA]/N+] N- NA + PD Relative specificity SP (%) [100xNA]/N-] Meat products Dairy products Fishery products and vegetables Various products Feed stuff Environmental samples All products Category PA NA ND PD N Relative accuracy AC (%) [100x(PA+NA])/N] N+ PA + ND Relative sensitivity SE (%) [100xPA]/N+] N- NA + PD Relative specificity SP (%) [100xNA]/N-] Raw meats 24 h Category PA NA ND PD N Beef meat Relative accuracy AC (%) [100x(PA+NA])/N] N+ PA + ND Relative sensitivity SE (%) [100xPA]/N+] N- NA + PD Relative specificity SP (%) [100xNA]/N-] 9 h h PA = positive agreement (R+/A+) PD =positive deviation (R-/A+) NA =negative agreement (R-/A-) ND = negative deviation (A-/R+) ADRIA Développement 11/60 December 4, 2014

12 Calculation of relative accuracy (AC), relative sensitivity (SE) and relative specificity (SP) For the alternative method, the percentage values for three criteria according to the ISO are the following: General protocol Raw meats Beef meats 24 h 9 h 24 h Relative accuracy 98.4 % 81.5 % 87.5 % 90.6 % Relative specificity 99.6 % 86.5 % 88.6 % 85.7 % Relative sensitivity 96.9 % 75.0 % 86.2 % 96.6 % Sensitivity of both methods, when the positive deviations of the alternative method are considered, is presented below: Alternative method (PA + PD) / PA + PD + ND) (SE) Reference method (PA + ND) / PA + PD + ND) (SE) General protocol 96.9 % 99.5 % Specific protocol Raw meats 78.8 % 84.7 % Specific protocol Beef meats 9 h 87.9 % 87.9 % 24 h 97.1 % 85.3 % Analysis of discordant results The analysis of discordant results is presented per protocol in the following table: Y = ND + PD m M Conclusion General protocol Y = = The methods are not different at < Specific protocol Raw meats Y = = The methods are not different at < Specific protocol Beef meats 9 h Y = = The methods are not different at < h Y = = The methods are not different at < The BAX System PCR Assay Salmonella spp. shows satisfying relative accuracy, relative specificity and sensitivity. ADRIA Développement 12/60 December 4, 2014

13 Confirmation protocols For the general protocol, the last enrichment broth (BPW, BPW + novobiocin or BHI) was streaked onto selective agar plates as detailed in ISO Any typical colonies were then confirmed by the tests described in the ISO method. For protocols dedicated to raw meats and raw beef meats enriched with BAX System MP Broth, the positive samples were confirmed by subculture of the MP Broth into RVS and then by streaking onto Oxoid Brilliance TM Salmonella Agar. Any typical colonies were then confirmed by a latex test and the tests described in the ISO method Relative detection level The relative detection level is the smallest number of culturable micro-organisms that can be detected in the sample in 50% of occasions by the alternative and reference methods Matrices The objective of this study is (i) to determine the target species minimal quantity that can be detected in food matrices, (ii) to compare both method results. Detection limits have been defined by analyzing the different matrix/strain pairs. Four levels were tested. Six replicates of each combination were prepared. The following matrices were tested: - ground poultry meat inoculated with Salmonella Hadar, - raw milk inoculated with Salmonella Typhimurium, - whole egg product inoculated with Salmonella Enteritidis, - fish fillet inoculated with Salmonella Virchow, - pâté for dog inoculated with Salmonella Senftenberg, - process water inoculated with Salmonella Infantis. - ground beef / Salmonella Infantis. ADRIA Développement 13/60 December 4, 2014

14 Contamination protocol Contaminations and enumerations were realized according to the AFNOR technical rules (protocol for low level inoculations). The contamination levels are presented below: - level 1: 0 UFC/g or /ml - level 2: level necessary to obtain 0 to 50% positives, - level 3: level necessary to obtain 50 to 75% positives, - level 4: level necessary to obtain 100% positives. The samples were analyzed by both methods, and the background microflora was enumerated Results Detection levels are presented in Table 13. Table 13 Relative detection level results Strain / matrix pairs Relative detection level (CFU / 25 g or 25 ml) according to Spearman-Kärber test Reference method Alternative method Ground poultry / Salmonella Hadar 0.3 [0.2 ; 0.5] 0.4 [0.3 ; 0.6] Raw milk / Salmonella Typhimurium 0.5 [0.3 ; 1.0] 0.6 [0.3 ; 1.0] Whole egg product / Salmonella Enteritidis 0.4 [0.2 ; 0.8] 0.4 [0.2 ; 0.8] Fish fillet / Salmonella Virchow 0.3 [0.2 ; 0.4] 0.3 [0.2 ; 0.4] Pâté for dog / Salmonella Senftenberg 0.4 [0.3 ; 0.7] 0.4 [0.3 ; 0.7] Process water / Salmonella Infantis 0.5 [0.3 ; 0.9] 0.54 [0.3 ; 0.9] Ground beef / Salmonella Infantis - BAX 9 h 0,4 [0,2 ; 1,1] 0,7 [0,4 ; 1,3] Ground beef / Salmonella Infantis - BAX 24 h 0,3 [0,1 ; 0,8] Conclusion The detection levels are comprised between: and 1.3 CFU/25 g or 25 ml for the reference method, and 1.1 CFU/25 g or 25 ml for the alternative method. ADRIA Développement 14/60 December 4, 2014

15 2.1.3 Inclusivity / exclusivity Inclusivity is the ability of the alternative method to detect the target analyte from a wide range of strains. Exclusivity is the lack of interference from a relevant range of non-target strains of the alternative method. The inclusivity and the exclusivity of the method are defined by analysis, respectively of 50 positive strains and 30 negative strains Test protocols Inclusivity - Study performed in 2002: the strains were grown in BHI broth and the BAX Salmonella test applied. - Studies performed in 2003 and 2005: the strains were grown in BHI for 24 h at 35 C; the BAX Salmonella tests were then applied on 1/10 diluted cultures (2003) or pure culture (2005). - Study performed in 2008: the strains were grown in BHI and diluted in order to inoculate, between 10 and 100 cells/225 ml, MP broth and then incubated for 9 h and 24 h at 42 C. The alternative method protocol was then applied. In order to be in agreement with the AFNOR technical rules, 5 additional strains were tested for the renewal project in 2014: they all gave positive PCR results. Exclusivity The studies were performed in 2002, 2003 and 2005 with the protocols as described for the inclusivity studies Results The raw data (in French) are given Appendix 5. The results observed for the different studies are summarised Table 14. ADRIA Développement 15/60 December 4, 2014

16 Table 14 Inclusivity and exclusivity study results Inclusivity Exclusivity Year of realisation Number of Number of Number of Number of Number of strains tested positive results negative results strains tested negative results obtained obtained obtained h / 24 h / / / (2) Salmonella gallinarum Ad 300 Salmonella gallinarum 1 The BAX System PCR Assay Salmonella spp. is specific and selective. 2.2 Inter-laboratory study The general protocol was tested in the inter-laboratory study (the study was performed in 2006) Study organization 12 laboratories received samples. The matrix used for the study was a delicatessen (pâté) in order to test the general protocol. The strain used for inoculation was Salmonella Typhimurium isolated from pork liver. 24 samples were prepared per laboratory. Each sample was divided in 3 levels of contamination, with 8 samples per level. ADRIA Développement 16/60 December 4, 2014

17 2.2.2 Control of experimental parameters Contamination levels obtained after artificial contamination The contamination levels and the confidence intervals were: Level Level 0 (L0) Low level (L1) High level (L2) Samples Theoretical target level (b/25 g) True level (b/25 g sample) Low limit / 25 g sample High limit / 25 g sample 0 0 / / Logistic conditions Temperature conditions are given below: Laboratory Temperature measured at receipt Temperature measured by the probe Comment A 7.1 C 4.9 C B 9.5 C 3.9 C C 8.6 C -0.1 C D 10.0 C 5.9 C E 7.5 C 6.9 C F 10.7 C 3.4 C G 3.3 C 3.4 C H 7.8 C 0.0 C I 7.0 C 3.9 C J 1.3 C 4.0 C K 4.0 C Not received Delivery at Day 2 L 8.0 C 5.4 C Among the 12 laboratories, one received the samples at Day 2 (Lab K) and did not realise the analyses. Labs B, C, D and F measured temperature at receipt above 8.5 C but the register curves show clearly that the temperatures were correct Conclusion Due to the delivery conditions, the results of 11 laboratories were kept for interpretations. ADRIA Développement 17/60 December 4, 2014

18 2.2.3 Results analyses Results obtained by collaborators The following tables give a summary of the collaborators results. Table 15 Positive results obtained with the reference method Contamination Level L0 L1 L2 Labs Number of Number of Number of Obtained Obtained Obtained samples samples samples A B C D E F G H I J L Total (a) (b) (c) Positive results obtained with the alternative method Contamination level L0 L1 L2 Labs Number of Number of Number of Obtained Obtained Obtained samples samples samples A B C D E F G H I J L Total (a) (b) (c) (a) : False positive (b) : True positive result observed for Level 1 (c) : True positive result observed for Level 2 ADRIA Développement 18/60 December 4, 2014

19 The results of the reference method and of the alternative method were in agreement with those expected for 10 labs. Lab B found one control sample positive with the BAX Salmonella test, the confirmatory tests concluded to the absence of the target in the enrichment broth. The lysis and PCR tests were repeated on the BHI broth after one night storage at 2 8 C and the test was also negative. A cross contamination probably occurred during the lysis or the PCR step. Lab E found a control sample (E16) positive with the reference method, but only one colony was isolated on selective plate. This Lab suspected a cross contamination, but the result was considered as positive. The same result was observed for Lab J on sample J Results interpretation Specificity and sensitivity for each method For the L0 level and for each method, specificity percentages are calculated according to: FP SP 1 x 100% N with : N- = total number of all L0 assays FP = number of false positive results For each contamination level and each method, the sensitivity percentages are calculated according to: TP SE x 100% N with : N+ = total number of all L1 or L2 assays TP = number of true positive results Results are reported in Table 16. ADRIA Développement 19/60 December 4, 2014

20 Table 16 Interpretation Level Reference method Alternative method SP/SE % LCL% SP/SE % LCL% L0 (SP) L1 (SE) L2 (SE) L1+L2 (SE) LCL: confidence interval Relative accuracy (AC) Results for all levels are given below: Table 17 - Paired results of the alternative and reference methods Alternative method Reference method + - Total + PA = 176 PD = ND = 2 NA = 86 (PPNA = 1) 88 Total N+ = 178 N- = 86 N =264 Relative accuracy (AC) (in %) is calculated according to: ( PA NA) AC x 100% N with : N = number of samples analysed PA = number of positive agreement NA = number of negative agreement The relative accuracy is 99.2 % ADRIA Développement 20/60 December 4, 2014

21 2.2.5 Interpretation Comparison of the relative accuracy, specificity and sensitivity values The values obtained for the two parts of the validation study (comparative and inter-laboratory studies) are reported in Table 18. Table 18 - Alternative method values calculated during the comparative and inter-laboratory studies Inter-laboratory study Method comparison study Relative accuracy (AC) 99.2 % 98.4 % Sensitivity (SE) % 99.6 % Specificity (SP) % 96.9 % The AFNOR Technical Committee requests the sensitivity of the two methods to be recalculated considering all the confirmed positives (true positive results): Alternative method Reference method (PA + PD) / (PA + PD + ND) = % (PA + ND) / (PA + PD + ND) = 99.2% Accordance (DA) Accordance values for both methods are: Table 19 Interpretation Level Reference method (DA) Alternative method (DA) L % % L % % L % % ADRIA Développement 21/60 December 4, 2014

22 Concordance Both methods concordance values are: Table 20 Interpretation Level Reference method Alternative method L % % L % % L % % Odds Ratio (COR) The odds ratio value is determined according to: Both method odds ratio values are: Accordance x 100 condorcance COR Concordance x (100 accordance ) Table 21 Interpretation Level Reference method (COR) Alternative method (COR) L L L ADRIA Développement 22/60 December 4, 2014

23 2.3 Practicability Practicability is studied according to the criteria defined by the AFNOR technical rules, by comparing the EN ISO reference method to the BAX System PCR Assay Salmonella spp. method performances. 1. Packaging 2. Reagent volumes 3. Storage conditions 4. Modalities of use after first use 5. Required equipment 6. Ready-to-use reagents or requiring reconstitution The BAX System PCR Assay Salmonella spp kit contains the reagents required for 96 analyses: - PCR tubes with tablets composed of strips (8 wells) - caps - lysis reagent (2 x 12 ml) and protease (400 µl) The storage temperature is between 2 to 8 C. Expiration date is shown on the kit package on the different reagent vials. Lysis reagent + protease, once reconstituted, should be stored 15 days at 2-8 C Among the required equipment, - 1 air incubator at 37 C ± 1 C - 3 heating blocks at 37 C and 95 C - BAX or BAX Q7 All the reagents are ready-to-use, except the lysis reagent 7. Training For an operator trained in standard techniques of microbiology, 2 or 3 days training are required. 8. Workflow study For an operator trained in PCR techniques, one day training is required. Steps Average time for a sample ISO 6579 standard (min) Average time for 48 samples (min) BAX method ISO 6579 BAX method Sampling, stomaching Subculture in - RVS and MKTTn - BHI 3 BAX PCR test / 1 / 45 Streaking enrichment broth onto selective agar plates 1 10 / 150 / 20 min 9 min 7 min 30 4 min 30 In case of positive samples, time for confirmation has to be added. The time needed to confirm one colony is estimated to 5 minutes. The major interest of the method is to screen the negative samples ADRIA Développement 23/60 December 4, 2014

24 9. Time to result Negative samples Step ISO 6579 General protocol (19 24 h) BAX method Short protocol (9 h) Sampling, enrichment step Day 0 Day 0 Day 0 Subcultures (RVS/MKTTn, BHI) Day 1 Day 1 / BAX PCR test / Day 1 Day 0 Streaking onto selective agar plates Day 2 / / Plates reading Day 3 / / Negative results obtention Day 3 Day 1 Day 0 Presumptive samples or and positive samples Step ISO 6579 General protocol (19 24 h) BAX method Short protocol (9 h) Sampling, enrichment step Day 0 Day 0 Day 0 Subcultures (RVS/MKTTn, BHI) Day 1 Day 1 / BAX PCR test / Day 1 Day 0 Streaking onto selective agar plates 10. Technician qualification Technician in food microbiology Common step with reference method 12.Traceability 13.Maintenance Day 2 Day 2 Day 0 Plates reading Day 3 Day 3 Day 1 Confirmatory tests Day 5 Day 3 Day 1 Positive results obtention Day 5 Day 3 Day 1 Pre-enrichment step (16 20 h at 37 C in BPW) All the results are saved in an electronic file format. The user s guide explains some possibly encountered problems ADRIA Développement 24/60 December 4, 2014

25 2.4 Conclusion The methods comparative study conclusions are: The BAX System PCR Assay Salmonella spp. shows satisfying relative accuracy, relative specificity and sensitivity. The detection levels are comprised between: and 1.3 CFU/25 g or 25 ml for the reference method, and 1.1 CFU/25 g or 25 ml for the alternative method. The BAX System PCR Assay Salmonella spp. is specific and selective. The inter-laboratory study conclusions are: The BAX System PCR Assay Salmonella spp. and the reference method show equivalent performances (accordance, concordance, odds ratio). ADRIA Développement 25/60 December 4, 2014

26 Appendix 1 Flow diagrams of the alternative method General protocol (raw meats and raw poultry meats excluded) 25 g ml BPW h at 37 C ± 1 C Transfer 10 µl into 500 µl BHI Incubation for 3 h at 37 C ± 1 C Lysis PCR Streaking onto selective agar plates Tests described in the ISO 6579 standard Protocol for raw meats and raw poultry meats 25 g ml BPW h at 37 C ± 1 C Lysis PCR Streaking onto selective agar plates Tests described in the ISO 6579 standard ADRIA Développement 26/60 December 4, 2014

27 Protocol for dairy products (milk powder excluded) 25 g ml BPW + 20 mg/l novobiocin h at 42 C ± 1 C Lysis PCR Confirmation: 0.1 ml enrichment broth + 10 ml/rvs 24 h ± 2h at 41.5 C Streaking onto selective agar plates Tests described in the ISO 6579 standard Protocol for raw meats 25 g ml MP broth 24 h at 42 C ± 1 C Lysis PCR Confirmation: subculture in RVS broth for 24 h ± 3 h at 41.5 C ± 1 C Streaking onto Brilliance TM Salmonella Latex tests or tests described in the ISO 6579 standard ADRIA Développement 27/60 December 4, 2014

28 Protocol for raw beef meats 25 g ml MP broth (pre-warmed) 9-24 h at 42 C ± 1 C Lysis PCR Confirmation: subculture in RVS broth for 24 h ± 3 h at 41.5 C ± 1 C Streaking onto Brilliance TM Salmonella Latex tests or tests described in the ISO 6579 standard ADRIA Développement 28/60 December 4, 2014

29 Appendix 2 NF EN ISO 6579: 2002 reference method: Microbiology of food and animal feeding stuffs Horizontal method for the detection of Salmonella spp. 25 g of sample ml BPW Incubation 18 h 2 h at 37 C 1 C 0.1 ml BPW 1 ml BPW 10 ml RVS 10 ml MKTTn Incubation 24 h 3 h at 41.5 C 1 C Incubation 24 h 3 h at 37 C 1 C Streak on XLD and a second medium Incubation 24 h 3 h at 37 C 1 C Streak 1 characteristic colony onto Nutrient agar (Take 4 other colonies if the first one is negative) Incubation 24 h 3 h at 37 C 1 C Biochemical and serological confirmation ADRIA Développement 29/60 December 4, 2014

30 Appendix 3 Relative accuracy: raw data in French (Study realized by Institut Pasteur de Lille) ADRIA Développement 30/60 December 4, 2014

31 Meat products ADRIA Développement 31/60 December 4, 2014

32 Meat products ADRIA Développement 32/60 December 4, 2014

33 Meat products ADRIA Développement 33/60 December 4, 2014

34 Dairy products ADRIA Développement 34/60 December 4, 2014

35 Dairy products ADRIA Développement 35/60 December 4, 2014

36 Fishery products and vegetables ADRIA Développement 36/60 December 4, 2014

37 Fishery products and vegetables ADRIA Développement 37/60 December 4, 2014

38 Fishery products and vegetables ADRIA Développement 38/60 December 4, 2014

39 Various products ADRIA Développement 39/60 December 4, 2014

40 Various products ADRIA Développement 40/60 December 4, 2014

41 Various products ADRIA Développement 41/60 December 4, 2014

42 Feed stuff ADRIA Développement 42/60 December 4, 2014

43 Feed stuff ADRIA Développement 43/60 December 4, 2014

44 Environmental samples ADRIA Développement 44/60 December 4, 2014

45 Environmental samples ADRIA Développement 45/60 December 4, 2014

46 Appendix 4 Relative accuracy: raw data (Study realized by ADRIA Développement) June 2008 Bold typing: artificial contaminations d: doubtful colonies NC : non typical colonies on nutrient agar ni: non isolated colonies PD: positive deviation PA: positive agreement ND: negative deviation NA: negative agreement Sample N Product (French name) Artificial contaminations Strain Origin Injury applied Injury evaluation Inoculation level (CFU/sample) Global result Reference method ISO 6579 RVS MKTTn Result XLD Hektoen XLD Hektoen PCR BAX Salmonella MP method Incubation for 24h a 42 C Confirmation Reference Result OSCMII Latex tests 257 Paupiette de veau / / / / / ni(citrobacter freundii) - - / / / - NA 258 Cuisse de poule / / / / / / / - ND 259 Pieds arrière de porc / / / / / d (Providencia) - - / / / - NA 260 Filet de poule / / / / / - +d(citrobacter youngae) / / / - NA 261 Poule / / / / / ni PA 262 VSM volaille / / / / / PA 263 Poule / / / / / ni +(Citrobacter freundii) Agreement PA 264 Jambon frais / / / / / - + +dni +d +d / / - NA 265 Poule / / / / / ?ni PA 266 Viande de porc hachée / / / / / / / / - NA 267 Jambon frais / / / / / / / / - NA 268 Cuisse de poule / / / / / / / / - NA 269 Morceaux de poule avec peau / / / / / PA 270 Paupiette e veau / / / / / + + +ni +ni +ni PA 271 Cuisse de poule / / / / / PA 272 Cuisse de poule / / / / / PA 273 Paupiette de veau / / / / / - +ni(citrobacter freundii) / / / - NA 274 Paupiette de veau / / / / / + +ni + +ni +ni / / - ND 275 Viande blanche / / / / / d / / / - NA 276 Lapin au paprika / / / / / / / / - NA 289 Steak haché frais Salmonella Newport 586 Beef carcass -20 C 2, ni PA 290 Bavette Salmonella Newport 586 Carcasse de bœuf -20 C 2, PA 291 Steak haché frais Salmonella Newport 586 Carcasse de bœuf -20 C 2, PD 292 Tranche à bifteck Salmonella Infantis 128 Ground beef -20 C >2,13 2, PD 293 Viande hachée Salmonella Infantis 128 Ground beef -20 C >2,13 2, PD 294 Jarret de mouton avec os Salmonella Typhimurium ST391 Environmental sample -20 C 0,51 1, PA 295 Côte d'agneau Salmonella Typhimurium ST391 Environmental sample -20 C 0,51 1, PA 296 Poitrine d'agneau Salmonella Typhimurium ST391 Environmental sample -20 C 0,51 1, PA 297 Rumsteak en tournedos Salmonella Infantis 128 Ground beef -20 C >2,13 2, ni / / - ND 298 Tranche à bifteck Salmonella Infantis 128 Ground beef -20 C >2,13 2, PD 342 Paupiette de veau / / / / / / / - ND 343 Paupiette de veau / / / / / ND 344 Jambon frais / / / / / / / / - NA 345 Blanc de poule sans peau / / / / / / / / - NA 346 Blanc de poule sans peau / / / / / / / - ND 347 Blanc de poule sans peau / / / / / / / / - NA 348 Viande blanche / / / / / / / / - NA 349 Mêlée de porc / / / / / PA Analyses performed according to the COFRAC accreditation ADRIA Développement 46/60 December 4, 2014

47 Sample N Product (French name) Artificial contaminations Strain Origin Injury applied Injury evaluation Inoculation level (CFU/sample) Global result Reference method ISO 6579 RVS MKTTn Result XLD Hektoen XLD Hektoen PCR BAX Salmonella MP method Incubation for 24h a 42 C Confirmation Reference Result OSCMII Latex tests 350 Morceaux de poule avec peau / / / / / / / / - NA 351 Morceaux de poule avec peau / / / / / ND 352 Poule / / / / / PA 431 Crépinettes / / / - NA 432 Crépinettes PA 433 Poitrine de porc / / / - NA 434 Poitrine de porc / / / - NA 435 Poitrine de porc d(nc) - - / / / - NA 436 Langue de porc / / / - NA 437 Langue de porc (Citrobacter koseri) Agreement - - / / / - NA 438 Langue de porc PA 439 Steak haché Cross contamination with carcass wipe - - +(NC) / / / - NA 440 Steak haché Cross contamination with carcass wipe - - +(NC) / / / - NA 441 Steak haché Cross contamination with carcass wipe - - +(NC) / / / - NA 442 Steak haché Cross contamination with carcass wipe - +(NC) - - +(NC) - - / / / - NA 443 Steak haché Cross contamination with carcass wipe - +(NC) +(NC) / / / - NA 444 Steak haché Cross contamination with carcass wipe - - +(NC) / / / - NA 445 Steak haché Cross contamination with carcass wipe / / / - NA 543 Boulettes au bœuf Salmonella Bredeney 396 Beef 2 days at 4 C / ,73 2 days at -20 C (13,2) PA 544 Steak haché Salmonella Typhimmurium 2 days at 4 C / Beef 0,62 AOOCO60 2 days at -20 C (1,8) PA 545 Steak grill oignons Salmonella Bredeney 396 Beef -20 C 0, (4,2) PA 546 Steak haché pur bœuf Salmonella Newport 586 Beef -20 C 1, (2,4) PD 547 Viande bovine à Salmonella Dublin Ad 529 Beef -20 C 0,63 bourguignon (4) PA 579 VSM volaille / / / - NA 580 VSM volaille / / / - NA 581 Peau de poule / / / - NA 582 Cuisse de poule / / / - NA OXOID ADRIA Développement 47/60 December 4, 2014

48 November 2008 Bold typing: artificial contaminations d: doubtful colonies NC : non typical colonies on nutrient agar ni: non isolated colonies PD: positive deviation PA: positive agreement ND: negative deviation NA: negative agreement Sample N Artificial contaminations Reference method ISO 6579 RVS MKTTn Bax Salmonella MP method Incubation 9h-42 c Incubation 24h-42 c Confirmation Confirmation OSCMII Latex Result Agreement PCR OSCMII Latex Result Product Inoculation Global (French name) Injury Injury level result Result Strain Origin XLD Hektoen XLD Hektoen PCR applied evaluation (CFU/ Reference Reference sample) tests tests 289 Steak haché frais Salmonella Newport 586 Beef carcass -20 C 2, ni PA PA 290 Bavette Salmonella Newport 586 Beef carcass -20 C 2, ND PA 291 Steak haché frais Salmonella Newport 586 Beef carcass -20 C 2, PD PD 292 Tranche à bifteck Salmonella Infantis 128 Ground beef -20 C >2,13 2, PD PD 293 Viande hachée Salmonella Infantis 128 Ground beef -20 C >2,13 2, PD PD 297 Rumsteak en tournedos Salmonella Infantis 128 Ground beef -20 C >2,13 2, ni / / - ND - - / / - ND 298 Tranche à bifteck Salmonella Infantis 128 Ground beef -20 C >2,13 2, PD PD 439 Steak haché Cross contamination with carcass wipe - - +(NC) / / - NA - - / / - NA 440 Steak haché Cross contamination with carcass wipe - - +(NC) / / - NA - - / / - NA 441 Steak haché Cross contamination with carcass wipe - - +(NC) / / - NA - - / / - NA 442 Steak haché Cross contamination with carcass wipe - + (NC) - - +(NC) / / - NA - - / / - NA 443 Steak haché Cross contamination with carcass wipe - + (NC) +(NC) / / - NA - - / / - NA 444 Steak haché Cross contamination with carcass wipe - - +(NC) / / - NA - - / / - NA 445 Steak haché Cross contamination with carcass wipe / / - NA - - / / - NA 543 Boulettes au bœuf Salmonella Bredeney 396 Beef 544 Steak haché Salmonella Typhimmurium AOOCO60 Beef 2 days at 4 C / 2 days at -20 C 2 days at 4 C / 2 days at -20 C 0,73 0, (13,2) (1,8) ND PA PA PA 545 Steak grill oignons Salmonella Bredeney 396 Beef -20 c 0,79 surgelé (4,2) ND PA 546 Steak haché pur bœuf Salmonella Newport 586 Beef -20 c 1, (2,4) NA PD 547 Viande bovine à Salmonella dublin Ad 529 Beef -20 c 0,63 bourguignon (4) PA PA 1281 Steak de bœuf mariné PA PA 1282 Hampe (1) col + 2 col + 2 col + 2 col PA PA 1283 Hampe (2) col +/- + NC PA PA 1284 Carpaccio de bœuf Salmonella Panama 195 Ground beef -20 C 0,42 surgelé 5(5,2) PA PA 1285 Viande hachée pur Salmonella Panama 195 Ground beef -20 C 0,42 bœuf surgelé 5(5,2) PA PA 1286 Tranche de tournedos Salmonella Panama 195 Ground beef -20 C 0,42 surgelé 5(5,2) PA PA 1287 Macreuse à braiser Salmonella Panama 195 Ground beef -20 C 0, (5,2) PA PA 1288 Viande hachée de bœuf surgelée / / / - NA - / / / - NA 1289 Rumsteak à griller / / / - NA - / / / - NA Agreement Analyses performed according to the COFRAC accreditation ADRIA Développement 48/60 December 4, 2014

49 Sample N Product (French name) Strain Artificial contaminations Origin Injury applied Injury evaluation Inoculation level (CFU/ sample) 1290 Viande bovine Aiguillette Viande bovine bavette surgelé Viande bovine bourguignon collier poitrine Viande bovine rumsteak à griller Salmonella Panama 195 Salmonella Panama 195 Ground beef Ground beef 1294 Gîte de noix à bifteak Salmonella Panama 195 Ground beef Viande bovine bavette d'aloyau Steak hachée frais 5% MG Steak hachée frais 15% MG Viande bovine tranche à fondue Viande bovine à bourguignon Viande bovine tranche à fondue Macreuse à braiser surgelé Viande hachée pur bœuf surgelé Salmonella Newport 586 Salmonella Newport 586 Salmonella Newport 586 Salmonella Panama 195 Salmonella Newport 586 Beef carcass Beef carcass Beef carcass Ground beef Beef carcass 1 month at 4 C 1 month at 4 C 1 month at 4 C 1 month at 4 C 1 month at 4 C 1 month at 4 C 1 month at 4 C 1 month at 4 C Global result Reference method ISO 6579 RVS MKTTn Result XLD Hektoen XLD Hektoen PCR +?3 col Hafnia alvei Bax Salmonella MP method Incubation 9h-42 c Incubation 24h-42 c Confirmation Confirmation OSCMII Latex Reference Result Agreement PCR Reference Result OSCMII Latex tests tests ADRIA Développement 49/60 December 4, 2014 OXOID Agreement / / / - NA - / / / - NA / / / - NA - / / / - NA 0, (1) PA PA 0, (1) PA PA 0, (1) PA PA 0, (1) PA PA 0, (1) PA PA 0, (1) PA PA 0, (1) PA PA 0, (1) PA PA /-Hafnia alvei / / / - NA - / / / - NA / / / - NA - / / / - NA /- Providencia stuartii + Citrobacter youngae / / / - NA - / / / - NA Steak hachée frais pur NC - Citrobacter - - / / / - NA - / / / - NA bœuf 5%MG youngae 1304 Gîte de noix à bifteak / / / - NA - / / / - NA 1306 Gîte de noix à bifteak / / / - NA - / / / - NA 1307 Viande bovine bavette d'aloyau / / / - NA - / / / - NA 1308 Viande bovine basse côte à griller / / / - NA - / / / - NA 1309 Viande bovine tranche à fondue / / / - NA - / / / - NA 1310 Viande bovine tranche en tournedos / / / - NA - / / / - NA 1311 Plat de côte avec os pour pot-au-feu / / / - NA - / / / - NA 1312 Gîte de noix à bifteak Salmonella Panama 8 Ground beef -20 C 0, (4,2) PA PA 1313 Viande bovine Salmonella Panama 8 Ground beef -20 C 0,57 rumsteak à griller 5(4,2) PA PA 1314 Viande hachée de bœuf Salmonella Typhimurium surgelée AOOCO60 Ground beef -20 C 0, (5) PA PA 1315 Boulettes au bœuf Salmonella Typhimurium surgelé AOOCO60 Ground beef -20 C 0, (5) PA PA 1316 Viande bovine à bourguignon / / / - NA - / / / - NA 1317 Viande bovine tranche à fondue / / / - NA - / / / - NA 1318 Steak haché 5%MG surgelé / / / - NA - / / / - NA

50 Sample N Product (French name) Strain Artificial contaminations Origin Injury applied Injury evaluation Inoculation level (CFU/ sample) Global result Reference method ISO 6579 RVS MKTTn Result XLD Hektoen XLD Hektoen PCR Bax Salmonella MP method Incubation 9h-42 c Incubation 24h-42 c Confirmation Confirmation OSCMII Latex Reference Result Agreement PCR Reference Result OSCMII Latex tests tests 1319 Steak haché 15%MG surgelé / / / - NA - / / / - NA 1320 Haché pur bœuf Salmonella Infantis 128 Ground beef -20 C >1,70 surgelé 9(7,8) PA PA 1321 Steak haché surgelé 100% Charolais / / / - NA - / / / - NA 1322 Steak haché extra Salmonella Infantis 128 Ground beef -20 C >1,70 moelleux surgelé 9(7,8) / PA PA 1323 Préparation à 50% de viande hachée de bœuf / / / - NA - / / / - NA 1324 Carpaccio de bœuf Salmonella Infantis 128 Ground beef -20 C >1,70 surgelé 9(7,8) PA PA 1325 Plat de côte avec os pour pot-au-feu - - +/- - +/- - - / / / - NA - / / / - NA 1494 Gîte de noix à bifteak / / / - NA - / / / - NA OXOID Agreement ADRIA Développement 50/60 December 4, 2014

51 Appendix 5 Inclusivity / exclusivity: raw data (in French) Inclusivity ADRIA Développement 51/60 December 4, 2014

52 Inclusivity ADRIA Développement 52/60 December 4, 2014

53 Inclusivity ADRIA Développement 53/60 December 4, 2014

54 Inclusivity ADRIA Développement 54/60 December 4, 2014

55 Inclusivity ADRIA Développement 55/60 December 4, 2014

56 Inclusivity (ADRIA Développement 2008) N Strain Reference Origin Inoculation level cfu/225ml PCR BAX Brilliance TM Salmonella Latex 1. Salmonella Anatum Ad 298 Dairy product Salmonella diarizonae 47:Iv:z53 Ad 478 Seafood Salmonella Bongori Ad 599 Environmental sample pale colonies + (weak) 4. Salmonella Bovismorficans 132 Meat product Salmonella Braenderup 111 Meat product Salmonella Brando 596 Meat product Salmonella Bredeney 396 / Salmonella Cerro Ad 689 Pet food Salmonella Derby 18 Meat product Salmonella Diarizonae Ad 594 Meat product Salmonella Diarizonae Ad 595 Dairy product (weak) 12. Salmonella Dublin Ad 529 Meat product Salmonella Enteritidis 657 Egg product Salmonella Gallinarum Ad 300 Environmental - (9h)/ + microscopic 14 sample +(24h) colonies Salmonella Hadar Meat product Salmonella Heidelberg A00E005 Environmental sample Salmonella houtenae Ad 596 Dairy product microscopic colonies Salmonella Indiana 2 Seafood Salmonella indica Ad 600 Environmental sample Salmonella Infantis F401B Dairy product Salmonella Kottbus 1 Environmental sample Salmonella Lagos 173 Meat product Salmonella Lille 37 / Salmonella Livingstone F104 Pet food Salmonella London 326 Meat product Salmonella Manhattan 900 Environmental sample Salmonella Mbandaka 81 Egg product fin 28. Salmonella Montevideo Ad 912 Dairy product ADRIA Développement 56/60 December 4, 2014