Performance Characteristics BRCA MASTR Plus Dx

Transcription

1 Performance Characteristics BRCA MASTR Plus Dx with drmid Dx for Illumina NGS systems Manufacturer Multiplicom N.V. Galileïlaan Niel Belgium

2 Table of Contents 1. Workflow Performance Characteristics of BRCA MASTR Plus workflow (Somatic Application) Samples Study Design Success Rate and On-Target Read Counts Limit of Detection Sensitivity, Specificity and Accuracy Repeatability and Reproducibility Lot Equivalence Disclaimer PERFORMANCE CHARACTERISTICS of BRCA MASTR Plus Dx workflow (Germline Samples Study Design Success Rate and On-Target Read Counts Sensitivity, Specificity and Accuracy Repeatability and Reproducibility Lot Equivalence Disclaimer List of Figures Figure 1: graphic representation of the MASTR workflow... 4 List of Tables Table 1: Composition proficiency panel... 5 Table 2: Expected VAF for the proficiency panel... 5 Table 3: PES design... 6 Table 4: Instrumentation used at the different study sites... 6 Table 5: Sensitivity, specificity and accuracy at 5 % VAFsample... 7 Table 6: Repeatability at 5 % VAFsample... 8 Table 7: Reproducibility at 5 % VAFsample... 8 Table 8: Repeatability of BRCA MASTR Plus Dx assay. For each proficiency sample, the mean VAF and the coefficient of variation (CV) are shown per variant Table 9: Reproducibility of BRCA MASTR Plus Dx assay. For each proficiency sample, the mean VAF and the CV are shown per variant Table 10: Reproducibility of BRCA MASTR Plus Dx assay. For each proficiency sample, the mean VAF and the CV are shown per variant. (continued)... 9 Table 11: Lot equivalence at 5 % VAFsample... 9 Revision date: August 23, 2017 Page 2 of 15

3 Table 12: PES design of the HMW samples assayed with BRCA MASTR Plus Dx in combination with Illumina MiSeq platform Table 13: PES design of the HMW samples assayed with BRCA MASTR Plus Dx in combination with Illumina MiniSeq platform Table 14: PES design of the HMW samples assayed with BRCA MASTR Plus Dx in combination with Illumina NextSeq platform Table 15: Instrumentation used in the PES Table 16: Success rate and on-target read counts Table 17: Estimated sensitivity, specificity and accuracy of the BRCA MASTR Plus on the Illumina MiSeq, MiniSeq and.. 14 Table 18: Repeatability of the BRCA MASTR Plus on the Illumina MiSeq, MiniSeq and NextSeq Table 19: Reproducibility of the BRCA MASTR Plus on the Illumina MiSeq, MiniSeq and NextSeq Abbreviations CV coefficient of variation DQC DNA quality coefficient FFPE formalin-fixed paraffin-embedded FN false negative FP false positive GIAB Genome In A Bottle HP homopolymer IFU instructions for use IGV integrative genomics viewer indels insertions and/or deletions IVD For In Vitro Diagnostic Use LOD limit of detection MASTR multiplex amplification of specific targets for re-sequencing NGS Next-Generation Sequencing PES performance evaluation study SNV single nucleotide variation TN true negative TP true positive VAF variant allele frequency Revision date: August 23, 2017 Page 3 of 15

4 1. WORKFLOW Figure 1: graphic representation of the MASTR workflow Revision date: August 23, 2017 Page 4 of 15

5 2. PERFORMANCE CHARACTERISTICS OF BRCA MASTR PLUS WORKFLOW (SOMATIC APPLICATION) 2.1. Samples The study population comprised 54 FFPE-derived DNA extracts from 51 clinical and 3 proficiency samples. The clinical samples were derived from 42 ovarian cancers, 4 breast cancers, 3 tonsillar cancers, 1 colon cancer and 1 fallopian tube cancer. Different DNA extraction kits were used to obtain DNA from the FFPE tumor tissue: QIAamp FFPE DNA tissue (Qiagen), QIAsymphony DSP DNA Mini Kit (Qiagen) and iprep ChargeSwitch Forensic kit (Life technologies). DNA extracts were subjected to the QC Plex (Multiplicom, QC ), those showing a DQC value 0.1 were included, indicating sufficient DNA quality for reliable amplification and sequencing. DNA was of excellent (5 samples), good (15 samples) or acceptable DNA quality (31 samples). The proficiency panel (Mix 1-3) was based on 3 Genome In A Bottle (GIAB) FFPE sections (Horizon Discovery). At Multiplicom, DNA was extracted from the FFPE sections using the QIAamp DNA FFPE Tissue Kit. The extracted DNA was mixed to obtain variants in a broad VAF range (5-100 %) (Table 1 and 2). Table 1: Composition proficiency panel GIAB samples NIST ID Coriell ID Reference Standard Horizon Discovery HG003 GM24149 Ashkenazim PGP Father - FFPE Reference Standard HG004 GM24143 Ashkenazim PGP Mother - FFPE Reference Standard HG005 GM24631 Asian PGP Son - FFPE Reference Standard Percentage GIAB sample per proficiency sample Mix 1 Mix 2 Mix 3 10 % 15 % 20 % 80 % 70 % 60 % 10 % 15 % 20 % Table 2: Expected VAF for the proficiency panel Variant Expected VAF Chrom pos rs number Mix 1 Mix 2 Mix 3 chr13: t>c rs % 7.5 % 10 % chr13: a>g rs % 7.5 % 10 % chr17: t>c rs % 15 % 20 % chr17: a>g rs % 15 % 20 % chr17: t>c rs % 15 % 20 % chr17: a>g rs % 15 % 20 % chr17: t>c rs % 15 % 20 % chr17: g>a rs % 15 % 20 % chr17: g>a rs % 15 % 20 % chr13: c>t rs % 42.5 % 40 % chr13: a>g rs % 92.5 % 90 % chr13: a>g rs % 92.5 % 90 % Revision date: August 23, 2017 Page 5 of 15

6 2.2. Study Design Three genetic centers (study sites SS1-3) performed the entire BRCA MASTR Plus Dx workflow on a total of 23 samples per site (Table 23). The PES assessed the performance of BRCA MASTR Plus Dx to detect SNVs and small indels for 5 % VAFsample, thus taking into account: (i) sequencing depth, (ii) expected VAF of the proficiency panel, (iii) data analysis settings. Table 3: PES design SS1 SS2 SS3 3 proficiency samples in triplicate 13 clinical samples (1 in duplicate) 3 proficiency samples 19 clinical samples (1 in duplicate) Illumina MiSeq platform Illumina MiSeq Reagent Kit v3 (600 cycles, run at 2x251bp) Sequencing Different instrumentation was used at the different study sites. Table 4: Instrumentation used at the different study sites depth for 5 % VAF sample 23 samples/run: ± 905,000 read pairs/sample 2 lots BRCA MASTR Plus Dx 1 lot MID Dx 1-48 for Illumina MiSeq 3 proficiency samples 19 clinical samples (1 in duplicate) Instrument SS1 SS2 SS3 Thermal cycler 9700 (Applied Biosystems) SimpliAmp (Applied Biosystems) Fragment analyzer 3130XL (Applied Biosystems) 3130XL (Applied Biosystems) C1000 Biorad (Bio-Rad) 3130XL (Applied Biosystems) Data analysis was performed with the SeqNext module version (JSI Medical Systems) and the Sophia DDM platform (ILL1MR1S5_BRCA_Tumorv2). The target region was defined as the BRCA1 and BRCA2 coding regions +/- 2 bp Success Rate and On-Target Read Counts The success rate of the BRCA MASTR Plus Dx is defined as the percentage of amplicons that show at least 0.2 x the mean amplicon coverage. The percentage of generated reads with a length of 60 bp or more that map back to the amplicons determines the on-target read counts. Of the 69 samples, one sample did not result in a measurable amplicon library and was not sequenced. The overall success rate of BRCA MASTR Plus Dx is 93.9 % (90.6 % 95.7 %). The on-target read count of BRCA MASTR Plus Dx is 99.1 % (98.5 % 99.5 %) Limit of Detection The limit of detection is defined as the lowest VAF that can be distinguished from the background, determined as the maximum measured noise over all target bases in all samples + 3 x standard deviation. The limit of detection of BRCA MASTR Plus Dx is 1 %. The LOD for variants in HP of 6, 7 or 8 bases is 3, 5 and 9 %, respectively. Revision date: August 23, 2017 Page 6 of 15

7 2.5. Sensitivity, Specificity and Accuracy Sensitivity is calculated as TP / (TP + FN). Specificity is calculated as TN / (TN + FP). Accuracy is calculated as (TP + TN)/ (TP+FP+FN+TN) with: True positive (TP): variant call based on BRCA MASTR Plus Dx data and on reference data in case of sufficient coverage (at least 50/VAF) True negative (TN): no variant in BRCA MASTR Plus Dx data or reference data in case of sufficient coverage (at least 50/VAFsample ) False positive (FP): variant call based on BRCA MASTR Plus Dx data but absent in reference data, in case of sufficient coverage (at least 50/VAF) False negative (FN): no variant in BRCA MASTR Plus Dx data but variant called in reference data, in case of sufficient coverage (at least 50/VAFsample ) Each clinical sample contributed once to these performance characteristics. For three clinical samples analyzed in duplicate, the replicate that performed worst was included. Since none of these samples showed FNs or FPs, the replicate with the lowest number of TNs was included. Of the 51 clinical samples that were included in the study, one sample was not sequenced because it did not result in a measurable amplicon library and one sample showed very low read counts. In total, 49 clinical samples contributed to the calculation of sensitivity, specificity and accuracy. GeneRead DNAseq Targeted Panels V2 - Human BRCA1 and BRCA2 Panel (Qiagen) was used as a targeted resequencing- based reference method. The true status of a variant is defined based on the GeneRead results in combination with the analysis of mapped raw read counts using integrative genomics viewer (IGV). Table 5: Sensitivity, specificity and accuracy at 5 % VAFsample SeqNext Sophia DDM TP FN 0 0 FP 1 * 0 TN 691, ,607 Sensitivity % [95 % CI %] % [95 % CI %] Specificity % [95 % CI %] % [95 % CI %] Accuracy % [95 % CI %] % [95 % CI %] * SeqNext reported a true positive 10 bp insertion with a VAF of 84 % at 75 %. Additionally, a false positive base change at 8 % was called at the same position Repeatability and Reproducibility Repeatability and reproducibility are defined as the number of concordant bases over the total number of bases. For each variant of the proficiency samples, the variability of the VAF is calculated (coefficient of variation (CV)). Only regions with a coverage of 50/VAFsample or higher are included. Repeatability is determined using three proficiency samples analyzed in triplicate at the same site. Reproducibility is determined using three proficiency samples analyzed at different sites. Since one site performed the same analysis in triplicate, the analysis that generated the highest discordance determines the reproducibility. Revision date: August 23, 2017 Page 7 of 15

8 Table 6: Repeatability at 5 % VAFsample SeqNext Sophia DDM Proficiency sample Concordant bases Discordant bases Concordant bases Discordant bases Mix 1 16, ,041 0 Mix 2 16, ,041 0 Mix 3 16,039 2 * 16,041 0 Repeatability % [95 % CI %] % [95 % CI %] * rs and rs both at ± 5 % did not appear in filtered variant list due to the data analysis settings (VAF sample 5 %, VAF: 4 %). Table 7: Reproducibility at 5 % VAFsample SeqNext Sophia DDM Proficiency sample Concordant bases Discordant bases Concordant bases Discordant bases Mix 1 16, ,041 0 Mix 2 15, ,067 0 Mix 3 16,039 2 * 16,041 0 Reproducibility % [95 % CI %] % [95 % CI %] * rs and rs both at ± 5 % did not appear in filtered variant list due to the data analysis settings (VAF sample 5 %, VAF: 4 %). Table 8: Repeatability of BRCA MASTR Plus Dx assay. For each proficiency sample, the mean VAF and the coefficient of variation (CV) are shown per variant. Variant Mix 1 Mix 2 Mix 3 Chrom pos rs number Mean VAF CV Mean VAF CV Mean VAF CV chr13: t>c rs % % 7.50 % 7.87 % % 5.09 % chr13: a>g rs % % 7.00 % 7.87 % % 5.41 % chr17: t>c rs % 4.33 % % 5.68 % % 0.00 % chr17: a>g rs % 4.33 % % 0.00 % % 2.37 % chr17: t>c rs % 5.41 % % 8.65 % % 4.55 % chr17: a>g rs % 9.90 % % 3.27 % % 0.00 % chr17: t>c rs % 8.33 % % 5.68 % % 2.47 % chr17: g>a rs % 5.41 % % 7.90 % % 2.55 % chr17: g>a rs % 4.56 % % 3.33 % % 2.47 % chr13: c>t rs % 1.30 % % 1.33 % % 2.50 % chr13: a>g rs % 0.00 % % 0.63 % % 0.65 % chr13: a>g rs % 0.61 % % 0.63 % % 0.64 % Revision date: August 23, 2017 Page 8 of 15

9 Table 9: Reproducibility of BRCA MASTR Plus Dx assay. For each proficiency sample, the mean VAF and the CV are shown per variant. Variant Mix 1 Mix 2 Mix 3 Chrom pos rs number Mean VAF CV Mean VAF CV Mean VAF CV chr13: t>c rs % % 6.33 % % % 9.09 % chr13: a>g rs % % 7.00 % % % 9.09 % chr17: t>c rs % 7.69 % % % % 2.28 % chr17: a>g rs % 9.12 % % % % 0.00 % Table 10: Reproducibility of BRCA MASTR Plus Dx assay. For each proficiency sample, the mean VAF and the CV are shown per variant. (continued) Variant Mix 1 Mix 2 Mix 3 Chrom pos rs number Mean VAF CV Mean VAF CV Mean VAF CV chr17: t>c rs % 0.00 % % % % 2.55 % chr17: a>g rs % 5.09 % % % % 0.00 % chr17: t>c rs % 4.95 % % % % 2.44 % chr17: g>a rs % 5.09 % % % % 2.47 % chr17: g>a rs % 4.56 % % % % 2.55 % chr13: c>t rs % 3.85 % % 6.01 % % 0.00 % chr13: a>g rs % 0.00 % % 1.65 % % 1.29 % chr13: a>g rs % 1.22 % % 0.00 % % 1.12 % 2.7. Lot Equivalence The lot equivalence is described as the number of concordant bases over the total number of bases. Lot equivalence is determined based on a clinical sample processed in duplicate using BRCA MASTR Plus Dx kits from two lots at each site. Table 11: Lot equivalence at 5 % VAFsample SeqNext Sophia DDM Clinical sample Concordant bases Discordant bases Concordant bases Discordant bases S1 13, ,433 0 S2 14, ,796 0 S3 14, ,584 0 Lot equivalence 100 % [95 % CI %] 100 % [95 % CI %] Revision date: August 23, 2017 Page 9 of 15

10 2.8. Disclaimer The purchase of this product enables the purchaser to use it for the amplification of nucleic acid sequences for human genes (BRCA MASTR Plus Dx) and of BRCA MASTR Plus Dx-derived amplicons (drmid Dx for Illumina NGS systems kits). The purchaser has to take into account that information obtained from amplicons generated using this product may not be used in procedures that are protected by valid claims owned and/or controlled by a third party, unless prior written approval of such party has been obtained. IVD product performance claims apply only when combining BRCA MASTR Plus Dx and drmid Dx for Illumina NGS systems and when both CE-Mark kits are used according to this CE-IVD IFU. The BRCA MASTR Plus Dx has been designed for SNV, small indels and CNV analysis on high molecular weight (HMW) DNA and for SNV and small indels analysis on FFPE-derived DNA. The kit has been validated only for analysis of SNV and small indels using a VAFsample 5 % on FFPE-derived DNA in combination with Illumina MiSeq systems. Revision date: August 23, 2017 Page 10 of 15

11 3. PERFORMANCE CHARACTERISTICS OF BRCA MASTR PLUS DX WORKflOW (GERMLINE Application) 3.1. Samples The study population comprised 49 high-molecular weight (HMW)-derived DNA extracts from 45 blood clinical and 4 proficiency samples. Clinical samples were derived from 45 individuals selected to contain a pathogenic BRCA mutation. Different DNA extraction methods were used to obtain DNA from blood: Genomic Midi Kit AX (AA Biotechnology) and Phenol/chloroform extraction methods. DNA samples were measured using spectrophotometry (Nanodrop ND-1000) and fulfilled all requirements for DNA extracted from blood. The proficiency panel (NA10863, NA11993, NA10838, NA10838 and NA12005) was based on HMW samples from the Coriell Institute for Medical Research. These samples were used to determine the repeatability and reproducibility of the BRCA MASTR Plus Study Design The entire BRCA MASTR Plus Dx workflow was performed in one genetic center, on a total of 45 blood-derived samples and 4 proficiency samples. The PES assessed the performance of BRCA MASTR Plus to detect SNVs and small indels on bloodderived DNA on the Illumina MiSeq, NextSeq and MiniSeq sequencing platforms. Performance characteristics were determined for the BRCA MASTR Plus on Illumina MiSeq, NextSeq and MiniSeq. The same study design was used for all 3 Illumina sequencing platforms (Table 12, Table 13 and Table 14). The same sequencing libraries were processed on Illumina MiSeq, MiniSeq and NextSeq platforms. Table 12: PES design of the HMW samples assayed with BRCA MASTR Plus Dx in combination with Illumina MiSeq platform Run1 Run2 4 proficiency samples in triplicate 4 proficiency samples 45 clinical samples Illumina MiSeq platform Illumina MiSeq Reagent Kit v3 (600 cycles) Run at 2x251bp Minimum coverage per sample (73,000 read pairs) 1 lot BRCA MASTR Plus Dx 1 lot drmid 1-48 for Illumina NGS systems 1 lot drmid for Illumina NGS systems Revision date: August 23, 2017 Page 11 of 15

12 Table 13: PES design of the HMW samples assayed with BRCA MASTR Plus Dx in combination with Illumina MiniSeq platform Run1 4 proficiency samples in triplicate 45 clinical samples Illumina MiniSeq platform Illumina MiniSeq Mid-Output kit (300 cycles) Run at 2x151bp Minimum coverage per sample (73,000 read pairs) 1 lot BRCA MASTR Plus Dx 1 lot drmid 1-48 for Illumina NGS systems 1 lot drmid for Illumina NGS systems Run2 4 proficiency samples Table 14: PES design of the HMW samples assayed with BRCA MASTR Plus Dx in combination with Illumina NextSeq platform Run1 4 proficiency samples in triplicate 45 clinical samples Illumina NextSeq platform Run2 4 proficiency samples Illumina NextSeq 500/550 v2 Mid-Output kit (300 cycles) Run at 2x151bp Minimum coverage per sample (73,000 read pairs) 1 lot BRCA MASTR Plus Dx 1 lot drmid 1-48 for Illumina NGS systems 1 lot drmid for Illumina NGS systems The instruments used in the PES are described in Table 15. Table 15: Instrumentation used in the PES Instruments Thermal Cycler Fragment Analyzer Spectrophotometer NGS system Details Veriti (Applied Biosystems) 3130XL (Applied Biosystems) Dropsense (Trinean) MiSeq, NextSeq, MiniSeq (Illumina) Data analysis was performed with the Multiplicom MASTR Reporter software. The target region was defined as the BRCA1 and BRCA2 coding regions +/- 30 bp. FASTQ files were downsized to allow analysis at 50 % VAF. Revision date: August 23, 2017 Page 12 of 15

13 3.3. Success Rate and On-Target Read Counts The success rate of the BRCA MASTR Plus Dx is defined as the percentage of amplicons that show at least 0.2 x the mean amplicon coverage. The percentage of generated reads with a length of 60 bp or more that are correctly allocated to the amplicons determines the on-target read counts. The success rate was determined on all 45 blood-derived clinical samples and in the replicates of the 4 proficiency samples, included in MiSeq Run 1, NextSeq Run 1 and MiniSeq Run 1. The success rate and on-target read counts of BRCA MASTR Plus Dx on the Illumina platforms is described in Table 16. Table 16: Success rate and on-target read counts Illumina Platform Success Rate On-target read counts MiSeq % % MiniSeq % % NextSeq % % 3.4. Sensitivity, Specificity and Accuracy Sensitivity is calculated as TP / (TP + FN). Specificity is calculated as TN / (TN + FP). Accuracy is calculated as (TP + TN)/ (TP+FP+FN+TN) with: TP: variant call based on BRCA MASTR Plus Dx data and on reference data in case of sufficient coverage (at least 20/VAF) TN: no variant in BRCA MASTR Plus Dx data or reference data in case of sufficient coverage (at least 40x) FP: variant call based on BRCA MASTR Plus Dx data but absent in reference data, in case of sufficient coverage (at least 20/VAF) FN: no variant in BRCA MASTR Plus Dx data but variant called in reference data, in case of sufficient coverage (at least 40x) In total, 45 clinical samples contributed to the calculation of sensitivity, specificity and accuracy. BRCA MASTR Dx (Multiplicom) was used as targeted resequencing-based reference method. Revision date: August 23, 2017 Page 13 of 15

14 Table 17: Estimated sensitivity, specificity and accuracy of the BRCA MASTR Plus on the Illumina MiSeq, MiniSeq and Illumina Sequencing Platform MiSeq MiniSeq NextSeq TP FN FP 0 1* 0 TN 834, , ,709 Sensitivity % [95 % CI %] % [95 % CI %] % [95 % CI %] Specificity % [95 % CI %] % [95 % CI %] % [95 % CI %] Accuracy % [95 % CI %] % [95 % CI %] % [95 % CI %] * MASTR Reporter reported one FP in the evaluation of the MiniSeq data. The FP is a result of a PCR stutter pattern caused by a heteropolymer constituted by a repeat of the CA dinucleotide, with the FP variant called at the start of the heteropolymer Repeatability and Reproducibility Repeatability (intra-run variability) and reproducibility (inter-run variability) are defined as the number of concordant bases over the total number of bases. Only regions with a coverage of 20/VAFsample or higher are included. Repeatability was determined using four proficiency samples analyzed in triplicate in MiSeq Run 1, NextSeq Run 1 and MiniSeq Run 1 (Table 18). Reproducibility was determined using four proficiency samples analyzed in Runs 1 and 2. The analysis that generated the highest discordance results in MiSeq Run 1, NextSeq Run 1 and MiniSeq Run 1 was included in the determination of the reproducibility (Table 19). Table 18: Repeatability of the BRCA MASTR Plus on the Illumina MiSeq, MiniSeq and NextSeq Illumina Platform Concordant bases Discordant bases Repeatability MiSeq 75, % [95 % CI %] MiniSeq 74, % [95 % CI %] NextSeq 74, % [95 % CI %] Table 19: Reproducibility of the BRCA MASTR Plus on the Illumina MiSeq, MiniSeq and NextSeq Illumina Platform Concordant bases Discordant bases Repeatability MiSeq 74, % [95 % CI %] MiniSeq 74, % [95 % CI %] NextSeq 74, % [95 % CI %] 3.6. Lot Equivalence The lot equivalence is described as the number of concordant bases over the total number of bases. Please refer to section ( ) for details of the lot equivalence of the BRCA MASTR Plus Dx. Revision date: August 23, 2017 Page 14 of 15

15 3.7. Disclaimer The purchase of this product enables the customer to use it for the amplification of nucleic acid sequences for human genes (BRCA MASTR Plus Dx) and of BRCA MASTR Plus Dx-derived amplicons (drmid Dx for Illumina NGS systems kits). The customer has to take into account that information obtained from amplicons generated using this product may not be used in procedures that are protected by valid claims owned and/or controlled by a third party, unless prior written approval of such party has been obtained. IVD product performance claims are only applicable when BRCA MASTR Plus Dx is performed in combination with drmid Dx for Illumina NGS systems (MiSeq, NextSeq and MiniSeq) and MASTR Reporter Software and when both CE-Mark kits are used according to this CE-IVD IFU. The BRCA MASTR Plus Dx has been designed for SNV, small indels and CNV analysis on blood-derived DNA. The kit has been validated only for analysis of SNV and small indels on blood-derived DNA in combination with Illumina NGS systems (MiSeq, NextSeq and MiniSeq) and data-analysis with the MASTR Reporter. PUB NUMBER ENN PR NUMBER PR Revision date: August 23, 2017 Page 15 of 15