Benchmark datasets for phylogenomic pipeline validation

Transcription

1 Benchmark datasets for phylogenomic pipeline validation GenomeTrakr Meeting Sept Ruth E. Timme, PhD Research Microbiologist GenomeTrakr data coordinator

2 Validation for phylogenomics Phylogenomic pipeline = raw reads -> phylogeny Goal: validate results to ensure accuracy and reproducibility. Do not enforce standards to overall approach. 2

3 CFSAN s WGS workflow modules CFSAN Wet lab data analysis Culture handling DNA Extraction Library Preparation Sequencing GenomeTrakr QC pipeline Upload to NCBI clustering and phylogenetic inference Purity? Quality, Quantity Library Concentration Run metrics (Q30, CD, CPF) Coverage, Q scores, ID check, Contamination, WGS serotyping PD QC pipeline 6 modules: each is independently validated Each module has routine QC checks 3

4 Phylogenomic analysis within the Gen-FS community CFSAN Wet lab data analysis Culture handling DNA Extraction Library Preparation Sequencing GenomeTrakr QC pipeline Upload to NCBI clustering and phylogenetic inference clustering and phylogenetic inference Purity? Quality, Quantity Library Concentration Run metrics (Q30, CD, CPF) Coverage, Q scores, ID check, Contamination, WGS serotyping PD QC pipeline NCBI PD pipeline Maximum compatibility phylogeny CFSAN SNP pipeline Maximum likelihood phylogeny BioNumerics wgmlst Distance phylogeny Lyve-SET pipeline Maximum likelihood phylogeny 4

5 Compare pipeline results to truth 1. SNPs and other variants 2. tree topology 5

6 Empirical dataset: What is truth? well-studied outbreak concordant epi and WGS signal **validated SNPs Synthetic dataset known SNPs and tree topology 6

7 Benchmark datasets Dataset Organism # isoaltes Type of dataset Stone Fruit recall L. monocytogenes 31 Empirical Spicy Tuna outbreak S. enterica 23 Empirical Raw Milk Outbreak C. jejuni 22 Empirical Sprouts outbreak E. coli 10 Empirical Simulated outbreak S. enterica 23 Synthetic Experimental evo. E. coli 100 in vitro evolution Manuscript: R Timme et al Bacterial pathogen genome datasets for bioinformatics pipelines. PeerJ 5, e

8 grab downloading script at GitHub 8

9 Standard template for datasets header body Organism Salmonella enterica subspecies enterica Serovar Bareilly Outbreak 1203NYJAP-1 - Tuna Scrape Outbreak pmid tree source Ruth Timme datatype empirical suggested sha256sum sha256su sha256sum biosample_acc strain genbankassembly SRArun_acc outbreak datasetname Reference Assembly mread1 Read2 vouchercontact SAMN CFSAN CP006053, CP SRR outgroup 1203NYJAP-1 - Tuna Scrape Outbreak TRUE Dwayne Roberson, FDA SAMN CFSAN JMMH SRR outgroup 1203NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JMMM SRR outgroup 1203NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRDM SRR outgroup 1203NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRCY SRR outgroup 1203NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JMMG SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRCQ SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRCO SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRCN SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA checksums SAMN CFSAN JRCM SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRCJ SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRCI SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRCG SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRCD SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRCB SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRCA SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRBZ SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JMMF SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRBT SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRBL SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRBK SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRBJ SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA SAMN CFSAN JRBI SRR NYJAP NYJAP-1 - Tuna Scrape Outbreak FALSE Dwayne Roberson, FDA 9

10 Permanent repository for data: NCBI sequence data, assemblies, biosamples opentreeoflife tree files Data Archiving tree.opentreeoflife.org/curator/study/edit/ot_

11 Working with the Datasets GenFSgopher.pl -o outdir spreadsheet.dataset.tsv Fastq files, fasta files, newick tree file Perform your own clustering pipeline(s) (FDA: SNP pipeline, CDC: LyveSet, NCBI: Pathogen Detection, ) Compare results! 11

12 TreeToReads: Simulated data pipeline Collaboration with Emily Jane McTavish, UC Merced Download here: McTavish, E. J., Pettengill, J., Davis, S., Rand, H., Strain, E., Allard, M., & Timme, R. E. (2017). TreeToReads - a pipeline for simulating raw reads from phylogenies. BMC Bioinformatics, 18(1),

13 Acknowledgements Gen-FS WGS Standards and Analysis workgroup (CDC/FDA/NCBI/FSIS) FDA: Ruth Timme, Hugh Rand, Steven Davis CDC: Lee S. Katz, Eija Trees, Heather A. Carleton NCBI: Richa Agarwala, Martin Shumway, Bill Klimke FSIS: Mustafa Simmons, Glenn Tillman, Philip Bronstein, Stephanie Defibaugh-Chávez Global Microbial Identifier WG3 Other contributors: Errol Strain Darlene Wagner Cheryl Tarr Maria Hoffmann Maria Sanchez-Leon 13

14