What are the benefits and harms of screening for Tyrosinaemia type 1?: A systematic review of test accuracy and early treatment

Transcription

1 What are the benefits and harms of screening for Tyrosinaemia type 1?: A systematic review of test accuracy and early treatment Review group: Sian Taylor-Phillips Karoline Freeman Julia Geppert Chris Stinton Hannah Fraser Sam Johnson Paul Sutcliffe Aileen Clarke Correspondence to: Dr Sian Taylor-Phillips Populations, Evidence and Technologies Division of Health Sciences Warwick Medical School University of Warwick Coventry CV4 7AL Tel: S.Taylor-Phillips@warwick.ac.uk Date completed: 8 th October 2015 The views expressed in this protocol are those of the authors and not necessarily those of the NSC. Any errors are the responsibility of the authors. The authors have no conflicts of interest. 1

2 Abbreviations CF CHT CINAHL FAH Cystic fibrosis Congenital hypothyroidism Cumulative Index to Nursing and Allied Health Literature Fumarylacetoacetate hydrolase, fumarylacetoacetase GA1 Glutaric aciduria type 1 HCU IVA MCAAD MS/MS MSUD NBS NICE NIHR NTBC OMIM PKU PPI PRISMA REA SCD SUAC TMS Homocystinuria Isovaleric acidaemia Medium-chain acyl-coa dehydrogenase deficiency Tandem mass spectrometry Maple syrup urine disease Newborn blood spot National Institute for Health and Care Excellence National Institute for Health Research 2-(2-nitro-4-trifluoromethylbenzoyl)-1,3-cyclohexanedione (also known as Nitisinone or Orfadin ) Online Mendelian Inheritance in Man Phenylketonuria Patient and public involvement Preferred Reporting Items for Systematic Reviews and Meta-Analyses Rapid Evidence Assessment Sickle cell disease Succinylacetone Tandem mass spectrometry TYR1 Tyrosinaemia type 1 UK NSC UK National Screening Committee 2

3 1 Decision Problem This report undertaken for the UK National Screening Committee (UK NSC) examines the effectiveness and appropriateness of expanding newborn screening using tandem mass spectrometry (TMS, also known as MS/MS), based on current guidelines for evaluating screening programmes, for Tyrosinaemia type 1 (TYR1). TYR1 (OMIM ) is a rare autosomal recessive disorder of amino acid metabolism and is caused by a defect in the final enzyme of the pathway of tyrosine degradation, namely fumarylacetoacetate hydrolase (FAH, also known as fumarylacetoacetase). About one person in 100,000 is affected with TYR-1 globally but incidence is more common in some regions, notably in Quebec (1). Deficiency of FAH causes an accumulation of tyrosine and toxic metabolites succinylacetone (SUAC), maleylacetoacetate and fumarylacetoacetate (Figure 1). TYR1 mainly affects liver, kidney and peripheral nerves. Without treatment, death from liver failure, neurological crisis or hepatocellular carcinoma frequently occurs before the age of 10 years (2). Figure 1. Tyrosine degradation pathway: TYR1 is caused by a defect in fumarylacetoacetase. Succinylacetone is a potent inhibitor of 5-aminolaevulinate dehydratase. Nitisinone inhibits 4- hydroxyphenylpyruvate dioxygenase reducing flux through the pathway (1). 3

4 TYR1 is treated with Nitisinone (2-[2-nitro-4-trifluoromethylbenzoyl]-1,3-cyclohexanedione, NTBC, Orfadin ) accompanied by a protein-restricted diet (usually low in phenylalanine, methionine and tyrosine). Nitisinone inhibits 4-hydroxyphenylpyruvate dioxygenase, an enzyme that is upstream of FAH in the tyrosine degradation pathway, and so reduces the formation of toxic metabolites (see Figure 1). At the present time, liver transplantation remains the only effective means of establishing normal enzyme activity and treating the disease in patients with hepatocellular carcinoma or with failure to respond to Nitisinone treatment. The current Public Health England NHS Newborn Blood Spot (NBS) Screening Programme recommends that all newborn babies in England are screened for the following nine conditions: sickle cell disease (SCD), cystic fibrosis (CF), congenital hypothyroidism (CHT), phenylkentonuria (PKU), medium-chain acyl-coa dehydrogenase deficiency (MCAAD), maple syrup urine disease (MSUD), isovaleric acidaemia (IVA), glutaric aciduria type 1 (GA1), and homocystinuria (HCU). The same nine conditions are part of the NBS screening test in Wales. Scotland is screening newborns for SCD, CF, CHT, PKU and MCAAD, while Northern Ireland s NBS screening also includes HCU and TYR1 (3). TYR1 is not included in the current NBS Screening Programme in England, Scotland and Wales. For NBS screening, a heel-prick blood sample is collected on a standard newborn screening collection card from babies on day 5 (in exceptional circumstances between day 5 and day 8) and sent to the regional newborn screening laboratory. Dried blood spots are analysed using TMS, alongside a variety of other techniques. The last UK NSC update review of screening for TYR1 was recently signed off at the March 2015 UK NSC meeting (4). The review contained literature from relevant publications between 2004 and August The recommendation from the TYR1 review suggested that additional work focussing on specific questions would be needed to assess the possibility of screening for this condition. This review will focus on key issues that were fed back through the consultation responses following the initial 2014 TYR1 review (4) : Inclusion of studies published since August 2012 which increase understanding of TYR1 incidence. Short- and long-term outcomes of Nitisinone treatment. Focus on studies using succinylacetone (SUAC) as the single TMS screening marker for TYR1. 4

5 2 Decision questions and objectives 2.1 Evidence review and decision questions Aim: To examine the effectiveness and appropriateness of expanding newborn screening using TMS, based on current guidelines for evaluating screening programmes, for TYR Decision questions The decision questions for this review are shown in the box below: 1. What is the UK prevalence of TYR1? (Criterion 2) 2. What is the test accuracy (sensitivity, specificity, and predictive values applicable to UK prevalence) of SUAC measurement in dried blood spots using TMS for TYR1 screening? 3. Does early treatment with Nitisinone (NTBC, Orfadin ) following screening, provide better long-term outcomes than later treatment with Nitisinone following symptomatic detection? (Criterion 10) Given these decision questions the three main objectives for this report are: A) To provide an assessment of the birth prevalence of TYR1 in the UK, prioritising UK studies or studies with European/largely Caucasian population (with similar 12% spread of Black, Asian and minority ethnic groups) as well as to provide, if possible, a comparison of screened vs. unscreened populations. B) To assess the accuracy (sensitivity, specificity, and predictive values applicable to UK prevalence) of newborn screening for TYR1 using TMS measurement of SUAC in dried blood spots against a reference standard of urine testing for SUAC and/or subsequent clinical detection of TYR1. C) To compare short- and long-term outcomes (benefits and harms) of early treatment with Nitisinone following screening, with later treatment following symptomatic detection. The review will not be designed to evaluate the following: The accuracy of the reference standard (urine testing for SUAC) and whether cases detected at screening by the reference standard result in overtreatment. 5

6 3 Methods We will use a rapid evidence assessment approach (REA). There is no accepted definition of what is a REA (5; 6) nor an established methodological guide in how to conduct them (7). REA can vary greatly in their purpose, methodological rigour, comprehensiveness, resources used, transparency, and the time for production (8). For the purpose of the current research we will use the following definition to REA as literature reviews that use methods to accelerate or streamline traditional systematic review processes (9). We will adopt an iterative approach to that of more traditional systematic review methods commonly reported (e.g. PRISMA), steps may be reduced or omitted through a restricted inclusion criteria, limited literature searches, study selection and minimised data extraction, brief or lack of quality assessment, and/or lack of meta-analysis (5). In order to maintain the methodological rigour and transparency of traditional systematic review, whilst at the same time streamlining the process, we will seek guidance from commissioners and a broad range of stakeholders. We plan to adopt an innovative approach to formulating and refining the research question and scope of the review by conducting a priority setting event with key stakeholders involved in this area. 3.1 Identification and selection of studies Search strategies A separate literature search will be performed for each of the three review questions. A copy of the main search strategies that are likely to be used in the major databases is provided in Appendix 1. For decision question 1 searches will start from August 2012 (the cut-off date of the initial 2014 review from Bazian (4) ). This will be supplemented with included studies from the 2014 review by Bazian. No date limits will be applied to the searches for decision question 2 and 3. These strategies may be further refined and appropriate concepts or limits may be added or removed. The search strategies developed for MEDLINE will be adapted as appropriate for other databases. Additional supplementary searches will be carried out as necessary. The search strategy will comprise the following main elements: Searching of electronic bibliographic databases Contact with experts in the field Scrutiny of references of included studies and relevant systematic reviews Bibliographic databases are likely to include: MEDLINE (Ovid); MEDLINE In-Process & Other Non-Indexed Citations (Ovid); EMBASE (Ovid); Web of Science, Cochrane Library. 6

7 3.1.2 Inclusion and exclusion of relevant studies Studies that satisfy the following criteria will be included: Population For question 1: All populations. For question 2: Neonatal or newborn infants. For question 3: Patients with Tyrosinaemia type 1 (TYR1). Target condition The condition under consideration is TYR1. Intervention For question 1: No intervention. For question 2: The index test is newborn screening for TYR1 using TMS measurement of SUAC in dried blood spots. For question 3: Early treatment with Nitisinone (NTBC, Orfadin ) following screening. Reference standard For question 1 & 3: No reference standard. For question 2: Urine testing for SUAC and/or subsequent clinical detection of TYR1. Comparator For question 1 & 2: No comparator. For question 3: Later treatment with Nitisinone following symptomatic detection. Outcome For question 1: Incidence or prevalence of TYR1. For question 2: Sensitivity, specificity, predictive values. For question 3: Any outcome of treatment. Study type For question 1: Any systematic review, cross-sectional study or cohort study ideally taken over at least 5 years. For question 2: Cross-sectional test accuracy studies, and studies which follow up screen negatives to identify false negatives. For question 3: Any study design in humans. 7

8 Papers that fulfil the following criteria will be excluded: For question 1: Papers published before 2012 (this will be an update of a previous review, (4) that covered published literature before this data). For question 1-3: Non-human studies, any papers not available in the English language, letters, editorials and communications, grey literature and conference abstracts. 3.2 Review strategy One reviewer will independently screen the titles and abstracts of all records identified by the searches. To perform double sifting, another author will independently screen a random sample of 20% of titles and abstracts. Disagreement will be resolved by retrieval of the full publication and consensus agreement. Full copies of all studies deemed potentially relevant, will be obtained and one reviewer will assess these for inclusion. A second reviewer will independently screen a random sample of 20% of full texts; any disagreements will be resolved by consensus or discussion with a third reviewer. Records rejected at full text stage and reasons for exclusion will be documented. 3.3 Data extraction strategy Data will be extracted by one reviewer, using an electronic, piloted data extraction form. A random sample of 20% of articles will be subjected to a second, independent extraction as a quality control measure. Any disagreements will be resolved by consensus or discussion with a third reviewer. An example of a data extraction sheet is provided in Appendix Quality assessment strategy Formal quality assessment will not be undertaken for question 1. For the quality appraisal of questions 2 and 3 standard quality assessment tools will be adapted and used (Question 2: QUADAS-2 (10) ; Question 3: Downs and Black (11) ; see Appendix 3). Quality assessment will be undertaken by one reviewer and a random sample of 20% will be checked by a second reviewer. Any disagreements will be resolved by a third reviewer through discussion. 3.5 Methods of analysis/synthesis Study, treatment, population, and outcome characteristics will be summarised in text and tables. TYR1 prevalence data will be split into UK/Western European populations (with similar 12% spread of Black, Asian and minority ethnic groups) vs. other to take into account racial/ethnic differences in TYR1 prevalence. Prevalence data will be also split into screened vs. unscreened populations to estimate overdiagnosis. Pooling study results by meta-analysis will be performed if data allow. 8

9 4 Competing interests of authors and advisors None of the authors have any competing interests. 5 Timescales - Start: 17 th July Delivery of evidence review: 23 rd December 2015 Task Start date End date Protocol development Adapt and pilot QUADAS tool Conduct literature search Export studies to EndNote Screen titles and abstracts Obtain and assess full text Screen references Data extraction Quality assessment Formal update to UK NSC Data synthesis Write up Patient and Public Involvement Patient and public involvement (PPI) has become an important element in health research, contributing to our understanding of its acceptability, relevance and appropriateness. Ultimately the aim of PPI is to enhance the quality of research. NIHR policy is to ensure PPI is embedded in all relevant health research and NIHR INVOLVE enables researchers to deliver on these goals. More recently studies have examined the PPI evidence base, in order to develop our understanding of PPI. In developing PPI activity we will draw on INVOLVE guidance and support, and the good practice developed by the NICE PPI Programme. 7 Team members contributions Name: Dr Sian Taylor-Phillips Title: Assistant Professor in Screening and Diagnosis Address: Populations, Evidence and Technologies, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry CV4 7AL Tel: S.Taylor-Phillips@warwick.ac.uk Contribution: Co-ordinate review process, protocol development, synthesis of findings and report writing 9

10 Name: Mrs Karoline Freeman Title: Research Fellow Address: Populations, Evidence and Technologies, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry CV4 7AL Tel: Contribution: Protocol development, assessment for eligibility, data extraction, data entry, data analysis, and report writing Name: Dr Julia Geppert Title: Research Assistant Address: Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry CV4 7AL Tel: Contribution: Protocol development, assessment for eligibility, data extraction, data entry, and report writing Name: Dr Chris Stinton Title: Research Fellow Address: Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry CV4 7AL Tel: Contribution: Protocol development, assessment for eligibility, data extraction, data entry, and report writing Name: Sam Johnson Title: Academic Support Librarian Address: University of Warwick Library, Coventry CV4 7AL Tel: Contribution: Protocol development, development and conduct of literature searches, and report writing 10

11 Name: Hannah Fraser Title: Research Project Administrator Address: Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry CV4 7AL Tel: Contribution: Protocol development, assessment for eligibility, and report writing Name Dr Paul Sutcliffe Title: Associate Professor and Head of Populations, Evidence and Technologies Address: Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry CV4 7AL Tel: Contribution: Protocol development, and report writing Name: Professor Aileen Clarke Title: Professor of Public Health Research Address: Populations, Evidence and Technologies, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry CV4 7AL Tel: Contribution: Co-ordinate review process, protocol development, synthesis of findings, and report writing 11

12 References 1. de Laet C, Dionisi-Vici C, Leonard JV et al. (2013) Recommendations for the management of tyrosinaemia type 1. Orphanet J Rare Dis 8, van Spronsen FJ, Thomasse Y, Smit GP et al. (1994) Hereditary tyrosinemia type I: a new clinical classification with difference in prognosis on dietary treatment. Hepatology 20, Taylor-Phillips S, Boardman F, Seedat F et al. (2014) The Ethical, Social and Legal Issues with Expanding the Newborn Blood Spot Test. legacy.screening.nhs.uk/policydb_download.php?doc= Ltd. B (2014) Screening for Tyrosinaemia I. External review against programme appraisal criteria for the UK National Screening Committee (UK NSC). legacy.screening.nhs.uk/policydb_download.php?doc=474: NSC UK National Screening Committee. 5. Featherstone RM, Dryden DM, Foisy M et al. (2015) Advancing knowledge of rapid reviews: an analysis of results, conclusions and recommendations from published review articles examining rapid reviews. Syst Rev 4, Harker J, Kleijnen J (2012) What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments. Int J Evid Based Healthc 10, Khangura S, Konnyu K, Cushman R et al. (2012) Evidence summaries: the evolution of a rapid review approach. Syst Rev 1, Hartling L, Guise JM, Kato E et al. (2015) An Exploration of Methods and Context for the Production of Rapid Reviews. Research White Paper. Rockville (MD): Agency for Healthcare Research and Quality (US). 9. Ganann R, Ciliska D, Thomas H (2010) Expediting systematic reviews: methods and implications of rapid reviews. Implement Sci 5, Whiting PF, Rutjes AW, Westwood ME et al. (2011) QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155, Downs SH, Black N (1998) The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 52,

13 Appendix 1. Search strategies for Ovid Medline A - Review question 1 (Prevalence search): Searches Results Search Type 1 exp Tyrosinemias/ 2 (tyrosinemia* or tyrosinaemia*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 3 (tyr1 or tyr-1).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 4 (tyri or tyr-i).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 5 (((fumarylacetoacetate adj hydrolase) or fumarylacetoacetase or fah) adj2 deficien*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 6 1 or 2 or 3 or 4 or 5 7 exp Epidemiologic Studies/ 8 epidemiolog*.mp. 9 exp Prevalence/ 10 exp Incidence/ 11 (prevalen* or inciden*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 12 7 or 8 or 9 or 10 or and limit 13 to yr="2012 -Current" 13

14 B - Review question 2 (Screening test search): Searches Results Search Type Actions 1 exp Tyrosinemias/ (tyrosinemia* or tyrosinaemia*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 3 (tyr1 or tyr-1).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 4 (tyri or tyr-i).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 5 (((fumarylacetoacetate adj hydrolase) or fumarylacetoacetase or fah) adj2 deficien*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] or 2 or 3 or 4 or suac.mp exp Heptanoates/ or succinylacetone.mp succinylacetoacetate.mp succinylacetone.rn ,6-dioxoheptanoate.mp exp Heptanoic Acids/ or 4,6-Dioxoheptanoic acid.mp or 8 or 9 or 10 or 11 or and

15 C - Review question 3 (Nitisinone treatment search): Searches Results Search Type Actions 1 exp Tyrosinemias/ (tyrosinemia* or tyrosinaemia*).mp. [mp=title, abstract, original title, 999 name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 3 (tyr1 or tyr-1).mp. [mp=title, abstract, original title, name of 963 substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 4 (tyri or tyr-i).mp. [mp=title, abstract, original title, name of substance 29 word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 5 (((fumarylacetoacetate adj hydrolase) or fumarylacetoacetase or fah) 127 adj2 deficien*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 6 1 or 2 or 3 or 4 or nitisinone.mp nitisinone.rn ntbc.mp orfadin.mp nitro 4 trifluoromethylbenzoyl.mp or 8 or 9 or 10 or and

16 Appendix 2. Data extraction form for included studies Data extraction form for primary studies Name of first reviewer: Name of second reviewer: Study details Study ID (Endnote ref) First author surname and year of publication Country Study design Study setting Number of centres Time period / study duration Follow up period Funding Competing interests / Role of sponsor Review question Prevalence (1) Screening test (2) NTBC treatment (3) Aim of the study Patient selection Inclusion criteria: Exclusion criteria: Study flow Item All Early NTBC Late NTBC Never NTBC Screened Randomised/Included in study Excluded from study (reasons) Missing participants Withdrawals Included in analysis Excluded from analysis (reasons) 16

17 Baseline characteristics For 1-3) All Early NTBC Late NTBC Never NTBC Total number of participants Age Mean (SD) Median (range) years Sex (male/female, % female) Ethnicity Education: Regular Special Socioeconomic status: Low Middle High Comments on the presence or absence of significant differences between treatment arms: Additional baseline characteristics For 3) NTBC treatment All Early NTBC Late NTBC Never NTBC Total number of participants Way of diagnosis Age at diagnosis, median (range) Pathogenic FAH mutation Age at start of NTBC, median (range) Patient-months without NTBC treatment Patient-months with NTBC treatment Current NTBC use (y/n) Consanguity (y/n) Affected family members (y/n) 17

18 Interventions & comparators 1) Prevalence Screened cohort Unscreened cohort Historic cohort Screening programme in operation? (y/n) If yes, fill 2) for details Confirmation of disease Follow-up (years) 2) TMS screening method Source and type of material Age at specimens collection Sample transport and storage Samples pooled? Method of extraction & TMS analysis Type of tandem MS Data management Quality assurance Analyte(s) Cut-off / Threshold Cut-off prespecified (y/n) Positive screening results Reference standard used Follow-up (years) Number received index test, n (%) Number received reference standard 3) NTBC treatment Early NTBC Late NTBC Never NTBC Total number Age at start of NTBC treatment, median (range) NTBC dose (mg/kg/day) NTBC frequency Other medication Dietary treatment Monitoring Follow-up (patient months) IQ test used Other (specify) 18

19 Outcomes 1) Prevalence / Incidence of TYR1 Reported outcomes: Number screened Number of identified cases Incidence Cases per 100,000 Notes / Comments: Screened cohort Unscreened cohort Historic cohort 2) Screening for TYR1 Reported outcomes: Total number screened TP TN FP FN Sensitivity, % (95% CI) Specificity, % (95% CI) PPV, % (95% CI) NPV, % (95% CI) Other (specify) Notes / Comments: 3) NTBC treatment Reported outcomes: Number included patients Death Death before liver transplantation (LT) Death after LT LT Age at LT Months with TYR1-related hospitalisations (neurological crises included) Months with neurological crises Acute liver failure Chronic liver disease Carcinoma Cirrhosis Hepatomegaly Rickets Renal dysfunction Renal tubular dysfunction Early NTBC Late NTBC Never NTBC 19

20 Nephromegaly Nephrocalcinosis Neurological crisis ADS, behavioural disorders Learning/language difficulties Impaired psychomotor development IQ Plasma tyrosine level Plasma phenylalanine level Others (specify): Notes / comments: Authors comments & conclusion Reviewer s comments & conclusion 20

21 Appendix 3. Quality assessment forms A QUADAS-2 tool with index questions adapted to the review for studies comparing performance of different tests (10) Name of first reviewer: Phase 1: State the review question Name of second reviewer: Patients (setting, intended use of index test, presentation, prior testing): Index test(s): Reference standard: Phase 2: Draw a flow diagram for the primary study Phase 3: Risk of bias and applicability judgements QUADAS-2 is structured so that four key domains are each rated in terms of the risk of bias and the concern regarding applicability to the review question (as stated in Phase 1). Each key domain has a set of signalling questions to help reach the judgements regarding bias and applicability. Domain 1: Patient selection A. Risk of bias Describe methods of patient selection: Was a consecutive or random sample of patients enrolled? Did the study avoid inappropriate exclusions? Could the selection of patients have introduced bias? Risk: B. Concerns regarding applicability Describe included patients (prior testing, presentation, intended use of intervention test and setting): Is there concern that the included patients do not match the review question? Concern: 21

22 Domain 2: Index test A. Risk of bias Describe the intervention test and how it was conducted and interpreted: Were the number of failed results and measurement repeats reported? Could the conduct or interpretation of the intervention test have introduced bias? Risk: B. Concerns regarding applicability Describe the preparation and storage of the sample before the intervention test was applied: Is there concern that the intervention test, its conduct, or interpretation differ from the review question? Concern: Domain 3: Reference standard A. Risk of bias Describe the reference standard and how it was conducted and interpreted: Is the reference standard likely to correctly classify the target condition? Could the reference standard, its conduct, or its interpretation have introduced bias? Risk: B. Concerns regarding applicability Is there concern that the target condition as defined by the reference standard does not match the review question? Concern: Domain 4: Flow and timing A. Risk of bias Describe any patients who did not receive the intervention test or who were excluded from the Bland-Altman plot: Describe the time interval and any interventions between intervention test and reference standard: Was there an appropriate interval between intervention test and reference standard? Were both intervention test and reference standard conducted on all samples? Did patients receive the same reference standard? Were all patients included in the Bland-Altman plot? Could the patient flow have introduced bias? Risk: 22

23 B Downs and Black checklist for non-randomised primary clinical studies (11) First author (year) study ID: Name of first reviewer: Reporting Name of second reviewer: 1. Is the hypothesis/aim/objective of the study clearly described? (Yes/No) 2. Are the main outcomes to be measured clearly described in the Introduction or Methods section? (Yes/No) If the main outcomes are first mentioned in the Results section, the question should be answered No 3. Are the characteristics of the patients included in the study clearly described? (Yes/No) In cohort studies and trials, inclusion and/or exclusion criteria should be given. In casecontrol studies, a case-definition and the source for controls should be givenfsan 4. Are the interventions of interest clearly described? (Yes/No) Treatments and placebo (where relevant) that are to be compared should be clearly described 5. Are the distributions of principal confounders in each group of subjects to be compared clearly described? (Yes/Partially/No) A list of principal confounders is provided 6. Are the main findings of the study clearly described? (Yes/No) Simple outcome data (including denominators and numerators) should be reported for all major findings so that the reader can check the major analyses and conclusions (This question does not cover statistical tests which are considered below) 7. Does the study provide estimates of the random variability in the data for the main outcomes? (Yes/No) In non-normally distributed data the inter-quartile range of results should be reported. In normally distributed data the standard error, standard deviation or confidence intervals should be reported. If the distribution of the data is not described, it must be assumed that the estimates used were appropriate and the question should be answered Yes 8. Have all important adverse events that may be a consequence of the intervention been reported? (Yes/No) This should be answered Yes if the study demonstrates that there was a comprehensive attempt to measure adverse events. (A list of possible adverse events is provided) 9. Have the characteristics of patients lost to follow-up been described? (Yes/No) This should be answered Yes where there were no losses to follow-up or where losses to follow-up were so small that findings would be unaffected by their inclusion. This should be answered No where a study does not report the number of patients lost to follow-up 10. Have actual probability values been reported (e.g., rather than <0.05) for the main outcomes except where the probability value is less than 0.001? (Yes/No) External validity 11. Were the subjects asked to participate in the study representative of the entire population from which they were recruited? (Yes/No/Unable to determine) The study must identify the source population for patients and describe how the patients were selected. Patients would be representative if they comprised the entire source population, an unselected sample of consecutive patients, or a random sample. Random sampling is only feasible where a list of all members of the relevant 12. Were those subjects who were prepared to participate representative of the entire population from which they were recruited? (Yes/No/Unable to determine) The proportion of those asked who agreed should be stated. Validation that the sample was representative would include demonstrating that the distribution of the main confounding factors was the same in the study sample and the source population 13. Were the staff, places, and facilities where the patients were treated, representative of the treatment the majority of patients receive? (Yes/No/Unable to determine) For the question Rating Rating 23

24 to be answered Yes the study should demonstrate that the intervention was representative of that in use in the source population. The question should be answered No if, for example, the intervention was undertaken in a specialist centre unrepresentative of the hospitals most of the source population would attend Internal validity bias 14. Was an attempt made to blind study subjects to the intervention they have received? (Yes/No/Unable to determine) For studies where the patients would have no way of knowing which intervention they received, this should be answered Yes 15. Was an attempt made to blind those measuring the main outcomes of the intervention? (Yes/No/Unable to determine) 16. If any of the results of the study were based on "data dredging", was this made clear? (Yes/No/Unable to determine) Any analyses that had not been planned at the outset of the study should be clearly indicated. If no retrospective unplanned subgroup analyses were reported, then answer Yes 17. In trials and cohort studies, do the analyses adjust for different lengths of follow-up of patients, or in case-control studies, is the time period between the intervention and outcome the same for cases and controls? (Yes/No/Unable to determine) Where follow-up was the same for all study patients the answer should Yes. If different lengths of follow-up were adjusted for by, for example, survival analysis the answer should be Yes. Studies where differences in follow-up are ignored should be answered No 18. Were the statistical tests used to assess the main outcomes appropriate? (Yes/No/Unable to determine) The statistical techniques used must be appropriate to the data. For example nonparametric methods should be used for small sample sizes. Where little statistical analysis has been undertaken but where there is no evidence of bias, the question should be answered Yes. If the distribution of the data (normal or not) is not described it must be assumed that the estimates used were appropriate and the question should be answered Yes 19. Was compliance with the intervention/s reliable? (Yes/No/Unable to determine) Where there was non-compliance with the allocated treatment or where there was contamination of one group, the question should be answered No. For studies where the effect of any misclassification was likely to bias any association to the null, the question should be answered Yes 20. Were the main outcome measures used accurate valid and reliable? (Yes/No/Unable to determine) For studies where the outcome measures are clearly described, the question should be answered Yes. For studies which refer to other work or that demonstrates the outcome measures are accurate, the question should be answered as Yes Internal validity - confounding (selection bias) 21. Were the patients in different intervention groups (trials and cohort studies) or were the cases and controls (case-control studies) recruited from the same population? (Yes/No/Unable to determine) For example, patients for all comparison groups should be selected from the same hospital. The question should be answered Unable to determine for cohort and case-control studies where there is no information concerning the source of patients included in the study 22. Were study subjects in different intervention groups (trials and cohort studies) or were the cases and controls (case-control studies) recruited over the same period of time? (Yes/No/Unable to determine) For a study which does not specify the time period over which patients were recruited, the question should be answered as Unable to determine 23. Were the subjects randomised to intervention groups? (Yes/No/Unable to determine) Studies which state that subjects were randomised should be answered Yes except where method of randomisation would not ensure random allocation. For example alternate allocation would score No because it is predictable Rating Rating 24

25 24. Was the randomised intervention assignment concealed from both patients and health care staff until recruitment was complete and irrevocable? (Yes/No/Unable to determine) All non-randomised studies should be answered No. If assignment was concealed from patients but not from staff, it should be answered No 25. Was there adequate adjustment for confounding in the analyses from which the main findings were drawn? (Yes/No/Unable to determine) This question should be answered No for trials if: the main conclusions of the study were based on analyses of treatment rather than intention to treat; the distribution of known confounders in the different treatment groups was not described; or the distribution of known confounders differed between the treatment groups but was not taken into account in the analyses. In nonrandomised studies if the effect of the main confounders was not investigated or confounding was demonstrated but no adjustment was made in the final analyses the question should be answered as No 26. Were losses of patients to follow-up taken into account? (Yes/No/Unable to determine) If the numbers of patients lost to follow-up are not reported, the question should be answered as Unable to determine. If the proportion lost to follow-up was too small to affect the main findings, the question should be answered Yes Power 27. Did the study have sufficient power to detect a clinically important effect where the probability value for a difference being due to chance is less than 5%? (Yes/No/Unable to determine)* Rating 25