Jessica Minion 1,2, Erika Leung 1,2, Dick Menzies 1,2, Madhukar Pai 1,2. Contact:

Transcription

1 Microscopically Observed Drug Susceptibility (MODS) and Thin Layer Agar (TLA) assays for the detection of drug resistance in Mycobacterium tuberculosis: A Systematic Review and Meta-analysis Jessica Minion 1,2, Erika Leung 1,2, Dick Menzies 1,2, Madhukar Pai 1,2 1 Dept of Epidemiology, Biostatistics & Occupational Health, McGill University, Montreal, Canada 2 Respiratory Epidemiology & Clinical Research Unit, Montreal Chest Institute, Montreal, Canada Contact: Dr. Madhukar Pai, MD, PhD Assistant Professor & CIHR New investigator Dept. of Epidemiology, Biostatistics & Occupational Health McGill University 1020 Pine Avenue West Montreal, Canada H3A 1A2 Tel: Fax: madhukar.pai@mcgill.ca 1

2 INTRODUCTION With the growing problem of drug resistance in Mycobacterium tuberculosis (MTB), so grows the demand for quicker, cheaper and easier ways to detect resistance (1). Patients infected with multidrug resistant (MDR) or extensively drug resistant (XDR) strains of MTB require modified treatment regimens including second line drugs as well as extended treatment duration (2). Even when treated appropriately, cases of MDR and XDR TB have significantly worse outcomes (3) and are often infectious for long periods (4). Therefore, preventing the spread of MDR and XDR TB becomes all the more important, and timely identification of these cases is the first and most critical step in that prevention. Several methods of detecting drug resistance are now available, none of which clearly satisfies the demands of quick, cheap and easy. Traditional agar-based methods can take up to 8 weeks for results. Using commercial liquid-culture based DST decreases turnaround time to less than 2 weeks, but often requires expensive equipment not available in most high-burden settings. Molecular detection of gene mutations associated with drug resistance (e.g. line probe assays) has also been developed commercially, and recently endorsed by WHO (5), however the level of expertise required of a laboratory offering molecular diagnostics is currently too great for these tests to be implemented in many peripheral settings. Non-commercial techniques using simple and widely available laboratory equipment and supplies are now being investigated for DST that could potentially be implemented in both peripheral and centralized laboratories with minimal cost and training. The microscopically observed drug sensitivity (MODS) assay and the thin layer agar (TLA) method are two such non-commercial techniques being investigated for expansion of DST services. The technique employed by both MODS and TLA is direct inoculation of patient specimens to drug-free and drug-containing media followed by microscopic examination of cultures (liquid media in the case of MODS; solid media in the case of TLA) in order to detect early growth (6, 7). These are also called microcolony methods. The growth of MTB in or on the drug-free media indicates a positive culture; the growth of MTB in or on both the drug-free and drug-containing media indicates resistance. Questions have been raised over the technical difficulty of both methods and the specificity of differentiating MTB from non-tuberculous mycobacteria (NTM) (8). There have been concerns regarding contamination rates and biohazard risks, particularly with the liquid culture used in MODS (9). Additionally, MODS 2

3 requires the use of an inverted microscope which would not be considered routine laboratory equipment. OBJECTIVES We undertook a systematic review of the literature and meta-analysis, where appropriate, of data examining the diagnostic accuracy and performance characteristics of MODS and TLA for the detection of drug resistance in MTB. METHODS We followed a standard protocol for systematic reviews and meta-analyses [Appendix 1]. Search Strategy We systematically searched 3 databases for relevant citations: PubMed, EMBASE and BIOSIS (January 1990 February 2009 inclusive). The search strategy used for PubMed is shown in Appendix 2. All searches were performed with the help of an experienced librarian. Publications in English, French or Spanish were considered. Reference lists from included studies were hand searched. Additionally, experts were contacted to identify additional studies. Published and unpublished studies were considered eligible. Unpublished studies were included if they were received by August Eligibility Criteria Predetermined eligibility criteria for the primary analysis were: evaluation of MODS or TLA for drug susceptibility of MTB, and use of an accepted reference standard. Accepted reference standards included indirect proportion methods (PM), absolute concentration (AC), resistance ratio (RR), commercial liquid systems (BACTEC 460, MGIT 960, MB/BacT) or microdilution methods. Studies using either direct patient specimen inoculation or indirect inoculation of isolates to MODS or TLA were included. Studies on use of MODS and TLA for TB diagnosis (case detection) were excluded from the primary analysis of this review. However, any study reporting contamination rates following the direct inoculation of patient specimens to MODS or TLA were extracted and used for pooled contamination rate estimates only. Study Selection 3

4 All selection steps were performed by two independent reviewers (JM and EL) and disagreements were resolved by consensus. Titles and abstracts were screened for relevance and any citations identified by either reviewer were selected for full text review. Articles retrieved for full text review along with reasons for exclusion are available from the authors. Data Extraction We created and piloted a data extraction form with a subset of eligible studies. Based on experience gained in the pilot study, the extraction form was finalized [Appendix 3]. All studies included in the final review were extracted independently by 2 reviewers (JM and EL) and any disagreements were resolved by consensus. Assessment of Study Quality Using the QUADAS criteria (10) for assessment of quality of diagnostic accuracy studies, we assessed quality characteristics that were considered important for this specific review: (i) blinded interpretation of the test result with reference standard results and vice-versa, (ii) complete verification of test results with the reference standard, (iii) recruitment of patients/specimens either consecutively or randomly, and (iv) study design (cross-sectional vs case-control). The full QUADAS instrument is reproduced within Appendix 3. Outcome measures Data were extracted for each drug tested to construct 2 by 2 tables of true positive (TP), false positive (FP), false negative (FN), and true negative (TN) values. True positives were defined as specimens found resistant by both MODS or TLA and the reference method. False positives were defined as specimens found resistant by MODS or TLA, but sensitive by the reference method. False negatives were defined as specimens found sensitive by MODS or TLA, but resistant by the reference method. True negatives were defined as specimens found sensitive by both MODS or TLA and the reference method. From these data we calculated the sensitivity and specificity for each drug tested. If studies tested more than one concentration of each drug, results were preferentially included for rifampin 1µg/mL, isoniazid 0.4µg/mL, ethambutol 2.5µg/mL, streptomycin 6µg/mL, ofloxacin 2mg/mL, pyrazinamide 0.5µg/mL, and kanamycin 6µg/mL. These concentrations were the most commonly reported and chosen to provide the most consistency between studies. If studies tested both diluted 4

5 and undiluted direct specimen inoculums, results from the undiluted tests were preferentially included, likewise, to provide the most comparable results between studies. Other outcomes that were extracted from included studies were: turnaround time (defined as the time from specimen receipt or processing in the laboratory to availability of results within the laboratory), and cost estimates (which may include costs for reagents, supplies, equipment, labour, overhead or other justified related costs). Contamination rates (defined as the proportion of specimens contaminated on first inoculation) were extracted from both studies reporting MTB DST results as well as studies reporting only MTB detection. Analysis Data were analyzed using STATA/IC Forest plots visually displaying sensitivity and specificity estimates and their 95% confidence intervals (CIs) from each study were constructed using MetaDiSc software (11). Since these measures tend to be correlated and vary according to thresholds (either explicit or implicit cut-off values determining positive vs. negative results), hierarchical summary receiver operating characteristic (HSROC) curves were analyzed to explore the influence of those thresholds (12, 13). Accuracy measures were pooled using bivariate random effects regression models (14), using the userwritten program metandi in STATA (15). Heterogeneity of accuracy estimates was assessed using the I 2 statistic. If fewer than 4 studies evaluated a given drug for either MODS or TLA, their results were not meta-analyzed, in part because bivariate random effects regression models do not converge with small number of studies. Subgroup Analysis Results are presented separately by index test (MODS or TLA) for each drug tested. Data were not combined across tests or across drugs tested. In our protocol, we had planned for subgroup analysis by the type of specimen inoculated (direct patient sample vs. indirect isolate), if sufficient studies were identified. RESULTS Selection of included studies is summarized in Figure 1. We identified 2067 citations from the initial searches and an additional 5 from other sources unique articles were left after excluding duplicate 5

6 articles. Screening of titles and abstracts identified 54 potentially relevant articles which were then retrieved for full text review. Of these, 12 studies (9 MODS and 3 TLA) were considered eligible for this review. Characteristics of Included Studies Studies meeting our selection criteria are described in Table 1. There were 9 studies that evaluated MODS resistance testing (Table 1a)(16-24) and 3 that evaluated TLA (Table 1b)(25-27). All but 1 evaluation presented results using rifampin and all but 2 presented results using isoniazid. Other drugs for which data were available included: ethambutol, streptomycin, ofloxacin, kanamycin, and pyrazinamide. Three MODS studies and 1 TLA study used previously cultured isolates (indirect inocula), while the remaining studies directly inoculated clinical specimens for resistance testing. Four studies (3 MODS and 1 TLA) used only smear positive specimens, while 7 studies (5 MODS and 2 TLA) used only sputum specimens. Only 2 studies (1 MODS and 1 TLA) provided information on the HIV status of patients providing clinical specimens for testing. Quality of Included Studies Key indicators of study quality are summarized in Table 2. All studies verified their complete study sample with a reference standard and 8 out of 12 indicated that they used a cross-sectional design. Half of the studies used either consecutive or random samples (3 indicated a convenience sample was used, 3 were unclear) and half of the studies stated that test interpretation was blinded with respect to the reference standard results (the other 6 did not report on blinding). Six studies were performed prospectively, while 3 were performed retrospectively and 3 did not clearly indicate this. Accuracy Estimates Figures 2 5 display sensitivity and specificity estimates for MODS rifampin, TLA rifampin, MODS isoniazid and TLA isoniazid testing. Studies that used indirect specimen inoculation are indicated with closed squares to differentiate them from those that used direct specimen inoculation. MODS rifampin DST estimates had less variability than MODS isoniazid DST (sensitivity ranges 92% - 100% for rifampin vs. 82% 100% for isoniazid; specificity ranges 83% - 100% for rifampin vs. 78% - 100% for isoniazid). All TLA evaluations of both rifampin and isoniazid yielded 100% sensitivity and specificity estimates, 6

7 however, the number of studies was small (3 rifampin, 2 isoniazid). HSROC plots of MODS rifampin and isoniazid studies are shown in Figures 6 and 7. Table 3 summarizes the results of all other drugs tested; studies using indirect specimen inoculation are highlighted with darker backgrounds. The misclassification of resistant strains as susceptible (reflected in sensitivity estimates) was a greater problem then the proper identification of susceptible strains (reflected in specificity estimates). Results were variable but generally poor for ethambutol and streptomycin testing. Kanamycin, pyrazinamide and ofloxacin testing appeared more accurate, but only single evaluations by test were available. Pooled estimates were calculated for groups with at least 4 studies evaluating the same drug and the same test (MODS vs. TLA). These pooled estimates of sensitivity and specificity, along with I 2 measures of heterogeneity are shown in Table 4. Overall, MODS testing of rifampin DST performed well with 98.0% sensitivity (95% CI: ) and 99.4% specificity (95% CI: ). MODS isoniazid testing was slightly inferior with 91.4% sensitivity (95% CI: ) and 97.7% specificity ( ). Metaanalyzed estimates of MODS ethambutol and streptomycin accuracy confirmed very poor sensitivity for detection of resistance (60.0% and 52.9% sensitivity respectively). There were too few studies on TLA to perform bivariate random effects pooling within homogenous groups. There were too few studies to compare the performance of MODS or TLA using direct versus indirect specimens for inoculation within homogenous groups. However, since this is an important question relating to the use of these tests, pooling was performed across drugs tested by MODS. If all drugs were considered, a total of 16 arms using direct specimens and 9 arms using indirect isolates were available for comparison and, using this coarse level of comparison, there was no difference in diagnostic accuracy between those using direct versus indirect specimens (Table 7). If only arms testing rifampin or isoniazid are considered, there is some suggestion that indirect testing may perform slightly better than direct testing, but this is not statistically significant (based on overlapping confidence intervals). There were too few arms of TLA evaluation to make a similar comparison. There were no direct head to head comparisons of MODS vs. TLA drug susceptibility testing on the same specimens. Nor were there any direct comparisons between using direct or indirect specimens identified. Hierarchical Summary Receiver Operating Curves (HSROC) 7

8 Figures 6 and 7 plot the sensitivity (or TPR) and 1-specificity (or FPR) in an HSROC curve for MODS rifampin and MODS isoniazid respectively. Sensitivity appears generally more variable than specificity estimates and this is more pronounced for isoniazid compared to rifampin MODS testing. Turnaround Time Median (or mean) turnaround times (TATs) were provided in 6 MODS studies and in 2 TLA studies. Combined estimates for all studies, those using direct specimens and those using indirect isolates, are shown in Table 8. The mean TAT for MODS (9.9 d, 95% CI: 4.1, 15.8) was slightly shorter than that of TLA (11.1 d, 95% CI: 10.1, 12.0), although overlapping confidence intervals indicate no statistically significant difference. MODS studies using direct specimens resulted in longer TATs (11.6 d, 95% CI: 1.5, 21.7) compared to those that used indirect isolates (6.9 d, 95% CI: 0.2, 12.9) although both estimates have large overlapping confidence intervals. Contamination Rates Table 8 also shows the pooled contamination rates for MODS and TLA. All studies reporting contamination rates used direct patient specimens for inoculation. When considering only studies reporting DST results, TLA had a significantly lower contamination rate (1.7%, 95%CI: 0.5, 4.4) compared to MODS (7.4%, 95% CI: 6.6, 8.1), based on non-overlapping confidence intervals. However, this was based on only 2 TLA studies (26, 27) and 3 MODS studies (16, 17, 24). If all studies reporting contamination rates are included, the pooled contamination rate for MODS decreases to 6.3% (95% CI: 5.8, 7.0; based on 7 studies) and the pooled rate for TLA increases to 11.8% (95% CI: 11.0, 12.6; based on 9 studies). Cost estimates Four studies reported estimates for the cost of performing DST using MODS or TLA. Caviedes et al (2000) estimated the cost for DST of 2 drugs (rifampin, isoniazid) to be $1.72/sample for MODS and $1.60/sample for TLA including reagents and supplies. The equivalent estimates for DST of 4 drugs (rifampin, isoniazid, ethambutol, streptomycin) is cited as $1.80/sample for MODS and $2.92/sample for TLA. Mengatto et al (2006) reported similar costs for MODS ($1.57/sample testing rifampin, isoniazid; $2.17/sample testing rifampin, isoniazid, ethambutol, streptomycin), as do Moore et al (2006; MODS $2/sample). Devasia et al (2009) reported their costs of testing for ofloxacin resistance would be $1.38/sample using MODS (including reagents and supplies), after purchase of an incubator (estimated 8

9 cost $8000) and an inverted microscope (estimated cost $4000). No studies addressed the cost of labour or overhead associated with MODS or TLA DST, nor were any cost-effectiveness analyses identified. All prices were reported in US dollars, current for the year of publication. Other Evidence: Patient Management Although patient outcomes were not directly assessed, Nic Fhogartaigh et al (2008) reported a retrospective database analysis of patients from Lima, Peru whose specimens were selected by their physicians to undergo MODS DST (28). Out of 209 patients whose MODS culture was positive, the culture confirmation and/or DST result was available from MODS before any standard method in more than 80%. In 41.4% of the culture positive patients, the results from MODS should have prompted a modification in patient management. DISCUSSION Primary Findings Through literature searches and contacting experts we were able to identify 9 studies evaluating DST in MODS and 3 studies evaluating DST in TLA. Six of the MODS evaluations used direct patient specimens, while the other 3 used previously isolated strains of MTB. Two TLA studies used direct patient specimens and 1 used indirect isolates. MODS had high accuracy when testing for rifampin resistance, but showed slightly lower sensitivity when detecting isoniazid resistance. MODS testing of other anti-tuberculous drugs was variable but generally suffered from low sensitivity. There was a paucity of data evaluating TLA DST, however, all studies evaluating rifampin or isoniazid testing found 100% concordance with their reference standards. Single evaluations for other anti-tuberculous drugs using TLA unanimously found 100% sensitivity and >98% specificity. The overall TAT for DST results was 9.9 days and 11.1 days for MODS and TLA respectively, making both techniques much faster than conventional proportion method DST (21 days post-isolation) and comparable to commercial liquid culture DST post-isolation (29). The TLA TAT estimate came from studies only using direct specimens. Stratified by specimen type, MODS TAT for direct specimens was 11.6 days versus 6.5 days for indirect testing. Although we excluded MODS and TLA studies for case detection from the primary analyses, these studies do report useful data on overall contamination rates 9

10 (17, 30-38). When all MODS and TLA studies reporting contamination rates were considered, pooled estimates were 6.3% and 11.8% respectively. Strengths and Limitations of the Review This review had several strengths, including a broad and inclusive search of the published literature as well as efforts to identify currently unpublished studies. All study selection steps and data extraction was performed independently by 2 reviewers. Additionally, rigorous statistical methods were employed using bivariate random effects models and HSROC curves for estimating diagnostic performance measures. The review was limited by the number of studies reporting on drug resistance testing evaluations, especially evaluating TLA and second line DST. Subgroup analysis to compare the use of direct patient specimens with the use of indirect isolates for specific DST was not possible. Likewise, other important test characteristics, including drug dosage, method of decontamination, and schedule of reading were not assessed due to the small number of studies available for comparison. Several studies reported estimates for costing of supplies and reagents, but there were no data on the cost-effectiveness of these assays in routine programmatic settings. Many studies also reported on TAT and contamination rates, however there were no studies that focused on patient outcomes. The translation of accurate, timely and useful results into patient outcomes is difficult to demonstrate and is dependent upon a multitude of programmatic factors. Nevertheless, the study by Nic Fhogartaigh et al (2008) provides good evidence that MODS culture and DST are potentially capable of improving patient care. Conclusions Microcolony detection methods such as MODS and TLA are inexpensive, rapid alternatives for drug susceptibility testing for MTB. There is limited published evidence on the performance of TLA for the detection of drug resistance, however accuracy evaluations to date have been excellent. Combined with low reported contamination rates and rapid TATs, TLA appears to be a promising diagnostic tool for DST, especially if a simple colorimetric test format can be used. Indeed, such a test is currently under development by a group in Peru (Dr C Evans), with support from FIND. Further studies are needed to confirm these initial evaluations. MODS testing for rifampin resistance appears consistently accurate across studies, although testing for other anti-tuberculous drugs suffered from variable and suboptimal 10

11 sensitivity. Although laboratories in Peru have greater experience implementing the MODS technique, DST appears to perform equally well in reports from laboratories in Peru and around the world. In conclusion, the results of this systematic review should be considered by experts and policy makers in combination with field experiences in real world settings, and also compared against competing options, including other non-commercial and commercial assays. Acknowledgements This study was supported by funding from the World Health Organization, Geneva, Switzerland. The authors are grateful to Karin Weyer (World Health Organization, Geneva, Switzerland) for helpful input and feedback. The authors are also grateful to the following investigators who provided unpublished and/or additional data: David Moore, Anandi Martin, Jaime Robledo, Susan Dorman, and Juan C. Palomino. 11

12 Figure 1. Study Selection 12

13 Figure 2. Forest Plot MODS Rifampin [N = 8 studies] Point estimates of sensitivity and specificity from each study are shown as open squares (studies using direct specimen inoculation) or closed squares (studies using indirect isolate inoculation). Size of the square is proportionate to the size of the study. Solid lines represent 95% confidence intervals. 13

14 Figure 3. Forest Plot TLA Rifampin [N = 3 studies] Point estimates of sensitivity and specificity from each study are shown as open squares (studies using direct specimen inoculation) or closed squares (studies using indirect isolate inoculation). Size of the square is proportionate to the size of the study. Solid lines represent 95% confidence intervals. 14

15 Figure 4. Forest Plot MODS Isoniazid [N = 8 studies] Point estimates of sensitivity and specificity from each study are shown as open squares (studies using direct specimen inoculation) or closed squares (studies using indirect isolate inoculation). Size of the square is proportionate to the size of the study. Solid lines represent 95% confidence intervals. 15

16 Figure 5. Forest Plot TLA Isoniazid [N = 2 studies] Point estimates of sensitivity and specificity from each study are shown as open squares (studies using direct specimen inoculation) or closed squares (studies using indirect isolate inoculation). Size of the square is proportionate to the size of the study. Solid lines represent 95% confidence intervals. 16

17 Figure 6. HSROC Plot of MODS Rifampin Testing [N=8] Individual studies are shown as open squares whose size is proportionate to the size of the study. Summary point is shown as a closed circle, representing sensitivity and specificity estimates pooled using bivariate random effects model. HSROC curve is truncated outside of the area for which data exist. 17

18 Figure 7. HSROC Plot for MODS Isoniazid Testing [N=8] Individual studies are shown as open square whose size is proportionate to the size of the study. Summary point is shown as a closed circle, representing sensitivity and specificity estimates pooled using bivariate random effects model. HSROC curve is truncated outside of the area for which data exist. 18

19 SCREENING OF PATIENTS AT RISK OF DRUG- RESISTANT TUBERCULOSIS; SEPT 8-9, 2009, GENEVA. Table 1a. Characteristics of Included MODS Studies [N = 9 studies] Ref Author Year Total N Country Drugs Tested Reference Inoculum % Smear + % Sputum % HIV + (17) Caviedes PERU RIF, INH MABA DIRECT NS (18) Devasia USA OFX PM INDIRECT NA NA NA (19) Ejigu ETHIOPIA RIF, INH MGIT 960 DIRECT NS (20) Mello BRAZIL RIF, INH PM DIRECT NS 100 NS (21) Mengatto ARGENTINA RIF, INH, EMB, SM PM INDIRECT NA NA NA (16) Moore PERU RIF, INH, EMB, SM MABA DIRECT NS NS NS (22) Moore PERU RIF, INH, EMB, SM PM DIRECT (23) Park USA RIF, INH, EMB, SM PM INDIRECT NA NA NA (24) Shiferaw ETHIOPIA RIF, INH PM DIRECT NS Table 1b. Characteristics of Included TLA Studies (N = 3 studies) Ref Author Year Total N Country Drugs Tested Reference Inoculum % Smear + % Sputum % HIV + (25) Martin BELGIUM RIF, OFX, KAN PM INDIRECT NA NA NA (26) Robledo COLUMBIA RIF, INH PM DIRECT NS (27) Schaberg GERMANY RIF, INH, EMB, SM, PZA PM DIRECT RIF = rifampin, INH = isoniazid, EMB = ethambutol, SM = streptomycin, OFX = ofloxacin, KAN = kanamycin, PZA = pyrazinamide MABA = microplate alamar blue assay, PM = proportion method Direct Inoculum refers to directly applying a patient specimen (processed or unprocessed) to drug-containing and drug-free media; Indirect Inoculum refers to applying a previously isolated strain of MTB to drug-containing and drug-free media NA not applicable (studies using indirect inocula) NS not specified (studies using direct inocula) 19

20 Table 2. Study Quality Characteristic MODS (n=9) TLA (n=3) Specimen or Patient Recruitment - Prospective - Retrospective - Unclear Study Design - Cross-Sectional - Unclear Sampling - Consecutive or Random - Convenience - Unclear 2 Verification with Accepted Reference - Complete 9 3 Blinded Interpretation of Index and Reference - Yes - Unclear

21 Table 3. Studies Evaluating Drugs other than RIF and INH [N = 7 studies] Study Specimen TP FP FN TN Sensitivity (95% CI) Specificity (95% CI) MODS Ethambutol Moore (2004) DIRECT % (34.9, 96.8) 98.2% (90.4, 100) Moore (2006) DIRECT % (23.5, 61.1) 92.1% (87.2, 95.6) Mengatto (2006) INDIRECT % (45.1, 79.6) 99.0% (97.0, 99.8) Park (2002) INDIRECT % (48.6, 83.3) 72.7% (49.8, 89.3) Streptomycin Moore (2004) DIRECT % (43.4, 87.4) 100% (92.1, 100) Moore (2006) DIRECT % (12.2, 73.8) 98.5% (95.6, 99.7) Mengatto (2006) INDIRECT % (53.4, 76.7) 98.8% (96.6, 99.8) Park (2002) INDIRECT % (19.1, 50.2) 100% (76.8, 100) Ofloxacin Devasia (2009) INDIRECT % (54.1, 100) 100% (98.4, 100) TLA Ethambutol Shaberg (1995) DIRECT % (47.8, 100) 99.5% (97.1, 100) Streptomycin Shaberg (1995) DIRECT % (83.9, 100) 100% (97.9, 100) Pyrazinamide Schaberg (1995) DIRECT % (54.1, 100) 100% (98.1, 100) Ofloxacin Martin (unpub) INDIRECT % (91.0, 100) 100% (96.2, 100) Kanamycin Martin (unpub) INDIRECT % (94.6, 100) 98.7% (93.1, 100) 21

22 Table 4. Pooled Estimates Using Bivariate Random Effects Regression MODS Drug (# of studies) Rifampin (n=8) Isoniazid (n=8) Ethambutol (n=4) Streptomycin (n=4) Pooled Sensitivity (95% CI) 98.0 (94.5, 99.3) 91.4% (86.7, 94.6) 60.0% (47.3, 71.6) 52.9% (36.5, 68.8) I 2 (p-value) 0.0% (p=0.856) 44.1% (p=0.085) 46.6% (p=0.132) 77.0% (p=0.005) Pooled Specificity (95% CI) 99.4 (95.7, 99.9) 97.7% (93.6, 99.2) 95.4% (83.1, 98.9) 98.9% (97.3, 99.5) I 2 (p-value) 68.5% (p=0.002) 70.7% (p=0.001) 80.6% (p=0.001) 0.0% (p=0.92) Table 5. Direct vs. Indirect Specimens Pooled Sensitivity (95% CI) MODS (all drugs) Specimen (# of arms) Direct (n=16) 90.6% (80.2, 95.8) Indirect (n=9) 90.9% (68.8, 97.8) MODS (RIF & INH) Specimen (# of arms) Direct (n=12) 94.1% (89.4, 96.8) Indirect (n=4) 97.2% (87.1, 99.5) I 2 (p-value) 84.4% (p=0.000) 90.6% (p=0.000) 58.4% (p=0.006) 62.7% (p=0.04) Pooled Specificity (95% CI) 98.2% (96.2, 99.2) 99.9% (89.2, 100) 98.3% (95.5, 99.4) 100% (0, 100) I 2 (p-value) 75.4% (p=0.000) 0.0% (p=0.45) 91.1% (p<0.0001) 0.0% (p=1.0) 22

23 Table 6. Turnaround Times and Contamination Rates # studies MODS (range) Turnaround Times Overall d ( ) Direct Only ( ) Indirect Only ( ) Contamination Rates DST studies only 3 7.4% ( ) DST and Detection studies 7 6.3% ( ) # studies TLA (range) ( ) ( ) % (0 4.2) % (0 26) Defined as the number of median (or mean) days between specimen receipt or processing in the laboratory to the availability of results within the laboratory Defined as the proportion of specimens contaminated upon first inoculation 23

24 REFERENCES 1. Wright A, Zignol M, Van Deun A, Falzon D, Gerdes SR, Feldman K, et al. Epidemiology of antituberculosis drug resistance : an updated analysis of the Global Project on Anti- Tuberculosis Drug Resistance Surveillance. Lancet May 30;373(9678): World Health Organization (WHO). Treatment of Tuberculosis: guidelines for national programmes. 3rd ed. Geneva WHO/CDS/TB/ Orenstein EW, Basu S, Shah NS, Andrews JR, Friedland GH, Moll AP, et al. Treatment outcomes among patients with multidrug-resistant tuberculosis: systematic review and meta-analysis. Lancet Infect Dis Mar;9(3): Wells CD, Cegielski JP, Nelson LJ, Laserson KF, Holtz TH, Finlay A, et al. HIV infection and multidrug-resistant tuberculosis: the perfect storm. J Infect Dis Aug 15;196 Suppl 1:S World Health Organization (WHO). Molecular line probe assays for rapid screening of patients at risk of multidrug resistant tuberculosis (MDR-TB) MODS: a user guide. v12.1 ed. Lima, Peru Universidad Peruana Cayetano Heredia; Martin A, Palomino JC. Procedure Manual: Thin Layer Agar (TLA) Microcolony detection. version 3 ed. Antwerp, Belgium: Institute of Tropical Medicine, Mycobacteriology Unit; Moore DAJ. Future prospects for the MODS assay in multidrug-resistant tuberculosis diagnosis. Future Microbiology. [Editorial] Apr;2(2): Moore DA, Caviedes L, Gilman RH, Coronel J, Arenas F, LaChira D, et al. Infrequent MODS TB culture cross-contamination in a high-burden resource-poor setting. Diagn Microbiol Infect Dis Sep;56(1): Whiting P, Rutjes A, Reitsma J, Bossuyt P, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Medical Research Methodology. [diagnostic methods]. 2003;3(1): Zamora J, Abraira V, Muriel A, Khan K, Coomarasamy A. Meta-DiSc: a software for metaanalysis of test accuracy data. BMC Med Res Methodol. 2006;6: Pai M, McCulloch M, Enanoria W, Colford JM, Jr. Systematic reviews of diagnostic test evaluations: What's behind the scenes? ACP J Club. [Instructive] Jul-Aug;141(1):A Littenberg B, Moses LE. Estimating diagnostic accuracy from multiple conflicting reports: a new meta-analytic method. Med Decis Making Oct-Dec;13(4): Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol Oct;58(10): Harbord R, Whiting P. metandi: Meta-analysis of diagnostic accuracy using hierarchical logistic regression. In: Sterne J, editor. Meta-Analysis in Stata: An Unpdated Collection from the Stata Journal. College Station, Texas, USA: Stata Press; p Moore DAJ, Mendoza D, Gilman RH, Evans CAW, Delgado MGH, Guerra J, et al. Microscopic observation drug susceptibility assay, a rapid, reliable diagnostic test for multidrug-resistant tuberculosis suitable for use in resource-poor settings. J Clin Microbiol Oct;42(10): Caviedes L, Lee TS, Gilman RH, Sheen P, Spellman E, Lee EH, et al. Rapid, efficient detection and drug susceptibility testing of Mycobacterium tuberculosis in sputum by microscopic observation of broth cultures. The Tuberculosis Working Group in Peru. J Clin Microbiol Mar;38(3):

25 18. Devasia RA, Blackman A, May C, Eden S, Smith T, Hooper N, et al. Fluoroquinolone resistance in Mycobacterium tuberculosis: an assessment of MGIT 960, MODS and nitrate reductase assay and fluoroquinolone cross-resistance. J Antimicrob Chemother Jun;63(6): Ejigu GS, Woldeamanuel Y, Shah NS, Gebyehu M, Selassie A, Lemma E. Microscopicobservation drug susceptibility assay provides rapid and reliable identification of MDR-TB. Int J Tuberc Lung Dis Mar;12(3): Mello FC, Arias MS, Rosales S, Marsico AG, Pavon A, Alvarado-Galvez C, et al. Clinical evaluation of the microscopic observation drug susceptibility assay for detection of Mycobacterium tuberculosis resistance to isoniazid or rifampin. J Clin Microbiol Oct;45(10): Mengatto L, Chiani Y, Imaz MS. Evaluation of rapid alternative methods for drug susceptibility testing in clinical isolates of Mycobacterium tuberculosis. Mem Inst Oswaldo Cruz Aug;101(5): Moore DA, Evans CA, Gilman RH, Caviedes L, Coronel J, Vivar A, et al. Microscopic-observation drug-susceptibility assay for the diagnosis of TB. N Engl J Med Oct 12;355(15): Park WG, Bishai WR, Chaisson RE, Dorman SE. Performance of the microscopic observation drug susceptibility assay in drug susceptibility testing for Mycobacterium tuberculosis. J Clin Microbiol Dec;40(12): Shiferaw G, Woldeamanuel Y, Gebeyehu M, Girmachew F, Demessie D, Lemma E. Evaluation of microscopic observation drug susceptibility assay for detection of multidrug-resistant Mycobacterium tuberculosis. J Clin Microbiol Apr;45(4): Martin A, Paasch F, Von Groll A, Fissette K, Almeida da Silva PE, Varaine F, et al. Thin layer agar for detection of resistance to rifampicin, ofloxacin and kanamycin in Mycobacterium tuberculosis isolates. Int J Tuberc Lung Dis. 2009;13(10): Robledo J, Mejia GI, Paniagua L, Martin A, Guzman A. Rapid detection of rifampicin and isoniazid resistance in Mycobacterium tuberculosis by the direct thin-layer agar method. Int J Tuberc Lung Dis Dec;12(12): Schaberg T, Reichert B, Schulin T, Lode H, Mauch H. Rapid drug susceptibility testing of Mycobacterium tuberculosis using conventional solid media. Eur Respir J Oct;8(10): Nic Fhogartaigh CJ, Vargas-Prada S, Huancare V, Lopez S, Rodriguez J, Moore DA. Physicianinitiated courtesy MODS testing for TB and MDR-TB diagnosis and patient management. Int J Tuberc Lung Dis May;12(5): Cruciani M, Scarparo C, Malena M, Bosco O, Serpelloni G, Mengoli C. Meta-analysis of BACTEC MGIT 960 and BACTEC 460 TB, with or without solid media, for detection of mycobacteria. J Clin Microbiol May;42(5): Martin A, Fissette K, Varaine F, Portaels F, Palomino JC. Thin layer agar compared to BACTEC MGIT 960 for early detection of Mycobacterium tuberculosis. J Microbiol Methods May Arias M, Mello FC, Pavon A, Marsico AG, Alvarado-Galvez C, Rosales S, et al. Clinical evaluation of the microscopic-observation drug-susceptibility assay for detection of tuberculosis. Clin Infect Dis Mar 1;44(5): Irfan S, Hasan R, Kanji A, Hassan Q, Azam I. Evaluation of a microcolony detection method and phage assay for rapid detection of Mycobacterium tuberculosis in sputum samples. Southeast Asian J Trop Med Public Health Nov;37(6): Michael J, Daley P, Kalaiselvan S, Latha A, Mathai D, John KR, et al. Diagnostic accuracy of microscopic observation drug susceptibility assay (MODS): a pilot study from India.(in press). 34. Tovar M, Siedner MJ, Gilman RH, Santillan C, Caviedes L, Valencia T, et al. Improved diagnosis of pleural tuberculosis using the microscopic- observation drug-susceptibility technique. Clin Infect Dis Mar 15;46(6):

26 35. Martin A, Munga Waweru P, Babu Okatch F, Amondi Ouma N, Bonte L, Varaine F, et al. Implementation of the thin layer agar for the diagnosis of smear-negative pulmonary tuberculosis in a high HIV prevalence setting in Homa Bay, Kenya. J Clin Microbiol Jun Mejia GI, Castrillon L, Trujillo H, Robledo JA. Microcolony detection in 7H11 thin layer culture is an alternative for rapid diagnosis of Mycobacterium tuberculosis infection. Int J Tuberc Lung Dis Feb;3(2): Mejia GI, Guzman A, Agudelo CA, Trujillo H, Robledo J. [Five year experience with thin layer agar medium for rapid diagnosis of tuberculosis]. Biomedica Jun;24 Supp 1: Robledo JA, Mejia GI, Morcillo N, Chacon L, Camacho M, Luna J, et al. Evaluation of a rapid culture method for tuberculosis diagnosis: a Latin American multi-center study. Int J Tuberc Lung Dis Jun;10(6):

27 APPENDIX 1. PROTOCOL FOR A DIAGNOSTIC META-ANALYSIS 27

28 APPENDIX 2. SEARCH STRING USED IN PUBMED Search #1: tuberculosis[mesh] OR mycobacter*[ti] OR acid-fast[ti] OR tuberculous[ti] Search #2: MODS[tw] OR "microscopic-observation"[tiab] OR "drug-susceptibility"[ti] OR (drug[ti] AND susceptibility[ti]) OR microcolony[tiab] OR (liquid[ti] AND culture[ti]) OR thinlayer[tiab] OR (thin[tiab] AND layer[tiab] AND agar[tiab]) OR agar[tiab] OR (mycobacterial[ti] AND culture[ti]) Search #3: #1 AND #2 Limited to January 1990 February

29 APPENDIX 3. EXTRACTION FORM MODS/TLA for the detection of resistance in M. Tuberculosis EXTRACTION FORM Study #: Reviewer: Author: Year: Language: Published: Y / N Country: Sponsor: Index Test: Reference: Specimens: unit of analysis Patients: unit of analysis Selection: liquid (MODS) media: solid (TLA) media: Begin exams day Then every days Until Culture: LJ MGIT BacT BACTEC 460 other (specify): pulmonary only extrapulmonary only mixed % pulm: not specifed adults only peds only mixed not specified case control design cohort design unclear other: direct specimen for isolation only direct specimen for isolation and resistance isolates for resistance only Controls for culture? Positive Negative not specified Resistance Testing: absolute concentration method proportion method resistance ratio method radiometric BACTEC 460 non-radiometric (MGIT, BacT) other (specify): smear + only smear only mixed % +: not specified inpatients only outpatients only mixed not specified consecutive selection random selection unclear other: Resistance Testing: rifampin isoniazid ethambutol pyrazinamide others (specify): Controls for resistance? Positive Negative not specified Validation? complete partial differential Blinding? double index read blind ref read blind neither not specified Decontamination? yes, NaLC yes, other (specify): no not specified HIV included % HIV not included not specified prospective retrospective not specified 29

30 Data on Cost no yes: Cost/specimen: Cost/positive: Index Ref [Cost breakdown] Labour: Materials: Capital Costs: Training: Space: Other Measures: Data on Time no yes: Turnaround from specimen collection Culture Result: Sensitivity Result: Index Ref Turnaround from specimen received in lab Culture Result: Sensitivity Result: Other measures: Data on Biohazards no yes: Qualitative Assessment: Index Ref Incidents reported: Contamination Rate: Data on Implementation no yes: Ease of Use Assessment: Index Ref Length of Training Required: QC/Systems Changes: 30

31 QUADAS CHECKLIST 1. Was the spectrum of patients representative of the patients YES NO UNCLEAR who will receive the test in practice? 2. Were selection criteria clearly described? YES NO UNCLEAR 3. Is the reference standard likely to correctly classify the target condition? 4. Is the time period between reference standard and index test short enough to be reasonably sure that the target condition did not change between the two tests? 5. Did the whole sample or a random selection of the sample, receive verification using a reference standard of diagnosis? YES NO UNCLEAR YES NO UNCLEAR YES NO UNCLEAR 6. Did patients receive the same reference standard regardless YES NO UNCLEAR of the index test result? 7. Was the reference standard independent of the index test YES NO UNCLEAR (i.e. the index test did not form part of the reference standard)? 8. Was the execution of the index test described in sufficient YES NO UNCLEAR detail to permit replication of the test? 9. Was the execution of the reference standard described in YES NO UNCLEAR sufficient detail to permit its replication? 10. Were the index test results interpreted without knowledge YES NO UNCLEAR of the results of the reference standard? 11. Were the reference standard results interpreted without YES NO UNCLEAR knowledge of the results of the index test? 12. Were the same clinical data available when test results YES NO UNCLEAR were interpreted as would be available when the test is used in practice? 13. Were uninterpretable/intermediate test results reported? YES NO UNCLEAR 14. Were withdrawals from the study explained? YES NO UNCLEAR RESULTS Strata Test Ref Specimen Patient TP FP TN FN Total 31

32 MODS/TLA: Reasons for exclusion of studies FIRST AUTHOR YEAR TITLE JOURNAL INCLUDE/EXCLUDE REASON FOR EXCLUSION Akselband, Y Rapid mycobacteria drug susceptibility testing using Gel Microd J Microbiol Method EXCLUDE NOT MODS/TLA Almeida da Silva, P 2007 Microcolony detection in thin layer culture as an alternative me Brazilian Journal of EXCLUDE INAPPROPRIATE REFERENCE Arias, M Clinical evaluation of the microscopic observation drug suscepticlin Infect Dis INCLUDE NO DST RESULTS Aziz, M Expanding culture and drug susceptibility testing capacity in tub Int J Tuberc Lung D EXCLUDE REVIEW Bae, W. H Reducing the string test intra gastric downtime for detection of Int J Tuberc Lung D EXCLUDE NO OUTCOMES OF INTEREST Bwanga, F Direct susceptibility testing for multi drug resistant tuberculosis BMC Infect Dis EXCLUDE REVIEW Caviedes, L Introducing mods: A low cost, low tech tool for high performan Indian Journal of M EXCLUDE EDITORIAL Caviedes, L Rapid, efficient detection and drug susceptibility testing of MycoJ Clin Microbiol INCLUDE Caviedes, L Fast and efficient detection and drug susceptibility testing of MyTubercle & Lung D EXCLUDE MEETING NOTES Caws, M Evaluation of the MODS culture technique for the diagnosis of t PLoS ONE INCLUDE NO DST RESULTS Coban, A. Y Blood agar for susceptibility testing of Mycobacterium tuberculoint J Tuberc Lung D EXCLUDE NOT MODS/TLA Coban, A. Y Susceptibilities of Mycobacterium tuberculosis to isoniazid and J Clin Microbiol EXCLUDE NOT MODS/TLA Coban, A. Y Comparative study for determination of Mycobacterium tubercuj Clin Microbiol EXCLUDE NOT MODS/TLA Devasia, R. A Fluoroquinolone resistance in Mycobacterium tuberculosis: an aj Antimicrob Chem INCLUDE Ejigu, G. S Microscopic observation drug susceptibility assay provides rapidint J Tuberc Lung D INCLUDE Grandjean, L Tuberculosis diagnosis and multidrug resistance testing by direc J Clin Microbiol EXCLUDE NOT MODS/TLA Hasan, R MODS assay for the diagnosis of TB N Engl J Med EXCLUDE LETTER Idigoras, P Rapid detection of tuberculous and non tuberculous mycobacteeur J Clin Microbio INCLUDE NO DST RESULTS Ingham, C. J Rapid drug susceptibility testing of mycobacteria by culture on aint J Tuberc Lung D EXCLUDE NOT MODS/TLA Irfan, S Evaluation of a microcolony detection method and phage assay Southeast Asian J T INCLUDE NO DST RESULTS Katoch, V. M New generation methods for drug susceptibility testing for tube Indian J Tuberc EXCLUDE EDITORIAL Lee, S Evaluation of a modified antimycobacterial susceptibility test us J Microbiol Method EXCLUDE NOT MODS/TLA Martin, A Thin layer agar compared to BACTEC MGIT 960 for early detectioj Microbiol Method INCLUDE NO DST RESULTS Martin, A Implementation of the thin layer agar for the diagnosis of smearj Clin Microbiol INCLUDE NO DST RESULTS Martin, A. unpublished Thin layer ager method for the detection of resistance to rifampicin, ofloxacin and INCLUDE McGowan, J. E., Jr Laboratory diagnosis of tuberculosis: past, present, and future J Med Assoc Ga EXCLUDE REVIEW Mejia, G. I [Five year experience with thin layer agar medium for rapid diagbiomedica INCLUDE NO DST RESULTS Mejia, G. I Microcolony detection in 7H11 thin layer culture is an alternativ Int J Tuberc Lung D INCLUDE NO DST RESULTS Mello, F. C Clinical evaluation of the microscopic observation drug suscepti J Clin Microbiol INCLUDE Mengatto, L Evaluation of rapid alternative methods for drug susceptibility temem Inst Oswaldo INCLUDE Michael unpublished Diagnostic accuracy of microscopic observation drug susceptibility assay (MODS) fo INCLUDE NO DST RESULTS Moore, D. A Microscopic observation drug susceptibility assay for the diagnon Engl J Med INCLUDE Moore, D. A Infrequent MODS TB culture cross contamination in a high burddiagn Microbiol In EXCLUDE NO OUTCOMES OF INTEREST Moore, D. A. J Future prospects for the MODS assay in multidrug resistant tubefuture Microbiolog EXCLUDE EDITORIAL Moore, D. A. J Microscopic observation drug susceptibility assay, a rapid, reliabjournal of Clinical M INCLUDE Muralidhar, S Evaluation of three methods to determine the antimicrobial sus Indian J Med Res EXCLUDE NOT MODS/TLA Nathavitharana, R Solar disinfection of MODS mycobacterial cultures in resource pplos ONE EXCLUDE NO OUTCOMES OF INTEREST Nic Fhogartaigh, C Physician initiated courtesy MODS testing for TB and MDR TB d Int J Tuberc Lung D EXCLUDE NO OUTCOMES OF INTEREST Oberhelman, R. A Improved recovery of Mycobacterium tuberculosis from childre Pediatrics INCLUDE NO DST RESULTS Pai, M New tools and emerging technologies for the diagnosis of tuber Expert Rev Mol Dia EXCLUDE REVIEW

33 Palacios, J. J Comparison of solid and liquid culture media with polymerase c Eur J Clin Microbio EXCLUDE NOT MODS/TLA Palacios, J. J Comparison of the ligase chain reaction with solid and liquid cul Eur J Clin Microbio EXCLUDE NOT MODS/TLA Palomino, J. C Microcolony detection for rapid diagnosis of Mycobacterium tubint J Tuberc Lung D EXCLUDE LETTER Palomino, J. C MODS assay for the diagnosis of TB N Engl J Med EXCLUDE LETTER Park, W. G Performance of the microscopic observation drug susceptibility J Clin Microbiol INCLUDE Robledo, J Rapid detection of rifampicin and isoniazid resistance in MycobaInt J Tuberc Lung D INCLUDE Robledo, J. A Evaluation of a rapid culture method for tuberculosis diagnosis: Int J Tuberc Lung D INCLUDE NO DST RESULTS Schaberg, T Rapid drug susceptibility testing of Mycobacterium tuberculosis Eur Respir J INCLUDE Shibayama, H Mycobacteriae isolation using silicone coated slide microcultureboletin de la Oficin EXCLUDE NOT MODS/TLA Shiferaw, G Evaluation of microscopic observation drug susceptibility assay fj Clin Microbiol INCLUDE Smith, W. L The use of spiral plating and microscopic colony counting for thelett Appl Microbio EXCLUDE NOT MODS/TLA Tovar, M Improved diagnosis of pleural tuberculosis using the microscopi Clin Infect Dis INCLUDE NO DST RESULTS Welch, D. F Timely culture for mycobacteria which utilizes a microcolony mej Clin Microbiol INCLUDE NO DST RESULTS Yagui, M Timely diagnosis of MDR TB under program conditions: is rapid Int J Tuberc Lung D EXCLUDE NOT MODS/TLA