Online Annexes (2-4) to WHO Policy update: The use of molecular line probe assays for the detection of resistance to isoniazid and rifampicin THE END TB STRATEGY
Online Annexes (2-4) to WHO Policy update: The use of molecular line probe assays for the detection of resistance to isoniazid and rifampicin
Contents Annex 2. Grade evidence profiles...3 Annex 3. Evidence to recommendations frameworks...10 Annex 4. References to studies excluded from the review of the diagnostic accuracy of line probe assays...48
ANNEX 2. GRADE EVIDENCE PROFILES POLICY UPDATE 3 Table 5. Accuracy of LPAs by direct testing for detecting rifampicin resistance in patients with signs and symptoms of TB Participants: Patients with signs and symptoms of TB Prior testing: None Role: Replacement test for culture-based drug-susceptibility testing Settings: Intermediate- or central-level laboratories Reference standard: Culture-based drug-susceptibility testing Studies: Case control or cohort studies comparing LPAs with a reference standard Index (new) tests: GenoType MTBDRplus version 1 assay (Hain Lifesciences, Nehren, Germany); GenoType MTBDRplus version 2 assay (Hain Lifesciences, Nehren, Germany); Nipro NTM+MDRTB detection kit 2 (Nipro, Tokyo, Japan). The tests were performed by direct testing on smear-positive specimens. Sensitivity 0.96 (95% CI: 0.95 0.97) Specificity 0.98 (95% CI: 0.97 0.99) Outcome Number of studies (number of patients) Study design Factors that may decrease the quality of evidence Risk of bias Indirectness Inconsistency Imprecision Publication bias Effect per 1 000 patients tested (number of patients) Pre-test Pre-test probability probability of 5% of 15% Test accuracy quality of evidence True positives (patients with rifampicin False negatives (patients incorrectly classified as not having rifampicin True negatives (patients without rifampicin 48 studies (2 876 patients) 48 studies (7 684 patients) Cohort and case controltype studies Cohort and case controltype studies Serious a Not serious b Not serious c Not serious d None 48 (47 49) 144 (142 146) 2 (1 3) 6 (4 8) Serious a Not serious b Not serious c Not serious e None 933 (923 939) 835 (826 840) Moderate False positives (patients incorrectly classified as having rifampicin 17 (11 27) 15 (10 24) Moderate a The QUADAS-2 tool was used to assess the risk of bias. The risk of bias was unclear for many studies, primarily with respect to the patient-selection domain (33/48 studies), because the method of patient sampling was unspecified (for example, consecutive or random). There was also uncertainty in the index-test and reference-test domains because many studies did not specify whether the operators of the index test and the reference test were blinded to the results of the other test (30/48 and 32/48, respectively). The risk of bias was low for the flow and timing domain. The evidence was downgraded by one point. b There was low concern about applicability. Given the tests high specificity and ability to provide results within a matter of days, the tests might improve patients outcomes by enabling earlier initiation of appropriate therapy. The evidence was not downgraded. c Although some heterogeneity was noted, this was predominantly driven by a few, small outlier studies. d Imprecision was considered to be present when the pooled confidence intervals were wider than 10% in either direction. e Imprecision was considered to be present when the pooled confidence intervals were wider than 5% in either direction.
4 ANNEX 2. GRADE EVIDENCE PROFILES POLICY UPDATE Table 6. Accuracy of LPAs for detecting rifampicin resistance by indirect testing of Mycobacterium tuberculosis complex culture isolates Participants: Patients with signs and symptoms of TB Prior testing: None Role: Replacement test for culture-based drug-susceptibility testing Settings: Intermediate- or central-level laboratories Reference standard: Culture-based drug-susceptibility testing Studies: Case control or cohort studies comparing LPAs with a culture-based drugsusceptibility reference test Index (new) tests: GenoType MTBDRplus version 1 assay (Hain Lifesciences, Nehren, Germany); GenoType MTBDRplus version 2 assay (Hain Lifesciences, Nehren, Germany); Nipro NTM+MDRTB detection kit 2 (Nipro, Tokyo, Japan). The tests were performed by indirect testing on culture isolates. Sensitivity 0.97 (95% CI: 0.95 0.98) Specificity 0.99 (95% CI: 0.99 1.00) Outcome Number of studies (number of patients) Study design Factors that may decrease the quality of evidence Risk of bias Indirectness Inconsistency Imprecision Publication bias Effect per 1 000 patients tested (number of patients) Pre-test Pre-test probability probability of 5% of 15% Test accuracy quality of evidence True positives (patients with rifampicin False negatives (patients incorrectly classified as not having rifampicin True negatives (patients without rifampicin 43 studies (3 913 patients) 43 studies (6 783 patients) Cohort and case controltype studies Cohort and case controltype studies Serious a Not serious b Not serious c Not serious d None 48 (48 49) 145 (143 147) 2 (1 2) 5 (3 7) Serious a Not serious b Not serious c Not serious e None 943 (937 946) 844 (838 847) Moderate False positives (patients incorrectly classified as having rifampicin 7 (4 13) 6 (3 12) Moderate a The QUADAS-2 tool was used to assess the risk of bias. The risk of bias was unclear for many studies, primarily with respect to the patient-selection domain (23/43 studies), because the method of patient sampling was unspecified (for example, consecutive or random). There was also uncertainty in the index-test and reference-test domains because many studies did not specify whether the operators of the index test and the reference test were blinded to the results of the other test (36/43 and 36/43, respectively). The risk of bias was low for the flow and timing domain. The evidence was downgraded by one point. b There was low concern about applicability. Given the tests high specificity and ability to provide results within a matter of days, the tests might improve patients outcomes by enabling earlier initiation of appropriate therapy. The evidence was not downgraded. c Although some heterogeneity was noted, this was predominantly driven by a few, small outlier studies. d Imprecision was considered to be present when the pooled confidence intervals were wider than 10% in either direction. e Imprecision was considered to be present when the pooled confidence intervals were wider than 5% in either direction.
ANNEX 2. GRADE EVIDENCE PROFILES POLICY UPDATE 5 Table 7. Accuracy of LPAs for detecting rifampicin resistance by indirect testing of Mycobacterium tuberculosis complex culture isolates compared with a composite reference standard Participants: Patients with signs and symptoms of TB Studies: Case control or cohort studies comparing LPAs with a reference standard Prior testing: None Index (new) tests: GenoType MTBDRplus version 1 assay (Hain Lifesciences, Nehren, Role: Replacement test for culture-based drug-susceptibility testing Germany); GenoType MTBDRplus version 2 assay (Hain Lifesciences, Nehren, Settings: Intermediate- or central-level laboratories Germany); Nipro NTM+MDRTB detection kit 2 (Nipro, Tokyo, Japan). The tests were Reference standard: Composite reference standard performed by indirect testing of Mycobacterium tuberculosis complex culture isolates. Sensitivity 0.95 (95% CI: 0.93 0.97) Specificity 0.99 (95% CI: 0.99 1.00) Outcome True positives (patients with rifampicin False negatives (patients incorrectly classified as not having rifampicin True negatives (patients without rifampicin Number of studies (number of patients) 43 studies (3 913 patients) 23 studies (3 392 patients) Study design Cohort and case controltype studies a Cohort and case controltype studies a Factors that may decrease the quality of evidence Risk of bias Indirectness Inconsistency Imprecision Publication bias Effect per 1 000 patients tested (number of patients) Pre-test Pre-test probability probability of 5% of 15% Serious b Not serious c Not serious d Not serious e None 48 (47 48) 143 (140 145) 2 (2 3) 7 (5 10) Serious b Not serious c Not serious d Not serious f None 945 (937 948) 846 (838 848) Test accuracy quality of evidence Moderate False positives 5 (2 13) 4 (2 12) Moderate (patients incorrectly classified as having rifampicin a The QUADAS-2 tool was used to assess the risk of bias. In total, 8/23 studies were cross-sectional; 8/23 were case control; and 7 studies had an unclear design. b The risk of bias was unclear for many studies, primarily with respect to the patient-selection domain (12/23 studies), because the method of patient sampling was unspecified (for example, consecutive or random). Additionally, 8/23 studies were assessed as having a high risk of bias due to the use of a case control design. Also, there was uncertainty in the index-test and reference-test domains because many studies did not specify whether the operators of the index test and the reference test were blinded to the results of the other test (14/23 and 15/23, respectively). The risk of bias was low for the flow and timing domain. The evidence was downgraded by one point. c Applicability was judged to be of low concern in the majority of studies because the population and the use of the index test matched the population of interest and the settings of intended use. The evidence was not downgraded. d Although some heterogeneity was noted, this was predominantly driven by a few, small outlier studies. The evidence was not downgraded. e Imprecision was considered to be present when the pooled confidence intervals were wider than 10% in either direction. The evidence was not downgraded. f Imprecision was considered to be present when the pooled confidence intervals were wider than 5% in either direction. The evidence was not downgraded.
6 ANNEX 2. GRADE EVIDENCE PROFILES POLICY UPDATE Table 8. Accuracy of LPAs by direct testing for detecting isoniazid resistance in patients with signs and symptoms of TB Participants: Patients with signs and symptoms of TB Prior testing: None Role: Replacement test for culture-based drug-susceptibility testing Settings: Intermediate- or central-level laboratories Reference standard: Culture-based drug-susceptibility testing Studies: Case control or cohort studies comparing LPAs with a reference standard Index (new) tests: GenoType MTBDRplus version 1 assay (Hain Lifesciences, Nehren, Germany); GenoType MTBDRplus version 2 assay (Hain Lifesciences, Nehren, Germany); Nipro NTM+MDRTB detection kit 2 (Nipro, Tokyo, Japan). The tests were performed by direct testing on smear-positive specimens. Sensitivity 0.89 (95% CI: 0.86 0.92) Specificity 0.98 (95% CI: 0.97 0.99) Outcome True positives (patients with isoniazid False negatives (patients incorrectly classified as not having isoniazid True negatives (patients without isoniazid False positives (patients incorrectly classified as having isoniazid Number of studies (number of patients) 46 studies (3 576 patients) 46 studies (6 896 patients) Study design Cohort and case controltype studies Cross-sectional (cohort-type accuracy study) Factors that may decrease the quality of evidence Risk of bias Indirectness Inconsistency Imprecision Publication bias Effect per 1 000 patients tested (number of patients) Pre-test probability of 5% Serious a Not serious b Not serious c Not serious d None 45 (43 46) 5 (4 7) Serious a Not serious b Not serious c Not serious e None 935 (926 940) 15 (10 24) Pre-test probability of 15% 134 (129 138) 16 (12 21) 836 (829 841) 14 (9 21) Pre-test probability of 90% 803 (772 827) 97 (73 128) 98 (97 99) 2 (1 3) Test accuracy quality of evidence Moderate Moderate a The QUADAS-2 tool was used to assess the risk of bias. The risk of bias was unclear for many studies, primarily with respect to the patient-selection domain (32/47 studies), because the method of patient sampling was unspecified (for example, consecutive or random). There was also uncertainty in the index-test and reference-test domains because many studies did not specify whether the operators of the index test and the reference test were blinded to the results of the other test (30/47 and 32/47, respectively). The risk of bias was low for the flow and timing domain. b Applicability was judged to be of low concern in the majority of studies because the population and the use of the index test matched the population of interest and the settings of intended use. c Although some heterogeneity was noted, this was predominantly driven by a few, small outlier studies. d Imprecision was considered to be present when the pooled confidence intervals were wider than 10% in either direction and the number of resistant specimens tested was < 15. e Imprecision was considered to be present when the pooled confidence intervals were wider than 5% in either direction and the number of sensitive specimens tested was < 15.
ANNEX 2. GRADE EVIDENCE PROFILES POLICY UPDATE 7 Table 9. Accuracy of LPAs for detecting isoniazid resistance by indirect testing of Mycobacterium tuberculosis complex culture isolates Participants: Patients with signs and symptoms of TB Prior testing: None Role: Replacement test for culture-based drug-susceptibility testing Settings: Intermediate- or central-level laboratories Reference standard: Culture-based drug-susceptibility testing Studies: Case control or cohort studies comparing LPAs with a reference standard Index (new) tests: GenoType MTBDRplus version 1 assay (Hain Lifesciences, Nehren, Germany); GenoType MTBDRplus version 2 assay (Hain Lifesciences, Nehren, Germany); Nipro NTM+MDRTB detection kit 2 (Nipro, Tokyo, Japan). The tests were performed by indirect testing on Mycobacterium tuberculosis complex culture isolates. Sensitivity 0.91 (95% CI: 0.89 0.93) Specificity 1.00 (95% CI: 0.99 1.00) Outcome True positives (patients with isoniazid False negatives (patients incorrectly classified as not having isoniazid True negatives (patients without isoniazid False positives (patients incorrectly classified as having isoniazid Number of studies (number of patients) 43 studies (4 559 patients) 43 studies (5 903 patients) Study design Cohort and case controltype studies a Cohort and case controltype studies a Factors that may decrease the quality of evidence Risk of bias Indirectness Inconsistency Imprecision Publication bias Effect per 1 000 patients tested (number of patients) Pre-test probability of 5% Serious b Not serious c Not serious d Not serious e None 46 (44 47) 4 (3 6) Serious b Not serious c Not serious d Not serious f None 947 (943 950) 3 (0 7) Pre-test probability of 15% 137 (133 140) 13 (10 17) 847 (844 850) 3 (0 6) Pre-test probability of 90% 819 (797 837) 81 (63 103) 100 (99 100) 0 (0 1) Test accuracy quality of evidence Moderate Moderate a The QUADAS-2 tool was used to assess the risk of bias. In total, 21/43 datasets were cross-sectional; 8/43 were case control; 2/43 datasets evaluated only strains from cases known to have MDR-TB without testing any controls; and 12/43 studies had an unclear design (for example, this includes studies in which the method of participant selection was unclear or there was uncertainty about whether specimens had been chosen for their resistance pattern). b The risk of bias was unclear for many studies, primarily with respect to the patient-selection domain (21/43 studies), because the method of patient sampling was unspecified (for example, consecutive or random). There was also uncertainty in the index-test and reference-test domains because many studies did not specify whether the operators of the index test and the reference test were blinded to the results of the other test (33/43 and 33/43, respectively). The risk of bias was low for the flow and timing domain. c Applicability was judged to be of low concern in the majority of studies because the population and the use of the index test matched the population of interest and the settings of intended use. d Although some heterogeneity was noted, this was predominantly driven by a few, small outlier studies. e Imprecision was considered to be present when the pooled confidence intervals were wider than 10% in either direction and the number of resistant specimens tested was < 15. f Imprecision was considered to be present when the pooled confidence intervals were wider than 5% in either direction and the number of sensitive specimens tested was < 15.
8 ANNEX 2. GRADE EVIDENCE PROFILES POLICY UPDATE Table 10. Accuracy of LPAs for detecting isoniazid resistance in patients with signs and symptoms of TB compared with a composite reference standard Participants: Patients with signs and symptoms of TB Prior testing: None Role: Replacement test for culture-based drug-susceptibility testing Settings: Intermediate- or central-level laboratories Reference standard: Composite reference standard Studies: Case control or cohort studies comparing LPAs with a composite reference standard Index (new) tests: GenoType MTBDRplus version 1 assay (Hain Lifesciences, Nehren, Germany); GenoType MTBDRplus version 2 assay (Hain Lifesciences, Nehren, Germany); Nipro NTM+MDRTB detection kit 2 (Nipro, Tokyo, Japan). Sensitivity 0.85 (95% CI: 0.81 0.89) Specificity 1.00 (95% CI: 1.00 1.00) Outcome True positives (patients with isoniazid False negatives (patients incorrectly classified as not having isoniazid True negatives (patients without isoniazid False positives (patients incorrectly classified as having isoniazid Number of studies (number of patients) 24 studies (2 346 patients) 24 studies (2 170 patients) Study design Cohort and case controltype studies Cohort and case controltype studies Factors that may decrease the quality of evidence Risk of bias Indirectness Inconsistency Imprecision Publication bias Effect per 1 000 patients tested (number of patients) Pre-test probability of 5% Serious a Not serious b Not serious c Not serious d None 43 (40 44) 7 (6 10) Serious a Not serious b Not serious c Not serious d None 949 (946 950) 1 (0 4) Pre-test probability of 15% 128 (121 133) 22 17 29) 849 (847 850) 1 (0 3) Pre-test probability of 90% f 766 (727 797) 134 (103 173) 100 (100 100) 0 (0 0) Test accuracy quality of evidence Moderate Moderate a The QUADAS-2 tool was used to assess the risk of bias. The risk of bias was unclear for many studies, primarily with respect to the patient-selection domain (13/24 studies), because the method of patient sampling was unspecified (for example, consecutive or random). Also, 9/24 studies were assessed as having a high risk of bias. There was also uncertainty in the index-test and reference-test domains because many studies did not specify whether the operators of the index test and the reference test were blinded to the results of the other test (63/90 and 65/90, respectively). The risk of bias was low for the flow and timing domain. The evidence was downgraded by one point. b Applicability was judged to be of low concern in the majority of studies because the population and the use of the index test matched the population of interest and the settings of intended use. The evidence was not downgraded. c Although some heterogeneity was noted, this was predominantly driven by a few, small outlier studies. The evidence was not downgraded. d Imprecision was considered to be present when the pooled confidence intervals were wider than 10% in either direction. The evidence was not downgraded. e Imprecision was considered to be present when the pooled confidence intervals were wider than 5% in either direction. The evidence was not downgraded. f A 90% prevalence was chosen to reflect the scenario in which molecular drug-susceptibility testing has already identified rifampicin resistance that is, when the negative predictive value of this test is lower.
ANNEX 2. GRADE EVIDENCE PROFILES POLICY UPDATE 9 Table 11. Accuracy of LPAs for diagnosing MDR-TB on all specimen types by direct and indirect testing Participants: Patients with signs and symptoms of TB Prior testing: None Role: Replacement test for culture-based drug-susceptibility testing Settings: Intermediate- or central-level laboratories Reference standard: Culture-based drug-susceptibility testing Studies: Case control or cohort studies comparing LPAs with a reference standard Index (new) tests: GenoType MTBDRplus version 1 assay (Hain Lifesciences, Nehren, Germany); GenoType MTBDRplus version 2 assay (Hain Lifesciences, Nehren, Germany); Nipro NTM+MDRTB detection kit 2 (Nipro, Tokyo, Japan). The tests were performed on all types of specimens using direct and indirect testing. Sensitivity 0.93 (95% CI: 0.90 0.95) Specificity 0.99 (95% CI: 0.99 1.00) Outcome True positives (patients with MDR-TB) False negatives (patients incorrectly classified as not having MDR-TB) True negatives (patients without MDR-TB) False positives (patients incorrectly classified as having MDR-TB) Number of studies (number of patients) 60 studies (4 248 patients) 60 studies (8 785 patients) Study design Cohort and case controltype studies a Cohort and case controltype studies a Factors that may decrease the quality of evidence Risk of bias Indirectness Inconsistency Imprecision Publication bias Effect per 1 000 patients tested (number of patients) Pre-test probability of 1% Serious b Not serious c Not serious d Not serious e None 9 (9 9) 1 (1 1) Serious b Not serious c Not serious d Not serious None 983 (977 986) 7 (4 13) Pre-test probability of 5% 46 (45 47) 4 (3 5) 943 (938 946) 7 (4 12) Pre-test probability of 10% 93 (90 95) 7 (5 10) 894 (888 896) 6 (4 12) Test accuracy quality of evidence Moderate Moderate a In total, 37/60 studies were cross-sectional; 8/60 studies used a case control or cases-only design; and 15/60 studies had an unclear design. b The QUADAS-2 tool was used to assess methodological quality. The risk of bias was unclear for many studies, primarily with respect to the patient-selection domain (34/60 studies), because the method of patient sampling was unspecified (for example, consecutive or random); the risk of bias was considered to be high for the 12 studies that used a case control design. There was also uncertainty in the index-test and reference-test domains because many studies did not specify whether the operators of the index test and the reference test were blinded to the results of the other test (37/60 and 39/60, respectively). The risk of bias was low for the flow and timing domain. The evidence was downgraded by one point. c Applicability was judged to be of low concern in the majority of studies because the population and the use of the index test matched the population of interest and the settings of intended use. d Although some heterogeneity was noted for sensitivity, this was predominantly driven by a few, small outlier studies. The estimates for specificity were more homogeneous. The evidence was not downgraded. e Imprecision was considered to be present when the pooled confidence intervals were wider than 10% in either direction. The evidence was not downgraded.
10 ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE Table 12. Accuracy of LPAs for detecting rifampicin resistance by direct testing in sputum smear-positive TB patients compared with phenotypic culture-based DST Undesirable Effects Desirable Effects Test accuracy Problem Judgement Research evidence Additional considerations Is the problem a priority? No Probably no Probably yes R Yes How accurate is the test? Very inaccurate Inaccurate R Accurate Very accurate How substantial are the desirable anticipated effects? Trivial Small Moderate R Large How substantial are the undesirable anticipated effects? Large Moderate R Small Trivial Currently, only 26% of an estimated 480 000 cases of MDR-TB are diagnosed, and often a diagnosis of MDR-TB comes too late. This is in large part due to a lack of access to accurate and rapid diagnostics. LPAs are able to detect Mycobacterium tuberculosis and resistance to rifampicin and isoniazid. LPAs normally take at least 1 working day to perform and require a controlled laboratory infrastructure. Test accuracy LPA for direct testing compared with phenotypic DST Sensitivity: 0.96 (95% CI: 0.95 0.97); specificity: 0.98 (95% CI: 0.97 0.99) Test result True positives (patients with rifampicin False negatives (patients incorrectly classified as not having rifampicin True negatives (patients without rifampicin False positives (patients incorrectly classified as having rifampicin Number of results per 1000 patients tested (95% CI) 5% Prevalence 48 (47 49) 2 (1 3) 933 (923 939) 17 (11 27) 15% Prevalence 144 (142 146) 6 (4 8) 835 (826 840) 15 (10 24) Number of participants (number of studies) 2 876 (48) 7 684 (48) Quality of the evidence (GRADE) Moderate Moderate The decrease in the time to results is a critical reason for the large benefits. LPA results are more likely to be interpretable compared with results from culture-based DST. Benefits are greater when direct LPA is compared with indirect. The toxic effects of anti-tb agents on patients who are false positive by LPA are of concern. When a composite reference standard is used, some of the false positives may become true positives, thus improving sensitivity.
ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE 11 Certainty of the evidence of test accuracy Certainty of the evidence of test s effects Certainty of the evidence of management s effects What is the overall certainty of the evidence of test accuracy? Very low Low R Moderate High No included studies What is the overall certainty of the evidence for any critical or important direct benefits, adverse effects or burden of the test? Very low Low Moderate High R No included studies What is the overall certainty if the evidence of effects of the management that is guided by the test results? Very low Low Moderate High R No included studies The risk of bias was considered to be serious for all studies. Indirectness was considered not to be serious. Inconsistency was considered not to be serious. Imprecision was considered not to be serious. Publication bias: none. No studies were included. In theory, test results should guide management decisions, provided that the use of the test is adopted as national policy. Given the high accuracy of LPAs, a positive test result should be sufficient to start treating a patient. There are insufficient data about how the test performs in smear-negative samples.
12 ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE Certainty of the evidence of test result/management Certainty of effects Values How certain is the link between test results and management decisions? Very low Low R Moderate High No included studies What is the overall certainty of the evidence of effects of the test? Very low Low R Moderate High No included studies Is there important uncertainty about or variability in how much people value the main outcomes? Important uncertainty or variability Possibly important uncertainty or variability Probably no important uncertainty or variability R No important uncertainty or variability No known undesirable outcomes Although this systematic review was not designed to evaluate the clinical impact of LPAs, it was noted that 12 studies attempted to measure the impact of LPAs on clinical impacts, such as turnaround time and cost. For turnaround time, most studies reported the time from a positive culture result to LPA results, with results varying from 8 hours to 5 days and most reporting 1 to 2 days. This was faster than phenotypic DST with liquid cultures, which typically took 9 to 25 days, and solid cultures, which took more than 30 days. One systematic review focused on reductions in diagnostic and treatment delays. The analysis showed that using LPAs reduced diagnostic delays by an average of 47 days (95% CI: 29 64) compared with culture. This question is intended to summarize information from the previous four questions about the certainty of the evidence. There is no important uncertainty or variability in how people value the main outcomes. For detecting rifampicin resistance: LPAs have demonstrated good diagnostic accuracy when compared with both the phenotypic as well as the composite reference standard.
ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE 13 Balance of effects Resources required Certainty of evidence of required resources Does the balance between desirable and undesirable effects Favour the intervention or the comparison? Favours the comparison Probably Favours the comparison Does not Favour either the intervention or the comparison Probably Favours the intervention R Favours the intervention How large are the resource requirements (costs)? Large costs Moderate costs Negligible costs and savings Moderate savings Large savings R Don t know What is the certainty of the evidence of resource requirements (costs)? Very low Low Moderate High R No included studies LPAs good performance in sensitivity and specificity for detecting rifampicin resistance indicates that they are accurate tests, with small numbers of false-negative and false-positive results. Reductions in diagnostic and treatment delays have been documented. Cost and cost effectiveness studies were not assessed. Potential areas needing investment include infrastructure, sample referral procedures, equipment and maintenance. Based on a cost effectiveness study done in 2011, LPAs are cost effective compared with conventional DST Cost and cost effectiveness studies were not assessed for this guideline. Potential areas needing investment include infrastructure, sample referral procedures, equipment and maintenance.
14 ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE Cost effectiveness Equity Acceptability Does the cost-effectiveness of the intervention Favour the intervention or the comparison? Favours the comparison Probably Favours the comparison Does not Favour either the intervention or the comparison Probably Favours the intervention Favours the intervention R No included studies What would be the impact on health equity? Reduced Probably reduced Probably no impact R Probably increased Increased Is the intervention acceptable to key stakeholders? No Probably no R Probably yes Yes Cost and cost effectiveness studies were not assessed for this guideline. Potential areas needing investment include infrastructure, sample referral procedures, equipment and maintenance. Because more patients would have access to the test, health equity may be positively affected. The test may be acceptable for implementation in settings with a high prevalence of MDR-TB. Implementing the test requires additional human resources, as it is labour intensive, as well as additional infrastructure (three separate rooms) and increased biosafety standards. For patients, the burdens and adverse effects are potentially insignificant.
ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE 15 Feasibility Is the intervention feasible to implement? No Probably no R Probably yes Yes In 2008, WHO recommended using this test to diagnose rifampicin-resistant TB in AFB-positive smears and cultures. During the Guideline Development Group meeting there was some disagreement about how feasible it would be to implement LPAs. A sophisticated laboratory infrastructure and skilled staff are required to perform the test, which are usually available at the intermediate- and reference-levels of laboratory networks. Hence, implementing the test would require additional funding and technical support to train staff and procure equipment. Quality assurance strategies will be needed as well. AFB: acid-fast bacilli; CI: confidence interval; DST: drug-susceptibility testing; GRADE: Grading of Recommendations Assessment, Development and Evaluation; LPA: line probe assay; MDR-TB: multidrug-resistant TB.
16 ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE Summary of judgments Judgement Problem No Probably no Probably yes Yes Varies Don t know Test accuracy Very inaccurate Inaccurate Accurate Very accurate Varies Don t know Desirable effects Trivial Small Moderate Large Varies Don t know Undesirable effects Large Moderate Small Trivial Varies Don t know Certainty of the evidence of the Very low Low Moderate High No included studies test s accuracy Certainty of the evidence of the Very low Low Moderate High No included studies test s effects Certainty of the evidence of effects Very low Low Moderate High No included studies on management Certainty of the evidence of link between test result Very low Low Moderate High No included studies and management Certainty of effects Very low Low Moderate High No included studies Values Balance of effects Important uncertainty or variability Favours the comparison Possibly important uncertainty or variability Probably favours the comparison Probably no important uncertainty or variability Does not favour either the intervention or the comparison No important uncertainty or variability Probably favours the intervention Favours the intervention No known undesirable outcomes Varies Don t know Implications
ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE 17 Resources required Large costs Moderate costs Certainty of evidence of required resources Cost effectiveness Equity Negligible costs and savings Judgement Moderate savings Large savings Varies Don t know Very low Low Moderate High No included studies Favours the comparison Reduced Probably Favours the comparison Probably reduced Does not Favour either the intervention or the comparison Probably no impact Probably Favours the intervention Probably increased Favours the intervention Varies No included studies Increased Varies Don t know Acceptability No Probably no Probably yes Yes Varies Don t know Feasibility No Probably no Probably yes Yes Varies Don t know Implications
18 ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE Conclusions Should LPA by direct testing (compared with phenotypic DST) be used to detect rifampicin resistance in pulmonary TB? Type of recommendation Recommendation Justification Subgroup considerations Implementation considerations Monitoring and evaluation Research priorities Strong recommendation against the intervention Conditional recommendation against the intervention Conditional recommendation for either the intervention or the comparison Conditional recommendation for the intervention R Strong recommendation for the intervention For patients with a sputum smear-positive specimen, the WHO guideline panel suggests using direct LPA for the detection of rifampicin resistance instead of phenotypic DST (conditional recommendation, moderate certainty in the evidence for test accuracy). There is uncertainty about the impact on cost. Feasibility concerns are moderated by international roll-out of LPA but cannot be ignored; patients who have rifampicin monoresistance by LPA should still have specimens cultured. Positive results should be interpreted with caution in settings with a very low prevalence of rifampicin resistance; such results possibly require confirmation and repeat testing, but therapy should not be delayed. Implementation should be phased-in gradually along with biosafety upgrades, starting at reference-level laboratories. Facilities requirements must be met (three separate rooms); there must be adequate supplies; and quality assurance strategies must be implemented, as well as reporting mechanisms. Staff training and internal laboratory procedures may need to be revised and changes should be implemented as necessary. Clinicians will need aids for interpreting results. Priorities for research include direct clinical trials to assess the impact on patient outcomes of knowing isoniazid-resistance status.
ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE 19 Table 13. Accuracy of LPAs for detecting rifampicin resistance by indirect testing of Mycobacterium tuberculosis complex culture isolates compared with phenotypic culture-based DST Undesirable Effects Desirable Effects Test accuracy Problem Judgement Research evidence Additional considerations Is the problem a priority? No Probably no Probably yes R Yes How accurate is the test? Very inaccurate Inaccurate R Accurate Very accurate How substantial are the desirable anticipated effects? Trivial Small Moderate R Large How substantial are the undesirable anticipated effects? Large Moderate R Small Trivial Currently, only 26% of an estimated 480 000 cases of MDR-TB are diagnosed, and often a diagnosis of MDR-TB comes too late. This is in large part due to a lack of access to accurate and rapid diagnostics. LPAs are able to detect Mycobacterium tuberculosis and resistance to rifampicin and isoniazid. LPAs normally take at least 1 working day to perform and require a controlled laboratory infrastructure. Test accuracy LPA for indirect testing compared with phenotypic DST Sensitivity: 0.97 (95% CI: 0.95 0.98); specificity: 0.99 (95% CI: 0.99 1.00) Test result True positives (patients with rifampicin False negatives (patients incorrectly classified as not having rifampicin True negatives (patients without rifampicin False positives (patients incorrectly classified as having rifampicin Number of results per 1000 patients tested (95% CI) 5% Prevalence 48 (48 49) 2 (1 2) 943 (937 946) 7 (4 13) 15% Prevalence 145 (143 147) 5 (3 7) 844 (838 847) 6 (3 12) Number of participants (number of studies) 3 913 (43) 6 783 (483) Quality of the evidence (GRADE) Moderate Moderate The decrease in the time to results is a critical reason for the large benefits. The time gained depends on the medium used: LPA takes at least 3 weeks less than solid culture and 1 week less than liquid culture. LPA results are more likely to be interpretable compared with results from culture-based DST. Benefits are greater when direct LPA is compared with indirect. The toxic effects of anti-tb agents on patients who are false positive by LPA are of concern. When a composite reference standard is used, some of the false positives may become true positives, thus improving sensitivity.
20 ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE Certainty of the evidence of test accuracy Certainty of the evidence of test s effects Certainty of the evidence of management s effects What is the overall certainty of the evidence of test accuracy? Very low Low R Moderate High No included studies What is the overall certainty of the evidence for any critical or important direct benefits, adverse effects or burden of the test? Very low Low Moderate High R No included studies What is the overall certainty if the evidence of effects of the management that is guided by the test results? Very low Low Moderate High R No included studies The risk of bias was considered to be serious for all studies. Indirectness was considered not to be serious. Inconsistency was considered not to be serious. Imprecision was considered not to be serious. Publication bias: none. No studies were included. In theory, test results should guide management decisions, provided that the use of the test is adopted as national policy. Given the high accuracy of LPAs, a positive test result should be sufficient to start treating a patient. There are insufficient data about how the test performs in smear-negative samples.
ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE 21 Certainty of the evidence of test result/management Certainty of effects Values How certain is the link between test results and management decisions? Very low Low Moderate High R No included studies What is the overall certainty of the evidence of effects of the test? Very low Low R Moderate High No included studies Is there important uncertainty about or variability in how much people value the main outcomes? Important uncertainty or variability Possibly important uncertainty or variability Probably no important uncertainty or variability R No important uncertainty or variability No known undesirable outcomes Although this systematic review was not designed to evaluate the clinical impact of LPAs, it was noted that 12 studies attempted to measure the impact of LPAs on clinical impacts, such as turnaround time and cost. For turnaround time, most studies reported the time from a positive culture result to LPA results, with results varying from 8 hours to 5 days and most reporting 1 to 2 days. This was faster than phenotypic DST with liquid cultures, which typically took 9 to 25 days, and solid cultures, which took more than 30 days. One systematic review focused on reductions in diagnostic and treatment delays. The analysis showed that using LPAs reduced diagnostic delays by an average of 47 days (95% CI: 29 64) compared with culture. This question is intended to summarize information from the previous four questions about the certainty of the evidence. There is no important uncertainty or variability in how people value the main outcomes. For detecting rifampicin resistance: LPAs have demonstrated good diagnostic accuracy when compared with both the phenotypic as well as the composite reference standard.
22 ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE Balance of effects Resources required Certainty of evidence of required resources Does the balance between desirable and undesirable effects Favour the intervention or the comparison? Favours the comparison Probably Favours the comparison Does not Favour either the intervention or the comparison Probably Favours the intervention R Favours the intervention How large are the resource requirements (costs)? Large costs Moderate costs Negligible costs and savings Moderate savings Large savings R Don t know What is the certainty of the evidence of resource requirements (costs)? Very low Low Moderate High R No included studies LPAs good performance in sensitivity and specificity for detecting rifampicin resistance indicates that they are accurate tests, with small numbers of false-negative and false-positive results. Reductions in diagnostic and treatment delays have been documented. Cost and cost effectiveness studies were not assessed. Potential areas needing investment include infrastructure, sample referral procedures, equipment and maintenance. Cost and cost effectiveness studies were not assessed for this guideline. Potential areas needing investment include infrastructure, sample referral procedures, equipment and maintenance.
ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE 23 Cost effectiveness Equity Acceptability Does the cost-effectiveness of the intervention Favour the intervention or the comparison? Favours the comparison Probably Favours the comparison Does not Favour either the intervention or the comparison Probably Favours the intervention Favours the intervention R No included studies What would be the impact on health equity? Reduced Probably reduced Probably no impact R Probably increased Increased Is the intervention acceptable to key stakeholders? No Probably no R Probably yes Yes Cost and cost effectiveness studies were not assessed for this guideline. Potential areas needing investment include infrastructure, sample referral procedures, equipment and maintenance. Because more patients would have access to the test, health equity may be positively affected. However, the test may introduce barriers to health equity in selfpayment environments. The test may be acceptable for implementation in settings with a high prevalence of MDR-TB. Implementing the test requires additional human resources, as it is labour intensive, as well as additional infrastructure (three separate rooms) and increased biosafety standards. For patients, the burdens and adverse effects are potentially insignificant.
24 ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE Feasibility Is the intervention feasible to implement? No Probably no R Probably yes Yes In 2008, WHO recommended using this test to diagnose rifampicinresistant TB in AFB-positive smears and cultures. During the Guideline Development Group meeting there was some disagreement about how feasible it would be to implement LPAs. A sophisticated laboratory infrastructure and skilled staff are required to perform the test, which are usually available at the intermediateand reference-levels of laboratory networks. Hence, implementing the test would require additional funding and technical support to train staff and procure equipment. Quality assurance strategies will be needed as well. AFB: acid-fast bacilli; CI: confidence interval; DST: drug-susceptibility testing; GRADE: Grading of Recommendations Assessment, Development and Evaluation; LPA: line probe assay; MDR-TB: multidrug-resistant TB.
ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE 25 Summary of judgments Judgement Problem No Probably no Probably yes Yes Varies Don t know Test accuracy Very inaccurate Inaccurate Accurate Very accurate Varies Don t know Desirable effects Trivial Small Moderate Large Varies Don t know Undesirable effects Large Moderate Small Trivial Varies Don t know Certainty of the evidence of the Very low Low Moderate High No included studies test s accuracy Certainty of the evidence of the Very low Low Moderate High No included studies test s effects Certainty of the evidence of effects Very low Low Moderate High No included studies on management Certainty of the evidence of link between test result Very low Low Moderate High No included studies and management Certainty of effects Very low Low Moderate High No included studies Values Balance of effects Important uncertainty or variability Favours the comparison Possibly important uncertainty or variability Probably favours the comparison Probably no important uncertainty or variability Does not favour either the intervention or the comparison No important uncertainty or variability Probably favours the intervention Favours the intervention No known undesirable outcomes Varies Don t know Implications
26 ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE Resources required Large costs Moderate costs Certainty of evidence of required resources Cost effectiveness Equity Negligible costs and savings Judgement Moderate savings Large savings Varies Don t know Very low Low Moderate High No included studies Favours the comparison Reduced Probably Favours the comparison Probably reduced Does not Favour either the intervention or the comparison Probably no impact Probably Favours the intervention Probably increased Favours the intervention Varies No included studies Increased Varies Don t know Acceptability No Probably no Probably yes Yes Varies Don t know Feasibility No Probably no Probably yes Yes Varies Don t know Implications
ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE 27 Conclusions Should LPA by direct testing (compared with phenotypic DST) be used to detect rifampicin resistance in Mycobacterium tuberculosis complex culture isolates? Type of recommendation Recommendation Justification Subgroup considerations Implementation considerations Monitoring and evaluation Research priorities Strong recommendation against the intervention Conditional recommendation against the intervention Conditional recommendation for either the intervention or the comparison Conditional recommendation for the intervention R Strong recommendation for the intervention For patients with a cultured isolate of Mycobacterium tuberculosis complex, the WHO guideline panel suggests using indirect LPA for the detection of rifampicin resistance instead of phenotypic culture-based DST on Mycobacterium tuberculosis complex culture isolates (conditional recommendation, moderate certainty in the evidence for test accuracy). There is uncertainty about the impact on cost. Feasibility concerns are moderated by international roll-out of LPA but cannot be ignored; patients who have rifampicin monoresistance by LPA should still have specimens cultured. Positive results should be interpreted with caution in settings with a very low prevalence of rifampicin resistance; such results possibly require confirmation and repeat testing, but therapy should not be delayed. Implementation should be phased-in gradually along with biosafety upgrades, starting at reference-level laboratories. Facilities requirements must be met (three separate rooms); there must be adequate supplies; and quality assurance strategies must be implemented, as well as reporting mechanisms. Staff training and internal laboratory procedures may need to be revised and changes should be implemented as necessary. Clinicians will need aids for interpreting results. Priorities for research include direct clinical trials to assess the impact on patient outcomes of knowing rifampicin-resistance status.
28 ANNEX 3. EVIDENCE TO RECOMMENDATIONS FRAMEWORKS POLICY UPDATE Table 14. Accuracy of LPAs for detecting isoniazid resistance by direct testing in sputum smear-positive TB patients compared with phenotypic culture-based DST Problem Judgement Research evidence Additional considerations Is the problem a priority? No Probably no Probably yes R Yes Mycobacterium tuberculosis causes 9 million cases of TB and 1.5 million deaths annually, and it is estimated that 3.6 million cases of TB go undiagnosed each year. The emergence of MDR-TB and XDR-TB is a major threat to global TB control. Culture and conventional DST using solid and liquid media take from 8 days to 2 months. Hence, the development of rapid molecular diagnostic tests for identifying M. tuberculosis and drug resistance have become research and implementation priorities. Test accuracy How accurate is the test? Very inaccurate Inaccurate R Accurate Very accurate LPAs detect isoniazid resistance by identifying mutations in kat G and inh A genes. However, the mutations that cause isoniazid resistance are located in several genes and regions. On average, 80 85% of isoniazid-resistant strains have been found to contain mutations in codon 315 of the kat G gene in the inh A regulatory region. Test accuracy LPA for direct testing compared with phenotypic DST Sensitivity: 0.89 (95% CI: 0.86 0.92); specificity: 0.98 (95% CI: 0.97 0.99)