CHEST RADIOGRAPH SCORING SYSTEMS FOR THE DIAGNOSIS OF ACTIVE PULMONARY TUBERCULOSIS

Transcription

1 CHEST RADIOGRAPH SCORING SYSTEMS FOR THE DIAGNOSIS OF ACTIVE PULMONARY TUBERCULOSIS Lancelot M Pinto, MD Department of Epidemiology and Biostatistics McGill University, Montreal May 2012 A thesis submitted to McGill University in partial fulfillment of the requirements of the degree of Master of Science Lancelot M Pinto

2 Table of Contents ABSTRACT... 3 RÉSUMÉ... 5 ACKNOWLEDGEMENTS... 8 PREFACE - CONTRIBUTIONS OF CO-AUTHORS... 9 CHAPTER 1: INTRODUCTION CHAPTER 2: SYSTEMATIC REVIEW OF THE LITERATURE (MANUSCRIPT 1) ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION CONCLUSIONS TABLES AND FIGURES CHAPTER 3: BRIDGING CHAPTER: THE NEED FOR A CHEST RADIOGRAPH SCORING SYSTEM FOR THE DIAGNOSIS OF PULMONARY TUBERCULOSIS CHAPTER 4: DEVELOPMENT OF A RELIABLE AND SIMPLE RADIOGRAPHIC SCORING SYSTEM TO AID THE DIAGNOSIS OF PULMONARY TUBERCULOSIS (MANUSCRIPT 2) ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION CONCLUSIONS TABLES AND FIGURES CHAPTER 5: CONCLUSIONS REFERENCES APPENDIX SEARCH STRATEGY FOR THE SYSTEMATIC REVIEW DATA EXTRACTION FORM FOR THE SYSTEMATIC REVIEW CHEST RADIOGRAPH READING AND REPORTING SYSTEM (CRRS) FORM

3 ABSTRACT Background: Chest radiography is often the only tool available for the investigation of tuberculosis (TB) suspects with negative sputum smears, thus playing a crucial role in clinical decision-making. However, chest radiographs lack specificity for TB, and their interpretation is subjective and not standardized, and therefore not highly reproducible. Efforts to improve the interpretation of chest radiography are warranted, especially with the growing use of digital radiology. Objectives: To systematically review the literature on the use of scoring systems to aid the diagnosis of active pulmonary TB (PTB), and to derive a new, simple scoring system using features noted on the Chest Radiograph Reading and Recording System (CRRS), a tool designed for the documentation of radiographic abnormalities in epidemiological surveys for PTB. Methods: A systematic review of the literature was performed to assess the utility of chest radiograph scoring systems for the diagnosis of PTB, and to use this information to derive a scoring system using the CRRS. Chest radiographs of outpatients with suspected PTB, consecutively recruited over 3 years at clinics in South Africa, were read by two independent readers using CRRS. Multivariable analysis was used to identify features significantly associated with culture-positive PTB, and these were assigned weights and used to generate a composite score. 3

4 Results: A systematic review of the literature identified 12 studies that used radiographic features as part of scoring systems for the diagnosis of PTB. Six of these were tested in smear-negative patients. There was no scoring system found that involved the exclusive use of radiographic features. Upper lobe infiltrates and cavities were the radiographic features most commonly associated with the disease. The sensitivities of the scoring systems were uniformly high, but all of them lacked specificity. For the study in South Africa, 473 patients were included in the analysis. Large upper lobe opacities, cavities, unilateral pleural effusion and adenopathy were significantly associated with culture-confirmed PTB, had high inter-reader reliability, and received 2, 2, 1 and 2 points, respectively in the final score. When applied to all TB suspects, using a cut-off of 2, the score had a high negative predictive value (92%, 95%CI 87,95). Among TB suspects with negative sputum smears, the score correctly ruled out active disease in 214 of 229 patients (NPV 93; 95%CI 89,96) Conclusions: Existing radiographic scoring systems for the diagnosis of PTB appear to be sensitive, but lack specificity. The scoring system derived from CRSS is a simple and reliable tool that may be useful for ruling out active PTB in smear-negative patients. Validation studies are needed to confirm these initial findings. 4

5 RÉSUMÉ Contexte: La radiographie thoracique est souvent le seul outil disponible pour le dépistage de la tuberculose (TB) chez les patients ayant des frottis d'expectoration négatifs, lui donnant ainsi un rôle crucial dans la prise de décision clinique. Toutefois, les radiographies thoraciques manquent de spécificité pour la tuberculose, et leur interprétation est subjective et non standardisée, et donc n est pas très reproductible. Les efforts visant à améliorer l'interprétation de la radiographie pulmonaire sont justifiés, surtout vu l'utilisation croissante de la radiologie numérique. Objectifs: Les objectifs incluent une recherche systématique de la littérature sur l'utilisation des systèmes de notation pour aider le diagnostic de la tuberculose pulmonaire active (TBP), et d'en tirer un nouveau système de notation simple à partir du Chest Radiograph Reading and Recording System (CRRS) (Système de Lecture et Notation des radiographies thoraciques), un outil conçu pour la documentation des anomalies radiologiques dans les études épidémiologiques sur la TBP. Méthodes: Une recherche systématique de la littérature a été effectuée pour évaluer l'utilité des systèmes de notation des radiographies thoraciques pour le diagnostic de la TBP, et pour utiliser ces informations pour dériver un système de notation à partir du CRRS. Les radiographies thoraciques de patients ambulatoires suspects de TBP, recrutés consécutivement sur 3 ans dans des cliniques en Afrique du Sud, ont été lues par deux lecteurs indépendants en utilisant CRRS. Une analyse multivariée a été utilisée pour identifier les caractéristiques significativement associées à la TBP à culture positive, et 5

6 ceux-ci ont reçu une importance respective et ont été utilisé pour générer un score composite. Résultats: Une recherche systématique de la littérature a identifié 12 études qui ont utilisé des systèmes de notation pour analyser les caractéristiques radiographiques dans le cours du diagnostic de la TBP. Six d'entre elles comprenaient seulement des patients à frottis négatif. Aucun système de notation ne comprenait l'usage exclusif des caractéristiques radiographiques. Des cavités et des infiltrats dans les lobes supérieurs étaient les caractéristiques radiographiques les plus couramment associées à la maladie. Les sensibilités des systèmes de notation étaient uniformément élevées, mais chacun d'eux manquait de spécificité. Dans l'étude en Afrique du Sud, 473 patients ont été inclus dans l'analyse. Les grandes opacités du lobe supérieur, les cavités, un épanchement pleural unilatéral ainsi que la présence d adénopathie étaient significativement associés à la TBP confirmée par culture, avaient un haut taux de fiabilité entre lecteur, et ont reçu 2, 2, 1 et 2 points, respectivement dans le final. Lorsqu'appliqué à tous les cas suspects de tuberculose, en utilisant un seuil de 2, le score avait une forte valeur prédictive négative (92%, IC 95% 87-95). Parmi les suspects de TB à frottis négatifs, le score a correctement exclu la présence de maladie active dans 214 des 229 patients (VPN 93, 95% CI 89-96). Conclusions: Les systèmes actuels de notation radiographiques pour le diagnostic de TBP semblent être sensibles, mais manquent de spécificité. Le système de notation 6

7 dérivée de la CRSS est un outil simple et fiable qui peut être utile pour exclure la TBP active chez les patients à frottis négatif. Des études de validation sont nécessaires pour confirmer ces premiers résultats. 7

8 ACKNOWLEDGEMENTS Sincere gratitude and heartfelt thanks go to my supervisor, Prof. Madhukar Pai, whose guidance and mentoring throughout the course of my graduate studies has been invaluable. Madhu leads by example, and his dedication to the cause of tuberculosis (TB) control, his passion for teaching Epidemiology and his zeal to use the highest quality research to answer questions that will hopefully help us overcome TB eventually, has been truly inspirational. I am grateful to Karen Steingart for being my guide and second reader for the systematic review. Karen embodies the zest for tireless perfection, and I am immensely grateful for her enthusiasm, patience and time spent in analyzing a huge amount of data. I thank Keertan Dheda, Dick Menzies, Kevin Schwartzman and Rodney Dawson for their insightful help and critiques throughout the course of the project. They have been most helpful in assisting me with the conduct and the analysis of the studies, and their comments and suggestions have helped me at every step of the way. I thank the team at the University of Cape Town who helped me and made my stay at Cape Town productive and enjoyable. I thank all my colleagues at the Pai TB group, and at the Respiratory Epidemiology and Clinical Research Unit (RECRU). Their constant support and encouragement has been priceless. Special thanks go to my wife Franzina, who has always been my best friend and a constant source of inspiration, and to my parents and family who have been pillars of support and guidance all my life. 8

9 PREFACE - CONTRIBUTIONS OF CO-AUTHORS For the systematic review, Lancelot M. Pinto (LP) was the lead reviewer and first author, while Madhukar Pai (MP) and Karen R. Steingart (KRS) contributed to the conception and design. LP was the lead reader, and KRS was the second reader for the screening of citations, full-text review, and data extraction. LP prepared the manuscript, and MP, Keertan Dheda (KD) and KRS provided editorial and methodological advice. For the study in South Africa, LP was the lead author, while KD and MP contributed to the conception and design. LP, MP and KD provided advice for the analysis and interpretation of results. LP prepared the manuscript, and MP and KD contributed in providing editorial advice. 9

10 CHAPTER 1: INTRODUCTION Tuberculosis (TB), caused by Mycobacterium tuberculosis, is a disease that burdens individuals and health systems globally. In 2010, there were 8.8 million new cases of TB, 12 million prevalent cases, and 1.45 million deaths, 0.35 million of which were among persons living with HIV (PLWH) (1). The South-east Asian, African and Western Pacific regions bore 85% of the global burden of disease, while Africa alone accounted for 80% of HIV-associated incident cases worldwide. The global case detection rate for all forms of TB is only 63-68%(1), and this low rate is a serious impediment to the control of the disease. Limitations of existing diagnostic tests are considered to contribute to the low-case detection rate(1). Sputum smear microscopy and chest radiography are two of the most commonly used tests for TB in most high TB burden countries. Smear microscopy has low sensitivity and fails to detect nearly half of all TB cases(2). Chest radiography is a rapid point-of-care test that has been used for over a century to diagnose pulmonary TB (PTB) (3). The test is easily performed and incorporated in screening and diagnostic algorithms, and can be especially valuable for the diagnosis of disease in patients suspected of having the disease (TB suspects) in whom sputum smears are negative for acid-fast bacilli (AFB) i.e. smear-negative TB. While chest radiography is acknowledged to be a sensitive tool for detecting pulmonary abnormalities, its use for the diagnosis of PTB has been limited by modest specificity and high inter- and intra-observer differences in reporting of radiographs(4). 10

11 Scoring systems for chest radiographs have been used successfully to standardize and improve the accuracy of detection of various pulmonary disorders. The International Labor Organization (ILO) employs a classification scheme that trains readers to describe chest radiographs in a standardized manner with regards to the location, nature, size and profusion of abnormalities when compared to standard films, which represent the various types of abnormalities. The system is specifically oriented towards documenting and identifying features associated with the different occupation-related lung diseases such as coal workers pneumoconiosis, silicosis, and asbestosis (5). The score has been found to improve reproducibility of the reading of chest radiographs for both pulmonary(6) and pleural abnormalities(7). The Chrispin-Norman score is a standardized scoring system used for grading chest radiographs in children with cystic fibrosis, with the aim of documenting, and objectively assessing progression of disease with serial radiographs(8). The score has been found to have good inter-reader reproducibility, and correlates well with lung function(9). The lung injury score is a scoring system that divides the visualized lung fields into zones, and assesses the extent of lung injury as a marker of used to assess severity of acute respiratory distress syndrome (ARDS), and has been found to be useful in improving agreement among readers, especially when read by radiologists(10). A similar standardized scoring system for TB that assigns weights to specific features of chest radiographs consistent with TB, if accurate and reproducible, could potentially augment TB case detection rates using largely pre-existing resources. Such a scoring system would ideally need to have a documentation of specific features visualized on a 11

12 chest radiograph, and a weighted score for the various features visualized relative to their association with active PTB. In this manuscript-based thesis, I attempt to explore whether a standardized scoring system for chest radiographs could improve the performance characteristics of chest radiography as a diagnostic test for pulmonary TB (PTB), both as a means for improving the inter-reader reproducibility, and for improving overall diagnostic accuracy. The first manuscript is a systematic review of the available evidence for the use of radiographic scoring systems for the diagnosis of PTB. This is a comprehensive literature review that involved searching multiple databases with the assistance of a second reader, with both readers having independently searched the literature for relevant studies that involved radiographic scoring systems for TB. The aim of the literature review was to identify such studies, and to try and assimilate a set of features that were consistently associated with PTB, with the aim of testing these features (decided a priori) for the derivation of a scoring system using the Chest Radiograph Reading and Recording System (CRRS). The CRRS is a tool, which was designed for use in epidemiological surveys for TB(11). The tool involves a checklist of features visualized on the chest radiograph, and has been found to be associated with high intra- and inter-reader reproducibility, making it a useful tool for use in the derivation of a weighted scoring system. The derivation of a scoring system for the diagnosis of PTB was attempted among subjects suspected of having TB 12

13 in a study conducted at the University of Cape Town, South Africa, using the CRRS. This study involved the reading of chest radiographs from 473 subjects suspected of having the disease, by two independent readers, and the analysis of various features visualized on the radiographs for their association with PTB, followed by deriving a weighted scoring system using these features. This study is described in the second manuscript of the thesis. Together, both manuscripts add to the evidence-base on use of scoring systems to improve the accuracy and reliability of chest radiography for pulmonary TB diagnosis, and will help improve the field of TB diagnostics within the existing framework of limited resources. 13

14 CHAPTER 2: SYSTEMATIC REVIEW OF THE LITERATURE (MANUSCRIPT 1) Chest radiograph scoring systems for the diagnosis of active tuberculosis in adults: A systematic review 2.1 ABSTRACT Rationale: The use of chest radiography as a diagnostic test for active pulmonary tuberculosis (PTB) is limited by the lack of standardization in the reading of chest radiographs. Thus, despite being sensitive for the detection of pulmonary abnormalities, it lacks both, specificity for PTB, and reproducibility. Scoring systems have been employed successfully for improving the performance characteristics of chest radiography for various pulmonary diseases, and could potentially improve the objectivity, accuracy and reproducibility of radiography for the diagnosis of PTB. Objectives: To systematically review the literature to assess the diagnostic accuracy of chest radiograph scoring systems for PTB in patients clinically suspected of having the disease. A secondary objective was to assess the reproducibility of such systems for PTB. Methods: We searched multiple databases for studies that evaluated the diagnostic accuracy and reproducibility of chest radiograph scoring systems for PTB. We summarized results for individual features visualized on a chest radiograph that were predictive of PTB, and the various features that were used in scoring systems to assess the likelihood of the disease. 14

15 Results: We identified 12 studies that described clinical-radiographic scoring systems, 11 of which were created with the aim of predicting the likelihood of PTB among patients who were to be admitted to hospitals. Six of these were tested in smear-negative patients, and, no scoring system involved the exclusive use of radiographic features. Upper lobe infiltrates and cavities were the radiographic features most commonly associated with TB disease. The sensitivity estimates of the scoring systems were uniformly high, but all of them lacked specificity. Studies involving newer techniques such as computer-assisted diagnosis (CAD) had to be excluded due to methodological inadequacies. Conclusions: The systematic review identified clinical-radiographic scoring systems, most of which are useful in ruling-out PTB as part of the assessment of the need for respiratory isolation of patients in healthcare settings. However, the low specificity precludes their use as rule-in tests for PTB. Scoring systems that rely exclusively on chest radiographs for the diagnosis of PTB are lacking. There is a need to derive accurate scoring systems for PLWH and patients evaluated in out-patients settings, especially in low-resource settings. 15

16 2.2 INTRODUCTION In 2010, there were 8.8 million new cases of TB, 12 million prevalent cases, and 1.45 million deaths, 0.35 million of which were among persons living with HIV (PLWH)(1). The global case detection rate for the disease is low, despite recognition of the need for early diagnosis as a key element in the efforts to curb the epidemic (1). The existing diagnostic tests in a majority of low-resource settings are sputum smear microscopy and chest radiography. Smear microscopy lacks sensitivity, and has a limited role in the diagnosis of extrapulmonary TB, pediatric TB, and TB in HIV-infected patients. Chest radiography, used for over a century, is a rapid point-of-care test that is known to be sensitive for detecting pulmonary abnormalities (3). The ease of use, relative low cost, and quick turnaround time make it a convenient test in high-burden, low-inclome settings. However, its use for the diagnosis of PTB has been limited by a lack of specificity, and a lack of reproducibility in reporting of radiographs(4). Consequently, the probability of diagnosing active PTB based on a chest radiograph reading is dependent on the reader and not well standardized. An analogous impediment to the use of chest radiography for the diagnosis of occupational lung diseases was overcome by the development of standardized methods for the reading of chest radiographs, a system that is now employed successfully by the International Union Against Cancer, the International Labor Organization, and the National Institute for Occupational Safety and Health(5, 12). Scoring systems have also been developed for grading the severity and extent of pulmonary disease among patients 16

17 with cystic fibrosis(8) and form part of the lung injury score for assessing the severity of adult respiratory distress syndrome(10, 13). Similarly, a standardized scoring system for TB that assigns weights to specific features of chest radiographs consistent with TB, if accurate and reproducible, could potentially augment TB case detection rates using largely pre-existing resources. Such a standardized scoring system for TB also has the potential to be combined with newer nucleic acid amplification tests such as Xpert MTB/RIF (Cepheid Inc, Sunnyvale, CA) (14), either as a triage test to reduce costs or as an add-on test in Xpert MTB/RIF negative persons. To our knowledge, no previous systematic reviews have assessed the performance of radiograph scoring systems for pulmonary TB. Therefore, we carried out a systematic review to estimate the diagnostic accuracy of chest radiograph scoring systems for TB in patients suspected of having the disease. A secondary objective was to assess the reproducibility of chest radiograph scoring systems for active PTB. 2.3 METHODS We followed guidelines for systematic reviews of diagnostic test accuracy recommended by the Cochrane Collaboration Diagnostic Test Accuracy Working Group, including writing a detailed protocol before starting the review(15, 16). Types of studies: We included randomized controlled trials and observational studies of all study designs (e.g. cross-sectional, case-control and cohort) that assessed the performance of radiographic scoring systems for the diagnosis of pulmonary TB. 17

18 Participants: Participants were subjects suspected of having pulmonary TB who were 15 years of age and older. We restricted studies to those that included a minimum of 10 cases of active TB. With the aim of evaluating subjects similar to those who present in routine clinical practice, we excluded studies that exclusively studied specific patient groups such as patients with pneumoconioses, malignancies (both hematological and solid organ), immune-mediated inflammatory disease, including patients on immunosuppressive medications such as tumor necrosis factor-alpha inhibitors, and patients on hemodialysis. Studies that were conducted on individuals who were not suspected of having TB, such as investigations of asymptomatic contacts of patients with active TB were also excluded. Index test: Any chest radiograph scoring system Comparator: No chest radiograph scoring system Target condition: TB of the pulmonary parenchyma, pleura, intrathoracic lymph nodes. We included miliary TB if the disease involved either pulmonary parenchyma or multiple sites, one of which was the lung. Reference standards: We considered liquid or solid cultures as the reference standards for active pulmonary TB. 18

19 Definitions: A radiograph scoring system was defined as a system that assigned numeric weights to specific features of chest radiographs consistent with PTB (such as cavitary lesions), with or without the presence of clinical or lab components in the system. True positives (TP) were TB suspects correctly classified as PTB by the scoring system when compared with the reference standard. False positives (FP) were TB suspects who did not have active PTB but were misclassified by the scoring system as having active PTB. False negatives (FN) were subjects with active PTB who were misclassified by the scoring system as not having the disease. True negatives (TN) were TB suspects who did not have active PTB and were correctly classified by the scoring system. Sensitivity refers to the proportion of patients with active PTB correctly identified by the index test when compared with the reference standard: [TP/(FN + TP)] *100 Specificity refers to the proportion of patients with active PTB correctly identified by the index test when compared with the reference standard: [(TN/(FP + TN)] *100 Positive predictive value (PPV) refers to the proportion of patients correctly identified as having active PTB by the scoring system when compared to all patients identified as having active PTB by the scoring system: [TP/(TP+FP)] Negative predictive value (NPV) refers to the proportion of patients correctly identified as not having active PTB by the scoring system when compared to all patients identified as not having active PTB by the scoring system: [TN/(TN+FN)] 19

20 Diagnostic Odds ratio (DOR) refers to the odds of a participant with active PTB having a specific clinical or radiographic manifestation as compared to the odds of a participant without active PTB having the same clinical or radiographic manifestation. It is computed by the formula: [(TP*TN)/(FP*FN)] Reproducibility refers to percent agreement on reported features when a chest radiograph is read more than once. The agreement could either be intra-reader, when the same chest radiograph is read more than once by the same reader, blinded to his/her previous reporting of the radiographs, or inter-reader, when more than one reader reports the features of the same chest radiograph. This agreement is a reflection of the repeatability of the test, and is independent of the accuracy with reference to the reference standard for the diagnosis of active PTB. The observed level of agreement is the ratio of the number of readings that are in agreement to the total number of readings. It is expressed as a percentage: Agreement = [Number of readings in agreement/total number of readings]*100. Cohen s kappa (Κ) is the chance-adjusted measure of agreement defined as the ratio of the actual agreement beyond chance to the potential agreement beyond chance. Search strategy and study selection We searched Medline ( ), Embase ( ) and Web of Science ( ) on 28 July 2011 for relevant articles, using published hedges for diagnostic tests to improve sensitivity (17, 18). We used the terms sensitiv*[tw] OR diagnos*[tw] OR di [fs] AND radiograph*[mesh] OR chest xray[tw] OR mass chest x-ray[mesh] OR photofluorograph*[tw] OR scor*[tw] AND tuberculosis (sub-headings : lymphnode / 20

21 miliary/ multidrug-resistant/ Pleural/ Pulmonary) [MeSH] OR Mycobacterium tuberculosis [MeSH]. The detailed search strategy for Medline can be found in Appendix 6.1. We also reviewed reference lists of included articles and any relevant review articles identified through the search, for possible eligible articles, and hand-searched relevant World Health Organization reports. Relevant studies, restricted to those published in English, French and Spanish, were selected independently by two reviewers (LP and KRS) and disagreements were resolved by consensus. Citations deemed appropriate by either reviewer after screening titles and abstracts were selected for full-text review. A list of excluded studies with their reasons for exclusion is available upon request from the authors. Assessment of study quality Two reviewers (LP and KRS) independently assessed study quality using the core set of 11 items from Quality Assessment of Diagnostic Accuracy Studies (QUADAS), a validated tool to evaluate the presence of bias and variation in diagnostic accuracy studies(19). As recommended, each item was be scored as yes, no, or unclear. Data extraction Data were extracted from each study, using a data extraction form, that was piloted and then finalized, based on the experience gained from the pilot data extraction process. Two reviewers (LP and KRS) independently extracted data, and disagreements were resolved by consensus. The following data were extracted: author, study design, manner of patient 21

22 selection, country income status, eligibility criteria for participants, demographic details of participants, details on the number and qualifications of the readers of the chest radiographs, and TP, FP, FN, and TN for both, the individual features visualized on the chest radiograph (such as infiltrates and cavities), and for the scoring system. The data extraction form is included in Appendix 5.2. Statistical analysis For the studies that provided TP, FP, FN, and TN values, sensitivity and specificity estimates and their corresponding 95% confidence intervals were calculated for the scoring system at the cut-off for the diagnosis of active PTB used by the study authors (mostly based on optimal sensitivities and specificities using receiver operator curves). Forest plots were generated to display sensitivity and specificity estimates using Meta- Disc (version 1.4) (20). Odds ratios for the presence of individual radiographic features for the diagnosis of active PTB were determined when the study provided the relevant data. Meta-analyses of odds ratios for specific radiographic features were performed only if the features were defined in a sufficiently homogenous manner across studies, the populations were similar, and the odds ratios were considered homogenous (Heterogeneity was assessed using the I-squared statistic; effects were pooled if the I 2 75%. When results were pooled, meta-analysis was performed using the DerSimonian and Laird random effects approach, with the aim of incorporating the heterogeneity of effects across studies(21). All analyses were performed using STATA 11 (Stata Corporation, College Station, Texas, USA). Pooling was performed using the command metandi (22).Formal assessment of publication bias using methods such as funnel plots 22

23 or regression tests were not performed because such techniques have not been found to be useful for diagnostic data(23). An estimation of language bias was attempted by retrieving citations from the search strategy with and without a language filter, and we report the filtered citations as a percentage of the overall citations retrieved. 2.4 RESULTS We identified citations, of which 8066 unique articles were identified after exclusion of duplicate articles. We conducted the search with and without the language filters to assess the degree of bias, and found that our search strategy included 80.5% of all studies. After screening of titles and abstracts, 168 articles satisfied the eligibility criteria for further review and their full-texts were retrieved. After full-text review, 156 articles were excluded for various reasons. Thus, 12 articles were included in the systematic review (24-35). Details of the selection process are outlined in Fig.1 in accordance with PRISMA guidelines for reporting of systematic reviews(36). Included studies We did not identify any scoring system that was based exclusively on radiographic criteria. The 12 included studies all involved scoring systems that comprised both clinical and radiographic features.. Table 1 lists the characteristics of the 12 included studies, containing a total of 5767 participants. The median number of TB suspects included in the studies was 283 (interquartile range 177 to 431). Six studies included all patients suspected of having TB (24-29), five studies included patients suspected of having TB who were found to have negative sputum smears (30-23

24 34), and one study excluded patients with HIV/AIDS(35). Nine studies were performed in high-income countries. Five studies involved radiologists; two studies included pulmonologists and five studies did not report the background of the radiograph reader. The demographic characteristics of the patients are shown in Table 2. When reported, the majority of patients were male. Eleven studies included PLWH, who represented 11% to 61% of eligible patients. Excluded studies We identified two studies that were designed with the aim of deriving a clinicalradiographic scoring system for PLWH among hospitalized patients found to be sputum smear-negative (37, 38). Both studies satisfied a majority of our inclusion criteria and found the presence of mediastinal adenopathy and cavities to be significantly associated with PTB in univariate analysis. However, neither of the studies derived a score for PTB. The study by Le Minor et al. concluded that the numbers were insufficient to develop a score for TB (38), while the study by Davis et al. stated that After exhaustive testing, we were unable to identify any combination of factors which reliably predicted bacteriologically confirmed tuberculosis (37). We also excluded 13 studies that used automated computer-assisted diagnosis (CAD) as none of these studies used culture as a reference standard, a criterion for inclusion in this review (39-51). Five studies that involved the grading of chest radiographs were excluded as these studies were designed to grade the severity of PTB based on the extent of abnormalities visualized on the chest radiograph, and not the diagnostic accuracy of 24

25 scoring systems (52-56). We also excluded three studies that used the Chest Radiograph Reading and Recording System (CRRS) (11, 57, 58), despite these studies demonstrating the CRRS tool to have good reliability for features of PTB visualized on a chest radiograph, as these studies did not use culture as a reference standard. Assessment of methodological quality As seen in Table 3, all studies suffered from incorporation bias, as the results of the chest radiograph and/or the clinical components of the scoring system played a role in the decision of those patients would who be investigated further with culture, the reference standard. Six (50%) of the total 12 studies also did not include a sample that was considered representative of the target population. Six (50%) studies were unclear about whether the person assigning scores to the patients for the various components of the scoring system was blinded to the results of the reference standard. Findings Studies that included all patients who were suspected of having TB We identified six studies that included all patients suspected of having TB (24-29). All studies were performed in an inpatient setting. All studies were aimed at deriving optimal prediction scores to identify patients who were likely to have PTB and require respiratory isolation (Tables 1 and 2). In univariate analyses, the most common radiographic features across studies found to be significantly associated with PTB were upper lobe infiltrates [odds ratio (OR) range, 2.38 to 10.11, pooled OR using a random effects model 6.65, 95% CI 4.42, 10.01, five studies)] (Table 4 and Figure 2), and cavities (OR range, 2.11 to 25

26 10.08, estimates not pooled due to heterogeneity of effects, three studies) (Table 5 and Figure 3). The details of the parameters included in the scores and their respective weights are summarized in Table 6, along with the performance characteristics of the scoring system and the final rule to aid in decision-making. Studies used several different methods to derive weights for the scoring system: logistic regression of the parameters found significant by univariate analysis (three studies); classification and regression tree (CART) analysis (one study) (25); general regression neural network (GRRN) analysis (one study) (26); and chi-squared recursive partitioning (one study) (27). All six studies achieved a sensitivity of the scoring system greater than 80% (median 95%, range, 81% to 100%). For the five studies that reported specificity data, specificity estimates were low (median 42%, range 22% to 72%), suggesting a poor rule-in value for PTB. Figure 4 presents individual study results of sensitivity and specificity estimates (and their 95% confidence intervals) in both forest plots for the four studies that provided sufficient data. As the scoring systems included varying clinical and radiographic parameters to calculate the likelihood of PTB, we did not consider these systems to be homogenous, and therefore, did not perform a meta-analysis of the accuracy estimates of these scoring systems. Studies that only included patients suspected of having TB who were found to be sputum smear negative 26

27 We identified five studies in this category (30-34). Four studies were conducted in an inpatient setting for the purpose of determining a clinical rule for respiratory isolation (30, 31, 33, 34), while one study was performed in an outpatient setting (32), Tables 1 and Table 2. As with the previously described set of studies that included all patients suspected of having TB, in the univariate analysis, the most common radiographic features across studies found to be associated with PTB were upper lobe infiltrates (OR range, 2.47 to 9.07, pooled OR using a random effects model 3.57, 95% CI 2.38, 5.37, five studies) (Table 4 and Figure 5), and cavities (OR range, 1.97 to 25.66, estimates not pooled due to heterogeneity of effects, three studies) (Table 5 and Figure 6). To derive weights for the scoring system, two studies used logistic regression of the parameters found significant in the univariate analysis, while three studies involved validation of previous studies. One of the validation studies used bootstrapping, which is a resampling method aimed at improving the internal validity of the data. (31) All studies achieved a sensitivity of the scoring system greater than 93% (median 96%, range, 93% to 98%). However, specificity estimates were low (median 35%, range 14% to 50%), again suggesting a poor rule-in value for PTB. Figure 7 presents individual study results of sensitivity and specificity estimates (and their 95% confidence intervals) in both forest plots for the four studies that provided sufficient data. As is the case with the above studies, we did not consider these systems to be homogenous, and therefore, did not perform a meta-analysis of the accuracy estimates of these scoring systems. 27

28 One study, performed in an inpatient setting, excluded PLWH(35), Tables 2 and 3. Logistic regression was used to derive weights for the score, but the study also validated the score derived by Wisnivesky et al.(34) This study found a sensitivity of 97% and specificity of 42%. The details of the scoring system and its performance characteristics can be found in table 5. Reproducibility None of the included studies reported data on intra-reporter or inter-reporter reproducibility. 2.5 DISCUSSION Chest radiography is an important tool for physicians to assess the probability of active pulmonary TB among individuals who have symptoms suggestive of disease, and often the only tool available for assessing this probability among those suspected of having the disease who are found to be negative on sputum smear examination. In the absence of newer tests for TB that are universally affordable and accessible, there is a need to improve existing tests such as chest radiography, which suffers from a lack of standardization. We conducted this systematic review with the aim of assessing the diagnostic accuracy of standardized radiographic scoring systems for the diagnosis of PTB, and whether standardization improves the performance of the test. Our review failed to find any study 28

29 that exclusively relied on radiographic features to derive a score, and all the included studies had a combination of defined radiographic criteria with different clinical criteria. Most of the included studies were hospital-based, decision-to-isolate studies. Patients with PTB patient can generate up to 44 quanta per hour (one quantum is defined as the infectious dose)(59), highlighting the necessity for rapid respiratory isolation of patients with PTB in the hospital setting. Yet, the unnecessary respiratory isolation of patients considerably increases costs to the healthcare system(60). Scoring systems that improve the accuracy of the decisions to subject those patients suspected of having PTB to respiratory isolation can considerably improve the efficiency of healthcare systems and utilization of resources. The scores developed suffered from low specificity, and had a high rule-out value (high negative predictive value) but a poor rule-in value (low positive predictive value) for PTB. However, such scores may still be useful for limiting the number of patients for whom further investigations would be warranted, especially among patients who are smear-negative. The prediction rule developed by Wisnivesky et al was validated in three studies, two of which were conducted in patients who had negative sputum smears. The scoring system consistently demonstrated sensitivity higher than 95%, but had poor specificity. As a rule-out test, this scoring system appears to be validated in multiple studies. The study by Soto et al(32) was a validation study of a score derived by the same research group in an earlier study (31). Although the cut-off for the score was modified in the validation cohort, it performed well in a sub-group of patients with no prior history of TB. However, 29

30 we limited the analysis of the performance characteristics of this scoring system in the population of all patients, as this was the intent of the validation study, and not the posthoc analysis of the performance in the selected subgroup. We identified only one study that assessed clinical-radiographic scoring systems for use in the out-patient setting. Our systematic review also failed to identify a clinical radiographic scoring system that was derived for PLWH suspected of having active PTB. Bock et al.(24) performed a sub-group analysis in PLWH, but found no radiographic feature to be significantly associated with active PTB in this sub-group, a finding that is consistent with the atypical nature of radiographic manifestations of PTB described among PLWH (61). Automated computer-assisted diagnosis (CAD) employs techniques such as texture analysis for reading chest radiographs, and appears to be a promising modality for standardizing and improving the diagnostic performance of digital chest radiography (62). However, our review suggested a lack of methodologically high-quality studies. Further development of the field should focus on the validation of such techniques in larger populations and with a structured epidemiological approach using appropriate reference standards. The strength of our systematic review is in the extensive review of the literature, with two reviewers independently performing the review, and basing every decision on discussion and consensus. We restricted our search to articles written in English, French and Spanish, but an assessment for language bias suggested that we included a high 30

31 proportion of the available literature. We conducted citation searches of the included articles and review articles to identify any published study that we may have failed to include because of the language restriction, but did not identify any such studies. However, we may have inadvertently failed to include articles of relevance in other languages, and acknowledge this as a shortcoming of the review. All the included studies suffered from incorporation bias, as the results of the chest radiograph and/or the clinical components of the scoring system played a role in the decision of which patients would be investigated further with TB culture. This may have over-estimated the accuracy of the scoring systems in relation to culture. Six of the 12 studies also did not include a sample that was considered representative of the target population (selection bias). Six of the 12 studies were unclear about whether the person assigning scores to the subjects for the various components of the scoring system was blinded to the results of the reference test. Selection bias and absence of blinding are features of study design that have been associated with inflated accuracy estimates(63, 64). These limitations in the quality of the included studies need to taken into consideration when drawing conclusions. 2.6 CONCLUSIONS Our systematic review revealed no scoring system designed to assess the likelihood of active PTB based exclusively on radiographic features. Measures to create such a system would help standardize the interpretation of chest radiographs for the diagnosis of active PTB. The systematic review identified clinical-radiographic scoring systems, most of which were created with the aim of predicting the likelihood of active PTB among patients who were to be admitted to hospitals. Such scoring systems are intended for assessing the need for respiratory isolation of patients in healthcare settings. Although 31

32 most of these systems have high sensitivity, they have low specificity for active PTB. There is a need to derive accurate scoring systems for PLWH and patients evaluated in out-patients settings, especially in low-resource settings. Technological advances in the interpretation of chest radiographs, such as CAD, need to be refined and validated in well-designed studies to assess their utility. 32

33 2.7 TABLES AND FIGURES Table 1. Characteristics of studies included Study Country Setting Studies that included all TB suspects No. of eligible TB suspects included( % of eligible participan ts) Bock et al (1996) 15 USA In-patient 295 (78) El-Solh et al (1997) 16 USA In-patient 286 (100)* Design Cross-sectional, retrospective Cross-sectional, retrospective El-Solh et al (1999) 17 USA In-patient 119 (100)* Cross-sectional Moran et al (2009) 18 USA In-patient 2535 (91)* Cross-sectional Mylotte et al (1997) 19 USA In-patient 220 (100)* Cross-sectional, retrospective Inclusion criteria 1.Patients with active TB 2.Patients with TB in the differential diagnosis 3.AFB smears and cultures ordered 4.HIV + with abnormal CXR All isolated patients, based on symptoms, prior history of TB exposure, HIV status, medical and social risk factors, and radiographic findings All patients in whom and AFB smear and culture was requested Admission diagnosis of pneumonia or suspected TB All patients in whom and AFB smear and culture was requested by the admitting physician Chest radiograph reader(s) Radiologist Radiologist and pulmonologist Radiologist and pulmonologist Emergency medicine resident Not reported Reference standard type of culture Solid and liquid Liquid Liquid NR Liquid 33

34 Solari et al (2008) 20 Peru Emergency Department 345 (70.8) Cross-sectional Studies that included smear-negative TB suspects Lagrange- Xelot et al (2010) 21 France In-patient 134 (100) Cross-sectional Soto et al (2008) 22 Peru In-patient 262 (100) Cross-sectional Soto et al (2011) 23 Peru Out-patient 663 (96.9) Cross-sectional Wisnivesky et al USA In-patient 112 (100) Case-control (2000) 25 Productive cough for > 1 week or Cough of any duration and 1.Fever > 3 weeks or 2.Weight loss of at least 3kg in previous month or 3.Night sweats or hemoptysis or differential diagnosis of PTB from attending physician Suspected TB, as recommended by French guidelines Cough > = I week AND one or more of the following: 1.Fever 2.Weight loss >= 4kg in 1 month 3.Breathlessness 4.Constitutional symptoms (malaise or hyporexia for a minimum of 2 months) Cough > = 2 weeks AND one or more of the following: 1.Fever 2.Weight loss 3.Breathlessness Cases - isolated TB patients controls - randomly selected from a log of patients who submitted smears and cultures matched on age (+/- 3 years), sex and year of presentation, 3 smears negative, culture negative and isolated in a hospital Internist, internal medicine resident Not reported Not reported 1.General practitioner 2.TB specialist Tie breaker: Experienced radiologist 1.Radiologist 2.Radiologist Solid Liquid Solid Solid, liquid or concentrated smear Solid and liquid 34

35 Wisnivesky et al USA In-patient 516 (100) Cross-sectional (2005) 24 Study that included only HIV-uninfected patients Rakoczy et al (2008) 26 USA In-patient 280 (100)* Case-control Patients admitted and isolated because of suspicion of PTB Cases- all TB inpatients controls - all inpatients placed under airborne precautions with negative smears and cultures matched with cases on time of admission (+/- 6 days) *Studies had derivation and validation cohorts. The number of TB suspects represents those in the validation cohorts Not reported Not reported Solid and liquid Not reported 35

36 Table 2. Demographic characteristics of subjects in the included studies Study Age (years) No. of Males (%) No. of Persons Living with HIV (%) Patients with Active TB (%) Studies that included all TB suspects Bock et al (1996) 15+ mean (79) 230 (61.0) 53 (14.1) 16 ## El-Solh et al (1997) mean(sd) PLWH: 36.6(0.4) non-plwh: 50.4(1.2) NR 316 (56.1) 47 (8.3) El-Solh et al (1999) 17 NR NR 66 (55.5) 11 (9.2) 18 ## median (IQR) 3567 (63)* 1058 (20.8) 224 (4.4) Moran et al (2009) 48(38-63) Mylotte et al (1997) 19 mean(sd) NR 129 (59.0) 8 (3.6) 44(16) Solari et al (2008) 20 median (64.4) 45(13.0) 109 (31.6) Studies that included smear-negative TB suspects mean (SD) Lagrange-Xelot et al (2010) (14.0) 94 (70) 60 (40.0) 26 (19.0) Soto et al (2008) 22 NR 166 (63.4) 28(10.9)** 27 (10.3) Soto et al (2011) (17.2) 370 (55.8) 98 (24.0) # 184 (27.8) Wisnivesky et al (2000) 25 (2) cases 40 controls 40 (2) 82 (73.2) NR 56 (50) Wisnivesky et al (2005) (11.9) 285 (55.2) 362(70.0) 19 (3.7) Study that excluded PLWH cases - Rakoczy et al (2008) 26 cases 60 33(67) controls 51.8 controls 29 (59) 0 33 (11.8) IQR, interquartile range; NR, not reported; PLWH, persons living with HIV * Sex not documented in 1.4% patients ** 6 patients refused testing # 255 patients refused testing ## The demographic characteristics represent those of the included subjects in the combined derivation and validation cohort + The demographic characteristics represent those of the eligible subjects 36

37 Bock et al (1996) 15 El-Solh et al (1997) 16 El-Solh et al (1999) 17 Moran et al (2009) 18 Mylotte et al (1997) 19 Solari et al (2008) 20 Lagrange- Xelot et al 21 (2010) Soto et al (2008) 22 Soto et al (2011) 23 Wisnivesk y et al (2000) 25 Wisnivesk y et al (2005) 24 Rakoczy et al (2008) 26 Table 3. Quality assessment of the included studies using the QUADAS tool Item Representative sample? Acceptable reference standard? Acceptable delay? Partial verification avoided? Differential verification avoided? Incorporation avoided? Reference standard blinded? Index results blinded? Relevant clinical information available? Uninterpretable results reported? Withdrawals explained? Yes No Unclear 37