Pooled analysis of the accuracy of five cervical cancer screening tests assessed in eleven studies in Africa and India

Transcription

1 Int. J. Cancer: 123, (2008) ' 2008 Wiley-Liss, Inc. Pooled analysis of the accuracy of five cervical cancer screening tests assessed in eleven studies in Africa and India Marc Arbyn 1 *, Rengaswamy Sankaranarayanan 2, Richard Muwonge 2, Namory Keita 3, Amadou Dolo 4, Charles Gombe Mbalawa 5, Hassan Nouhou 6, Boblewende Sakande 7, Ramani Wesley 8, Thara Somanathan 8, Anjali Sharma 9, Surendra Shastri 10 and Parthasarathy Basu 11 1 Scientific Institute of Public Health, Unit of Cancer Epidemiology, Brussels, Belgium 2 Screening Group, International Agency for Research on Cancer, Lyon, France 3 Service de Gynecologie/Obstetrique, Universite de Conakry, Guinee-Conakry 4 Departement de Gynecologie-Obstetrique, Hôpital Gabriel Toure, Bamako, Mali 5 Departement de Gynecologie, Universite Marien Ngouabi, Brazzaville, Congo 6 Faculte des Sciences de la Sante, Universite de Niamey, Niamey, Niger 7 Laboratoire de Biologie Medicale et d Histo-Cyto-Pathologie, Clinique Philadelphie, Ouagadougo, Burkina Faso 8 Regional Cancer Center, Medical College Campus, Trivandrum, Kerala State, India 9 Bhagwan Mahaveer Cancer Hospital & Research Centre (BMCHRC), Jaipur, Rajasthan, India 10 Preventive Oncology, Tata Memorial Hospital & Cancer Research Institute, Mumbai, India 11 Gynaecological Oncology, Chittaranjan National Cancer Institute, Calcutta, India Cervical cancer is the main cancer among women in sub-saharan Africa, India and other parts of the developing world. Evaluation of screening performance of effective, feasible and affordable early detection and management methods is a public health priority. Five screening methods, naked eye visual inspection of the cervix uteri after application of diluted acetic acid (VIA), or Lugol s iodine (VILI) or with a magnifying device (VIAM), the Pap smear and human papillomavirus testing with the high-risk probe of the Hybrid Capture-2 assay (HC2), were evaluated in 11 studies in India and Africa. More than 58,000 women, aged years, were tested with 2 5 screening tests and outcome verification was done on all women independent of the screen test results. The outcome was presence or absence of cervical intraepithelial neoplasia (CIN) of different degrees or invasive cervical cancer. Verification was based on colposcopy and histological interpretation of colposcopy-directed biopsies. Negative colposcopy was accepted as a truly negative outcome. VIA showed a sensitivity of 79% (95% CI 73 85%) and 83% (95% CI 77 89%), and a specificity of 85% (95% CI 81 89%) and 84% (95% CI 80 88%) for the outcomes CIN21 or CIN31, respectively. VILI was on average 10% more sensitive and equally specific. VIAM showed similar results as VIA. The Pap smear showed lowest sensitivity, even at the lowest cutoff of atypical squamous cells of undetermined significance (57%; 95% CI 38 76%) for CIN21 but the specificity was rather high (93%; 95% CI 89 97%). The HC2- assay showed a sensitivity for CIN21 of 62% (95% CI 56 68%) and a specificity of 94% (95% CI 92 95%). Substantial interstudy variation was observed in the accuracy of the visual screening methods. Accuracy of visual methods and cytology increased over time, whereas performance of HC2 was constant. Results of visual tests and colposcopy were highly correlated. This study was the largest ever done that evaluates the cross-sectional accuracy of screening tests for cervical cancer precursors in developing countries. The merit of the study was that all screened subjects were submitted to confirmatory investigations avoiding to verification bias. A major finding was the consistently higher sensitivity but equal specificity of VILI compared with VIA. Nevertheless, some caution is warranted in the interpretation of observed accuracy measures, since a certain degree of gold standard misclassification cannot be excluded. Because of the correlation between visual screening tests and colposcopy and a certain degree of over-diagnosis of apparent CIN21 by study pathologists, it is possible that both sensitivity and specificity of VIA and VILI were overestimated. Gold standard verification error could also explain the surprisingly low sensitivity of HC2, which contrasts with findings from other studies. ' 2008 Wiley-Liss, Inc. Key words: cervical cancer; cervical intraepithelial neoplasia; screening methods; VIA; pap smear; HPV; developing countries Cervical cancer, an eminently preventable cancer, is the second most common cancer among women worldwide, affecting an estimated 490,000 women each year, and causing 270,000 deaths annually. 1 About 85% of this global burden of cervical cancer is experienced in sub-saharan Africa, South and South-East Asia, Oceania, Central and South America and the Caribbean. Although cervical cancer risk has substantially declined among women in developed countries due to effective cervical cytology screening programs, cervical cancer continues to be the most common cause of premature death among middle-aged women in their most productive years and the largest single cause of year-life lost to cancer in developing countries. 2 4 This is mainly due to lack of screening or existing inefficient cytology screening programs in these countries. Different screening tests such as conventional cytology, visual inspection using acetic acid (VIA) or using Lugol s iodine (VILI), VIA with magnification (VIAM) and human papillomavirus (HPV) testing have been investigated in the last few years for their accuracy in different settings in developing countries by different providers as part of the research efforts to find feasible and effective options for implementing screening in developing countries. The International Agency for Research on Cancer, as part of the Alliance for Cervical Cancer Prevention supported by the Bill and Melinda Gates Foundation, conducted 11 cross-sectional studies in 4 locations in India and in 5 locations in French speaking Africa, in collaboration with the national institutions, to evaluate the accuracy of the above screening tests in detecting cervical neoplasia and to establish a platform from where screening services can be provided and health personnel can be trained in different aspects of cervical screening Results from these studies have showed varying estimates of test accuracies, due to several factors including differences in testing, training of providers, quality assurance methods, quality and consistency of reference standards used to establish true positive disease, reporting of histopathology and monitoring of studies. In this article, we present the results of a meta-analysis of these 11 independent studies. We computed the overall sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and pooled relative accuracy of the screening tests. Additionally, we explored the sources of heterogeneity by assessing the association between test accuracies and individual and study characteristics. *Correspondence to: Scientific Institute of Public Health, Unit of Cancer Epidemiology, Brussels, Belgium. marc.arbyn@iph.fgov.be Received 11 December 2007; Accepted after revision 16 January 2008 DOI /ijc Published online 10 April 2008 in Wiley InterScience ( wiley.com). Publication of the International Union Against Cancer

2 154 ARBYN ET AL. TABLE I NUMBER OF INCLUDED PARTICIPANTS AND CERVICAL CANCER SCREENING TESTS EVALUATED IN FIVE AFRICAN AND SIX INDIAN STUDY CENTRES Number Centre Country Tests evaluated Number of women included 1 Bamako Mali VIA, VILI Brazzaville Congo VIA, VILI Conakry Guinea VIA, VILI Niamey Niger VIA, VILI Ouagadougou Burkina Fasso VIA, VILI Calcutta 1 India VIA, VIAM, Pap smear, HC Calcutta 2 India VIA, VILI, VIAM, HC Jaipur India VIA, VILI, Pap smear Mumbai India VIA, VILI, VIAM, Pap smear, HC Trivandrum 1 India VIA, VILI, Pap smear Trivandrum 2 India VIA, VILI, Pap smear, HC Total 58,679 VIA, visual inspection after application of acetic acid; VILI, visual inspection after application of Lugol s iodine; VIAM, visual inspection after application of acetic acid and using a magnifying loupe; HC2, hybrid-capture 2 assay. Material and methods Study population, evaluators and tests Between 1999 and 2003, the test accuracy of 5 cervical cancer screening methods were evaluated in more than 58,000 women aged 25 to 64 from 11 urban settings in India and 5 African countries. 5 9 The assessed tests were naked eye visual inspection of the uterine cervix after application of acetic acid (VIA) 5,6 or after application of Lugol s iodine (VILI) 5,6 VIA with a low level magnifying lens (43) (VIAM), 7 the conventional Papanicolaou smear 8 and an HPV DNA detection test targeting 13 high-risk HPV types, 9 using the B-probe of the Hybrid Capture-2 assay (HC2) (Digene, Gaithersburg, MD). The number of women examined and tests assessed per centre is shown in Table I. VIA was evaluated in all 11 studies, VILI in all but Calcutta 1 study. VIAM, cytology and HC2 were only assessed in India, respectively, in 3, 5 and 4 studies. The characteristics of the participating population, involved health personal, the technical details and results of each separate screening test were described previously. 5 9 The Pap smear was reported in the following 4 categories: negative for neoplastic cellular changes, atypical squamous cells of undetermined significance, low-grade (LSIL) and high-grade intraepithelial lesion or worse (HSIL1). The other tests were categorized as negative or positive using criteria described elsewhere. 5 9,11 Briefly, positivity for VIA or VIAM was defined as presence of opaque, dense, well-defined aceto-white areas touching the squamo-columnar junction or close to the external os or presence of aceto-white growth, observed 1 min after application of 4% acetic acid solution on the cervix; whereas presence of mustard or saffron yellow lesions after application of lugol was considered as positivity criterion for VILI. 11 A signal with RLU (relative light unit) higher than one using controls that contained 1 pg/ml of HPV DNA was used as cutoff for the HC2 test. 9,12 All the screening tests were applied independently by different examiners, who were blinded toward the results of the other tests. Visual screening tests, and collection of cervical cells for cytology and HPV testing were performed by 51 female health workers with a variety of different educational qualifications: auxiliary nurse midwives (n 5 33) (Burkina Faso, Congo, Guinea, Mali, Niger), registered nurses (n 5 2) (Trivandrum), cyto-technicians (n 5 2) (Trivandrum), university graduates in science (n 5 9) (Jaipur, Calcutta) or high-school graduates (n 5 5) (Mumbai). All were trained in the performing and reporting of VIA and VILI during a 5-day intensive course, using a training manual prepared by the International Agency for Research on Cancer. 11 The training included lectures, discussions, review of photographs of normal and abnormal cervix, as well as practical experience observing examinations of volunteer subjects. Refresher courses (1 2 days) were given for all health workers during the study period. All participants were subsequently investigated with colposcopy on the same day and, if a colposcopically suspect or abnormal lesion was identified, punch biopsies were taken. Colposcopists and histologists examining biopsies were masked with respect to the screening test results. The histopathological diagnosis was used as gold standard for women from whom biopsies were taken; otherwise the colposcopic impression was used for determination of the final cervical status. This final outcome was categorized into 5 classes: normal or non-neoplastic changes, cervical intraepithelial neoplasia grade 1 (CIN1) including HPV changes, CIN2, CIN3 and invasive cancer. Patients with CIN or cancer were offered appropriate follow-up and treatment. The technicians and doctors involved in the study were trained and reoriented at the beginning of the study and retrained and assessed periodically during the course of the study. Internal and external quality control measures were introduced in the pathology laboratories. Laboratory procedures and manuals were reviewed. Statistical methods We used meta-analytical methods to pool test accuracy measures from the different study sites. For each category of CIN, sensitivity, specificity, predictive values of the 5 tests, and the ratio of the sensitivity and specificity of VIA compared with the other tests were assessed using random effect models, allowing for intersetting heterogeneity. 13,14 We also estimated the pooled absolute sensitivity and specificity using a 2-level model, with independent binomial distributions for the true positives and true negatives conditional on the sensitivity and specificity in each study, and a bivariate normal model for between-study variation of the logarithms of odds of sensitivity and specificity. 15 This method incorporates the intrinsic negative correlation between the sensitivity and specificity and it allows for sparse data. The overall accuracy measure, the diagnostic odds ratio (DOR) (odds of sensitivity/odds of (1-specificity), was estimated using Hierarchical Summary Receiver Operating Characteristic regression. 16 Models were fitted with metandi, a Generalized Linear Latent and Mixed Models-based procedure developed by Harbord. 17 The influence of age, study centre, and time period on study outcomes was assessed by SROC regression. 18 Age was aggregated by 5-year groups and study period by tertiles, using date of screening or chronological rank ID. Study period was considered as a proxy for accumulated experience of the assessors. The correlation between screen test and colposcopy results were assessed by Spearman s rank correlation coefficient. All statistical analyses were performed with STATA, version 9 (Stata Corp., College Station, TX). 13

3 POOLED ANALYSIS OF CERVICAL CANCER SCREENING TESTS IN DEVELOPING COUNTRIES 155 FIGURE 1 Forest plots of the sensitivity and specificity of visual inspection of the cervix after application of acetic acid (VIA) or Lugol s iodine (VILI) to detect CIN2 or more serious cervical disease. The diamond at the bottom of each forest plot represents the 95% confidence interval of the pooled measure computed from a random effect model. Results Test accuracy of VIA and VILI for CIN21 The forest plots in Figure 1 display the variation of the sensitivity and specificity VIA and VILI, considering CIN2 or worse disease as outcome. The sensitivity of VIA varied between 61.5% (CI %), in Calcutta 1, and 91.1% (CI %) in Conakry. The pooled sensitivity was 79.2% (CI %). The lowest specificity of VIA (74.2%; CI %) was observed in Ouagadougou, and the highest specificity (94.5%; CI %) was found in Niamey. High specificity was also observed in Conakry: 93.8% (CI %). The pooled specificity of VIA was 84.7% (CI %). In general, the sensitivity of VILI was higher than VIA at the exception of Jaipur (87.5%; CI %) and Trivandrum 2 (80.2%; CI %) where sensitivities were equal to those of VIA. The specificity values of VILI varied over a similar range as VIA, between 73.0 and 91.6%. The overall pooled sensitivity for VILI (91.2%; CI %) was statistically significantly higher than for VIA. On the other hand, the pooled specificity of VILI (84.5%; CI %) was not significantly different from that of VIA. Table II shows, besides sensitivity and specificity, also the predictive values, the prevalence of CIN21 and the detection rates of CIN21 in all settings where VIA and VILI were evaluated. The pooled prevalence of CIN21 was 2.3% (CI %) with a lowest value of 0.8%, noted in Niamey, and a highest value of 3.3% in Trivandrum 1. This yielded pooled PPVs with wide overlapping confidence intervals, for VIA (11.6%; CI %) and VILI (12.9%; CI %). The NPV of both inspection methods were less heterogeneous and in general higher than 99%. Moreover, the pooled NPV of VILI (99.8%; CI %) was significantly higher than that of VIA (99.4%; CI %). The test positivity rates of VIA and VILI varied widely, but were highly correlated (Spearman s correlation coefficient ), ranging from 6.0 and 9.0%, observed in Niamey, to 27.4 and 28.7%, observed in Ouagadougou, for VIA and VILI, respectively. Summary of test accuracy of all screening test for all categories of CIN The sensitivity and specificity for all tests and outcomes are summarized in Table III. The sensitivity rose substantially with increasing severity of the outcome (>22% difference in sensitivity for respectively CIN11 and cancer), whereas the specificity decreased (3% difference in specificity for, respectively, CIN11 and cancer). All accuracy measures showed statistically significant interstudy heterogeneity (p for Cochrane s Q test <0.01) at the exception of the sensitivity of the HC2 test for outcomes of CIN21, CIN31 and cancer, which were statistically homogenous (p for Cochrane s Q test >0.2). HSROC curves Figure 2 displays the joint variation in observed sensitivity and specificity in each study, the summary ROC curve fitted by HSROC regression, the pooled sensitivity specificity point and its 95% confidence area for VIA and VILI for the outcomes of CIN21 and CIN31. Pooled sensitivities and specificities were very similar to those obtained by the separate meta-analyses shown in Figure 1 and Table III. SROC curves and summary points for VILI were located more upper-left than for VIA. The fitted DORs for CIN21 and CIN31 were, respectively, 25.0 and 64.7 for VIA and, 30.3 and 88.5 for VILI.

4 156 ARBYN ET AL. TABLE II NUMBER OF TRUE AND FALSE POSITIVE AND NEGATIVE RESULTS, TEST ACCURACY OF VIA AND VILI TO DETECT CIN2 OR MORE SEVERE CERVICAL NEOPLASIA; TEST POSITIVITY RATE, PREVALENCE OF CIN21 AND DETECTION RATE OF CIN21; VALUES FROM 11 CENTRES IN AFRICA AND INDIA AND META-ANALYTICALLY POOLED VALUES Study TP FN FP TN Se Sp PPV NPV Test1 Prev Det rate VIA Bamako Brazzaville Conakry Niamey Ouagadougou Calcutta Calcutta Jaipur Mumbai Trivandrum Trivandrum TOTAL/Pooled VILI Bamako Brazzaville Conakry Niamey Ouagadougou Calcutta Jaipur Mumbai Trivandrum Trivandrum TOTAL/Pooled VIA, visual inspection after application of acetic acid; VILI, visual inspection after application of Lugol s iodine; CIN, cervical intra-epithelial neoplasia; TP/FN/FP/TN, number of true positives, false negatives, false positives and true negatives; Se, sensitivity; Sp, specificity; PPV, positive predictive value; NPV, negative predictive value; Test1, test positivity rate; Prev, prevalence of CIN21; Det rate, detection rate of CIN21. TABLE III SENSITIVITY AND SPECIFICITY OF 5 SCREENING TESTS FOR CIN1 OR MORE SEVERE DISEASE (CIN11), CIN21, CIN31 AND CANCER; MINIMUM, MAXIMUM AND META-ANALYTICALLY POOLED MEASURES Test Outcome Test Cutoff Sensitivity Specificity Min Max Pooled (95% CI) Min Max Pooled (95% CI) VIA CIN 11 AW lesions ( ) ( ) CIN21 or growth ( ) ( ) CIN ( ) ( ) Cancer ( ) ( ) VILI CIN 11 Non iodine uptake ( ) ( ) CIN21 yellow areas ( ) ( ) CIN31 or growth ( ) ( ) Cancer ( ) ( ) VIAM CIN11 AW lesions ( ) ( ) CIN21 or growth ( ) ( ) CIN ( ) ( ) Cancer ( ) ( ) Pap smear CIN11 ASCUS ( ) ( ) CIN ( ) ( ) CIN ( ) ( ) Cancer ( ) ( ) CIN11 LSIL ( ) ( ) CIN ( ) ( ) CIN ( ) ( ) Cancer ( ) ( ) CIN21 HSIL ( ) ( ) CIN ( ) ( ) Cancer ( ) ( ) HC2 CIN11 RLU> ( ) ( ) CIN ( ) ( ) CIN ( ) ( ) Cancer ( ) ( ) VIA, visual inspection after application of acetic acid; VILI, visual inspection after application of lugol s iodine; VIAM, visual inspection after application of acetic acid and using a magnifying loupe; HC2, Hybrid-Capture 2 assay; CIN, cervical intra-epithelial neoplasia; AW lesion, aceto-white lesion; ASCUS1, atypical squamous cells of unspecified significance; LSIL, low grade squamous intraepithelial lesion; HSIL, highgrade intraepithelial lesion; RLU, relative light units. Pooled relative accuracy of the screening tests The forest plots in Figure 3 display the relative sensitivity and specificity of VILI compared with VIA with respect to the prediction of CIN21 or CIN31. The relative accuracy of the other screening tests is documented in Table IV. Overall, the sensitivity of VILI for CIN21 and CIN31 was, respectively, 10.5% (CI 4.8

5 POOLED ANALYSIS OF CERVICAL CANCER SCREENING TESTS IN DEVELOPING COUNTRIES %) and 7.4% (CI %) higher than the sensitivity of VIA. The relative sensitivity of VILI was considerably higher than VIA in the African studies. Specificities of both tests were not statistically significantly different. The accuracy of VIAM was similar to that of VIA. The Pap smear had a significantly lower sensitivity for CIN21 than VIA, even at the lowest cytological cutoff of atypical squamous cells of undetermined significance 1 (relative sensitivity ; CI ), but also showed a significantly higher specificity, and this difference rose with the test threshold. The sensitivity of HC2 was lower than that of VIA, but this difference did not reach the level of statistical significance. On the other hand, the specificity of HC2 was 7 to 8% higher and this difference was significant. For all histological outcomes and cytological cutoffs, the Pap smear was significantly less sensitive but more specific than VILI. The HC2 assay also had lower sensitivity than VILI, but this finding was only significant for CIN21. On the other hand, the specificity of the HPV test was significantly higher. The Pap smear showed a lower sensitivity and a higher specificity than the HC2 test. The difference in sensitivity was never significant but was significant for specificity when LSIL1 and HSIL1 were considered as cutoff. FIGURE 2 HSROC curves displaying the fitted sensitivity as a function of specificity of VIA (at left) and VILI (at right) with respect to detection of underlying CIN2 or worse (top) or CIN3 or worse (bottom). Individual studies are represented by small circles. The square corresponds with the pooled sensitivity and specificity and the area surrounded by an interrupted line corresponds with the 95% confidence ellipse. Sources of variation of the diagnostic performance There was no statistically significant variation of the DOR for the outcome of CIN21 by age group. The DOR of VIA increased by study period and also varied significantly by setting. The DORs were also higher in Calcutta 2 or Trivandrum 2 compared with Calcutta 1 or Trivadrum 1 studies. The DORs of VILI, VIAM and cytology at cutoff LSIL1 were significantly higher in the third period compared with the first, but there was no significant difference between the first and second period. There was a country effect for VILI with significantly higher DORs in Congo, Mali, Guinea and Niger compared with FIGURE 3 Forest plots of the relative sensitivity and specificity of VILI compared with VIA for the prediction of cervical intraepithelial neoplasia, grade 2 or more severe neoplasia (CIN21) and for CIN31.

6 158 ARBYN ET AL. TABLE IV RELATIVE SENSITIVITY AND SPECIFICITY OF VIA, VILI, VIAM, CYTOLOGY (AT CUTOFFS ASCUS, LSIL AND HSIL) AND THE HC2 ASSAY VERSUS ANOTHER SCREENING TEST TO DETECT CIN2 OR CIN3 OR MORE SERIOUS CERVICAL DISEASE Test combinations Outcome Relative sensitivity pooled (95% CI) Relative specificity pooled (95% CI) VILI/VIA CIN ( ) ( ) CIN ( ) ( ) VIAM/VIA CIN ( ) ( ) CIN ( ) ( ) Pap smear (ASCUS1)/VIA CIN ( ) ( ) 1 CIN ( ) ( ) 1 Pap smear (LSIL1)/VIA CIN ( ) ( ) 1 CIN ( ) ( ) 1 Pap smear (HSIL1)/VIA CIN ( ) ( ) 1 CIN ( ) ( ) 1 HC2/VIA CIN ( ) ( ) 1 CIN ( ) ( ) 1 Pap smear (ASCUS1)/VILI CIN ( ) ( ) 1 CIN ( ) ( ) 1 Pap smear (LSIL1)/VILI CIN ( ) ( ) 1 CIN ( ) ( ) 1 Pap smear (HSIL1)/VILI CIN ( ) ( ) 1 CIN ( ) ( ) 1 HC2/VILI CIN ( ) ( ) 1 CIN ( ) ( ) 1 Pap smear (ASCUS1)/HC2 CIN ( ) ( ) CIN ( ) ( ) Pap smear (LSIL1)/HC2 CIN ( ) ( ) 1 CIN ( ) ( ) 1 Pap smear (HSIL1)/HC2 CIN ( ) ( ) 1 CIN ( ) ( ) 1 VIA, visual inspection after application of acetic acid; VILI, visual inspection after application of lugol s iodine, VIAM, visual inspection after application of acetic acid and using a magnifying loupe; HC2, Hybrid-Capture 2 assay; CIN, cervical intra-epithelial neoplasia; AW lesion, aceto-white lesion; ASCUS1, atypical squamous cells of unspecified significance; LSIL, low grade squamous intraepithelial lesion; HSIL, highgrade intraepithelial lesion. 1 95% confidence interval (CI) does not include unity, indicating statically different accuracy. Relative sensitivity or specificity >1 indicates higher sensitivity or specificity of the first test; <1 indicates lower sensitivity or specificity of the 1st test. TABLE V CORRELATION BETWEEN RESULTS OF SCREENING TESTS AND COLPOSCOPY ASSESSED WITH THE RANK CORRELATION COEFFICIENT OF SPEARMAN (q) VILI VIAM HC2 Cytology (LSIL1) Colposcopy VIA VILI VIAM HC Cytology 0.26 India. The DOR of VIAM and cytology varied significantly among the Indian settings where the tests were evaluated. HPV testing with HC2 was the only screening method whose accuracy did not vary by period. Nevertheless, a significant variation by setting was observed. For the outcome CIN31, SROC regressions showed similar results, except for HC2 where there was no significant setting effect any more. Readers can find tables containing the coefficients of the SROC regressions in ACCP_ Meta9a.pdf. Correlation between tests The Spearman rank correlation between the respective tests and colposcopy is shown in Table V. High correlations (q > 0.60) were observed for all visual screening tests and colposcopy. Discussion The appropriateness of a screening test depends not only on its accuracy, as measured mainly by sensitivity and specificity but also on its simplicity and safety. Additionally, the PPV and NPV can be used to measure the suitability of the screening tests, but these 2 measures greatly depend on the prevalence of the disease. All these measures of test characteristics can be evaluated, provided that a suitable reference test is carried out, to distinguish the subjects who are truly diseased from those who are not. Our study allows comparative evaluation of the accuracy of 5 different screening tests, VIA, VILI, VIAM, conventional cytology and HPV testing, in the detection of cervical neoplasia. Assessment of the test characteristics of these screening tests in different settings is of considerable importance, as they are useful in determining screening policy decisions. However, there are very few studies from the developing world that permit the test characteristics of these screening tests to be established with minimal verification bias. In our study, the reference standard consisted of histology or colposcopy if no histology result was available. All women, irrespective of the screening test result, underwent a colposcopic examination and biopsies were directed in those who had colposcopic abnormalities. The analyses from this study show that screening with VIA or VILI allows detecting presence of cervical cancer and its precursors with an accuracy as good or even better than the standard Pap smear test or testing for the presence of high-risk HPV with HC2 assay. However, the interstudy variation of VIA and VILI accuracy parameters was wide. Similar sensitivity (83%) and specificity (89%) for VIA was reported in a recent study in Mongolia 19 and in Kenya 20 (sensitivity 73% and specificity 80%). Studies from other researchers have shown similar test sensitivities but far lower test specificities or test specificities similar to ours but with far lower sensitivities Lower sensitivity and specificity have been reported by other workers for VILI. 27,28 This inconsistency across studies reflects the considerable subjectivity in interpreting visual tests by different providers as a result of different levels of competencies, training methods, monitoring and quality assurance and also reflects the fact that visual inspection methods have low reproducibility. The accuracy of VIA, increased signifi-

7 POOLED ANALYSIS OF CERVICAL CANCER SCREENING TESTS IN DEVELOPING COUNTRIES 159 cantly by study phase. It also increased in Trivandrum 2 and Calcutta 2, where the same teams as in Trivandrum 1 and Calcutta 1 did the examinations. These findings underline the need for experience, continuous training and supervision. VILI was 10% more sensitive for detecting CIN21 than VIA, but had the same specificity, thus looks the most preferred method to detect high-grade CIN in developing countries. The reported sensitivity for VILI in other studies has been generally lower. 27,28 VILI requires further evaluation by more providers in other settings. The HC2, showed an unexpectedly low sensitivity (62%) for a high-grade CIN, for a rather high specificity (94%), in the 4 Indian settings. In all reported studies from other developing countries, where the accuracy of HPV testing by HC2 was documented without verification bias, a higher sensitivity exceeding 80% has been observed 20,29 32 and in studies in the developed world a consistently higher sensitivity exceeding 90% has been reported Possible explanations for this low sensitivity in our study could be contamination of the sample by acetic acid or Lugol s iodine or deterioration of the sample because of exposure at high temperature. Contamination of the sample by acetic acid or Lugol s iodine could normally not have occurred, since, according to the protocol, the sample for HC2 was collected before application of vinegar or iodine solution. A laboratory testing problem, is also hardly probable, because of the high concordance between the Indian results and the those performed in a specialized virological French laboratory on a random subsample. 9 Presence of other HPV types not included in the hrhpv DNA probe cocktail is another possibility. However, recent HPV type distribution studies and case control studies conducted in India do not provide evidence for this latter hypothesis. 37,38 Finally, misclassification of the outcome could also explain the low observed sensitivity of HC2. The interstudy variation in HC2 test accuracy was narrow and most often nonsignificant, which most probably reflects high reproducibility, independent of training or experience. Among all evaluated tests, cytology showed the lowest sensitivity, even at the lowest cytological cutoff (57% for CIN21). This adds further to the inconsistent results of cytology observed in low resource settings in which repeated cytology testing is difficult due to logistic problems. It is known that colposcopy followed by biopsy taken from colposcopically suspect lesions is not a perfect gold standard. 39,40 In this study, all visual inspection methods yielded highly correlated results, which might explain the high apparent accuracy (sensitivity and specificity) of the visual methods. The test sensitivity of colposcopy itself was not evaluated in the Alliance for Cervical Cancer Prevention trials. The sensitivity of colposcopy-directed biopsy for CIN21 in women with satisfactory colposcopy was only 57% in a Chinese study where multiple random biopsies were taken from all tested women. 41 Pretorius showed that it is possible that the sensitivity of VIA is overestimated if colposcopically directed biopsy and VIA miss similar small lesions. 42 Moreover, suboptimal blinding of gold standard verification in certain settings may have occurred, for instance, in Conakry where outlying high sensitivity and specificity of VIA were observed. Histological interpretation of small punch biopsies is subjective. Overinterpretation of CIN lesions, which in fact were not CIN21, but VIA or VILI positive and HC2 negative, could explain the apparent high sensitivity of the former and low sensitivity of the latter. In a recent reevaluation of a diagnostic study on cervical cancer screening tests, conducted in Zimbabwe, including correction for gold standard misclassification yielded substantially higher estimates of the sensitivity of HC2 and lower for VIA, compared with original estimates based on colposcopy-based biopsies. 43 It looks plausible that gold standard misclassification was less evident in Indian sites where providers had more experience in carrying out both the screening and confirmatory tests than their African counterparts. It might be clear, for the future, that higher standards for disease confirmation are needed such as p16 immunostaining of histological preparations, strict blinding of assessors, quality review by highly experienced colposcopists and histologists on random sub-samples, taking multiple random biopsies and, last but not least, robust statistical methods adjusting for misclassification and verification biases. It was recently demonstrated in a randomized population trial that once in a lifetime VIA screening reduces incidence of cervical cancer with 25% and cause-specific mortality with 35%. 44 The major finding of our multicentre study was that VLII was more sensitive and as specific compared with VIA. This might generate further hope for improvement of screening with simple and affordable technologies, which challenge other innovative promising preventive policies such as screening with a simple and cheap HPV testing method and HPV vaccination. 45,46 Acknowledgements The authors acknowledge the support received from the Bill & Melinda Gates Foundation (Seattle, WA) through the Alliance for Cervical Cancer Prevention (ACCP) the DWTC/SSTC (Federal Services for Science, Culture and Technology, Brussels, Belgium). The Gynaecological Cancer Cochrane Review Collaboration (Bath, UK). 1. Ferlay J, Bray F, Pisani P, Parkin DM, GLOBOCAN Cancer Incidence, Mortality and Prevalence Worldwide. IARC CancerBase No. 5, version 2.0. Lyon: IARC Press, Yang BH, Bray FI, Parkin DM, Sellors JW, Zhang Z-F. Cervical cancer as a priority for prevention in different world regions: an evaluation using years of life lost. Int J Cancer 2004;109: Parkin DM, Bray F, Ferlay J, Pisani P. Global cancer statistics, CA Cancer J Clin 2005;55: Parkin DM, Bray F. The burden of HPV-related cancers. Vaccine 2006;24 (Suppl 3, Chap 2):S11 S Sankaranarayanan R, Basu P, Wesley RS, Mahe C, Keita N, Mbalawa CCG, Sharma R, Dolo A, Shastri SS, Nacoulma M, Nayama M, Somanathan T, et al. Accuracy of visual screening for cervical neoplasia: results from an IARC multicentre study in India and Africa. Int J Cancer 2004;110: Sankaranarayanan R, Wesley R, Thara S, Dhakad N, Chandralekha B, Sebastian P, Chithrathara K, Parkin DM, Nair MK. Test characteristics of visual inspection with 4% acetic acid (VIA) and Lugol s iodine (VILI) in cervical cancer screening in Kerala, India. Int J Cancer. 2003;106: Sankaranarayanan R, Shastri SS, Basu P, Mahe C, Mandal R, Amin G, Roy C, Muwonge R, Goswami S, Das P, Chinoy R, Frappart L, et al. The role of low-level magnification in visual inspection with acetic acid for the early detection of cervical neoplasia. Cancer Detect Prev 2004;28: References 8. Sankaranarayanan R, Thara S, Sharma A, Roy C, Shastri S, Mahe C, Muwonge R, Fontaniere B, The Multicentre Study Group On Cervical Cancer Early Detection In India. Accuracy of conventional cytology: results from a multicentre screening study in India. J Med Screen 2004;11: Sankaranarayanan R, Chatterji R, Shastri SS, Basu P, Mahe C, Muwonge R, Seigneurin D, Somanathan T, Roy C, Kelkar R, Chinoy R, Dinshaw K, et al. Accuracy of human papillomavirus testing in primary screening of cervical neoplasia: results from a multicentre study in India. Int J Cancer 2004;112: Arbyn M, Gaffikin L, Sankaranarayanan R, Muwonge R, Keita N, Wesley R, Tsu V. Assessment of Innovative Approaches to Cervical Cancer Screening, Follow-up and Treatment of Screen Detected Cervical Lesions in Developing Countries. A Pooled Analysis of ACCP Trial Results IPH/EPI-REPORTS 2005;2: Seattle, Brussels: PATH & IPH, pp Sankaranarayanan R, Wesley RS. A practical manual on visual screening for cervical neoplasia. Lyon: IARC Press and World Health Organization Distribution and Sales, 2003: pp Lorincz AT. Methods of DNA hybridisation and their clinical applicability to human papillomavirus detection. In: Franco EL, Monsonego J, eds. New developments in cervican cancer screening and prevention. Oxford: Blackwell Science, 1997: Sharp S, Sterne J. Meta-analysis. Stata Tech Bull 1997:9 14.

8 160 ARBYN ET AL. 14. Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. Methods for meta-analysis in medical research. Chichester: Wiley, 2000: Chu H, Cole SR. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol 2006;59: Rutter CM, Gatsonis CA. A hierarchical regression approach to metaanalysis of diagnostic test accuracy evaluations. Stat Med 2002;20: Harbord RM, Deeks JJ, Egger M, Whiting P, Sterne JA. A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2007;8: Moses LE, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med 1993;12: Elit L, Baigal G, Tan J, Munkhtaivan A. Assessment of 2 cervical screening methods in mongolia: cervical cytology and visual inspection with acetic Acid. J Low Genit Tract Dis 2006;10: De Vuyst H, Claeys P, Njiru S, Muchiri L, Steyaert S, De Sutter P, van Marck E, Bwayo J, Temmerman M. Comparison of pap smear, visual inspection with acetic acid, human papillomavirus DNA-PCR testing and cervicography. Int J Gynecol Obstet 2005;89: Londhe M, George SS, Seshadri L. Detection of CIN by naked eye visualization after application of acetic acid. Indian J Cancer 1997;34: University of Zimbabwe/JHPIEGO, Anonymous. Visual inspection with acetic acid for cervical-cancer screening: test qualities in a primary-care setting. Lancet 1999;353: Cronje HS, Parham GP, Cooreman BF, de Beer A, Divall P, Bam RH. A comparison of four screening methods for cervical neoplasia in a developing country. Am J Obstet Gynecol 2003;188: Bhatla N, Mukhopadhyay A, Joshi S, Kumar A, Kriplani A, Pandey RM, Verma K. Visual inspection for cervical cancer screening: evaluation by doctors versus paramedical workers. Indian J Cancer 2004; 41: Denny L, Kuhn L, Pollack A, Wainwright H, Wright TC. Evaluation of alternative methods of cervical cancer screening for resource-poor settings. Cancer. 2000;89: Cronje HS, Cooreman BF, Beyer E, Bam RH, Middlecote BD, Divall PD. Screening for cervical neoplasia in a developing country utilizing cytology, cervicographyand the acetic acid test. Int J Gynecol Obstet 2001;72: Sarian LO, Derchain SF, Naud P, Roteli-Martins C, Longatto-Filho A, Tatti S, Branca M, Erzen M, Serpa-Hammes L, Matos J, Gontijo RC, Braganca JF, et al. Evaluation of visual inspection with acetic acid (VIA). Lugol s iodine (VILI), cervical cytology and. HPV testing as cervical screening tools in Latin America. J Med Screen 2005; 12: Sangwa-Lugoma G, Mahmud S, Nasr SH, Liaras J, Kayembe PK, Tozin RR, Drouin P, Lorincz A, Ferenczy A, Franco EL. Visual inspection as a cervical cancer screening method in a primary health care setting in Africa. Int J Cancer 2006;119: Wright TC, Denny L, Kuhn L, Pollack A, Lorincz AT. HPV DNA testing of self-collected vaginal samples compared with cytologic screening to detect cervical cancer. JAMA 2000;283: Womack SD, Chirenje ZM, Blumenthal PD, Gaffikin L, McGrath JA, Chipato T, Ngwalle E, Shah KV. Evaluation of a human papillomavirus assay in cervical screening in Zimbabwe. BJOG 2000;107: Belinson JL, Qiao YL, Pretorius R, Zhang WH, Elson P, Li L, Pan QJ, Fischer C, Lorincz AT, Zahniser D. Shanxi province cervical cancer screening study: a cross-sectional comparative trial multiple techniques to detect cervical neoplasia. Gynecol Oncol 2001;83: Almonte M, Ferreccio C, Winkler JL, Cuzick J, Tsu V, Robles S, Takahashi R, Sasieni P. Cervical screening by visual inspection. HPV testing, liquid-based and conventional cytology in Amazonian Peru. Int J Cancer 2007;121: Franco EL. Primary screening of cervical cancer with human papillomavirus tests. J Natl Cancer Inst Monogr 2003;31: Koliopoulos G, Arbyn M, Martin-Hirsch P, Kyrgiou M, Prendiville W, Paraskevaidis E. Diagnostic accuracy of human papillomavirus testing in primary cervical screening: a systematic review and metaanalysis of non randomised studies. Gynecol Oncol 2007;104: Arbyn M, Sasieni P, Meijer CJ, Clavel C, Koliopoulos G, Dillner J. Chapter 9: Clinical applications of HPV testing: a summary of metaanalyses. Vaccine. 2006;24(Suppl 3): Cuzick J, Clavel C, Petry KU, Meijer CJ, Hoyer H, Ratnam S, Szarewski A, Birembaut P, Kulasingam S, Sasieni P, Iftner T. Overview of the European and North American studies on HPV testing in primary cervical cancer screening. Int J Cancer 2006;119: Franceschi S, Rajkumar T, Vaccarella S, Gajalakshmi V, Sharmila A, Snijders PJ, Munoz N, Meijer CJ, Herrero R. Human papillomavirus and risk factors for cervical cancer in Chennai. India: a case-control study. Int. J Cancer. 2003;107: Franceschi S, Rajkumar R, Snijders PJ, Arslan A, Mahe C, Plummer M, Sankaranarayanan R, Cherian J, Meijer CJ, Weiderpass E. Papillomavirus infection in rural women in southern India. Br J Cancer 2005;92: Gage JC, Hanson VW, Abbey K, Dippery S, Gardner S, Kubota J, Schiffman M, Solomon D, Jeronimo J. Number of cervical biopsies and sensitivity of colposcopy. Obstet Gynecol 2006;108: Jeronimo J, Schiffman M. Colposcopy at a crossroads. Am J Obstet Gynecol 2006;195: Pretorius RG, Zhang WH, Belinson JL, Huang MN, Wu LY, Zhang X, Qiao YL. Colposcopically directed biopsy, random cervical biopsy, and endocervical curettage in the diagnosis of cervical intraepithelial neoplasia II or worse. Am J Obstet Gynecol 2004;191: Pretorius RG, Kim RJ, Belinson JL, Elson P, Qiao YL. Inflation of sensitivity of cervical cancer screening tests secondary to correlated error in colposcopy. J Low Genit Tract Dis 2006;10: Gaffikin L, McGrath JA, Arbyn M, Blumenthal PD. Accuracy of visual inspection with acetic acid as a cervical cancer test validated using Latent Class Analysis. BMC Med Res Methodol 2007;7: Sankaranarayanan R, Esmy PO, Rajkumar R, Muwonge R, Swaminathan R, Shanthakumari S, Fayette JM, Cherian J. Effect of visual screening on cervical cancer incidence and mortality in Tamil Nadu India: a cluster-randomised trial. Lancet 2007;370: Goldie S. CHAPTER 10 A public health approach to cervical cancer control: Considerations of screening and vaccination strategies. Int J Gynaecol Obstet 2006;94(Suppl 1):S95 S Sherris J, Wright TC, Jr, Denny L, Sankaranarayanan R, Pollack AE, Sanghvi H, Sellors JW. Alliance for cervical cancer prevention: setting the record straight. Am J Public Health 2007;97:200 1.