Computer-Assisted Pattern Recognition of Autoantibody Results

CLINICAL AND DIAGNOSTIC LABORATORY IMMUNOLOGY, Dec. 2005, p. 1353 1357 Vol. 12, No. 12 1071-412X/05/$08.00 0 doi:10.1128/cdli.12.12.1353 1357.2005 Copyright 2005, American Society for Microbiology. All Rights Reserved. Computer-Assisted Pattern Recognition of Autoantibody Results Steven R. Binder, 1 * Mark C. Genovese, 2 Joan T. Merrill, 3 Robert I. Morris, 4 and Allan L. Metzger 4 Bio-Rad Laboratories, Hercules, California 94547 1 ; Division of Immunology and Rheumatology, Stanford University, Stanford, California 94305 2 ; Oklahoma Medical Research Foundation, Oklahoma City, Oklahoma 73104 3 ; and RDL Reference Laboratory, Los Angeles, California 90034 4 Received 29 July 2005/Returned for modification 6 September 2005/Accepted 5 October 2005 Immunoassay-based anti-nuclear antibody (ANA) screens are increasingly used in the initial evaluation of autoimmune disorders, but these tests offer no pattern information comparable to the information from indirect fluorescence assay-based screens. Thus, there is no indication of next steps when a positive result is obtained. To improve the utility of immunoassay-based ANA screening, we evaluated a new method that combines a multiplex immunoassay with a k nearest neighbor (knn) algorithm for computer-assisted pattern recognition. We assembled a training set, consisting of 1,152 sera from patients with various rheumatic diseases and nondiseased patients. The clinical sensitivity and specificity of the multiplex method and algorithm were evaluated with a test set that consisted of 173 sera collected at a rheumatology clinic from patients diagnosed by using standard criteria, as well as 152 age- and sex-matched sera from presumably healthy individuals (sera collected at a blood bank). The test set was also evaluated with a HEp-2 cell-based enzymelinked immunosorbent assay (ELISA). Both the ELISA and multiplex immunoassay results were positive for 94% of the systemic lupus erythematosus (SLE) patients. The knn algorithm correctly proposed an SLE pattern for 84% of the antibody-positive SLE patients. For patients with no connective tissue disease, the multiplex method found fewer positive results than the ELISA screen, and no disease was proposed by the knn algorithm for most of these patients. In conclusion, the automated algorithm could identify SLE patterns and may be useful in the identification of patients who would benefit from early referral to a specialist, as well as patients who do not require further evaluation. Screening for anti-nuclear antibodies (ANA) has been performed for many years to identify patients with autoimmune diseases. The strong association of ANA with systemic lupus erythematosus (SLE) is well established, and this finding satisfies 1 of the 11 American College of Rheumatology (ACR) criteria available for establishing the diagnosis. However, the high sensitivity of the screen is not accompanied by a strong positive predictive value. In a study of 1,010 patients in which the estimated sensitivity of the ANA test for SLE was 100%, the positive predictive value was 11% for SLE and 11% for other rheumatic diseases (30). An evidence-based meta-analysis established that the ANA screen had an excellent negative likelihood ratio (0.11), but the positive likelihood ratio for SLE was only 2.2, and the positive likelihood ratio for other major rheumatic diseases was below 2.0, indicating that the screen was not useful (32). Almost all of the literature that addresses the clinical performance of the ANA screen is based on the detection of antibodies by immunofluorescence (indirect fluorescence assay [IFA]), which requires microscopic examination of positive results at multiple dilutions. Recently, there has been a recommendation to increase the cutoff for reporting a positive test for the ANA so that the false-positive rate is roughly 5% in the nondiseased population (17). Nevertheless, the 1:40 cutoff is still widely used, in order to maintain a high level of sensitivity. Even higher cutoffs have been recommended for patients under 18 years of age (23). * Corresponding author. Mailing address: Bio-Rad Laboratories, 4000 Alfred Nobel Drive, Hercules, CA 94547. Phone: (510) 741-4603. Fax: (510) 741-6499. E-mail: steve_binder@bio-rad.com. Over the past 10 years, enzyme-linked immunosorbent assays (ELISA) have been introduced to identify patients with ANA (37). These tests can be automated, and they do not require highly trained operators who can recognize microscopic patterns. However, these newer tests yield only numeric results and cannot produce pattern information, which has traditionally aided in the interpretation of IFA results. Recently, methods for the simultaneous measurement of multiple anti-nuclear antibodies using arrays have been described (7, 14, 19, 25, 26). These methods show a more selective response than traditional tissue-based screens, since a limited set of antigens must be defined. They offer the advantages of complete automation, consistent performance, and more precise measurement of antibody levels. Further, these methods provide numeric results for multiple antibodies, providing an opportunity to identify antibody patterns associated with SLE using a computer-assisted algorithm. We describe here the use of an array method to identify autoantibodies in sera from a large cohort of diseased and nondiseased individuals. These data were then used to develop a k nearest neighbor (knn) (pattern recognition) algorithm for the identification of antibody patterns associated with SLE, antibody patterns associated with other connective tissue diseases, and patterns associated with the absence of rheumatic disease. MATERIALS AND METHODS Study objective and design. The objective of this study was to compare three methods for screening sera to detect connective tissue disorders (CTDs), and specifically to detect SLE. The three methods we examined were (i) an ELISA method, (ii) a multiplex autoimmune assay, and (iii) a pattern recognition algorithm operating on the multiplex assay results. 1353

1354 BINDER ET AL. CLIN. DIAGN. LAB. IMMUNOL. TABLE 1. Comparison of the diagnoses of individuals who provided sera in the training set and test set Training set (n 1,152) No. (%) of sera Test set (n 325) No CTD 458 (39.8) 190 (58.4) Nontargeted CTD 253 (22.0) 90 (27.7) Targeted CTD 441 (38.3) 45 (14.2) SLE 303 (26.3) 32 (9.8) MCTD 19 (1.6) 4 (1.2) SLE and MCTD 1 (0.1) 2 (0.6) Sjögren s syndrome 21 (1.8) 2 (0.6) Scleroderma 51 (4.4) 4 (1.2) Polymyositis 46 (4.0) 1 (0.3) We acquired two sets of sera with known clinical diagnoses from a number of sources. We used the first set of sera to train the pattern detection algorithm, and we used the second set of sera to evaluate the performance of all three methods. Our endpoint was the positive and negative likelihood ratios for the different methods. We describe the diagnostic methods, sera, and statistical methods in the following sections. Diagnostic methods. The ELISA method was the Helix ANA screen (West Sacramento, CA), which utilizes a mixture of HEp-2 cell extract and several supplementary antigens as the antigenic target (15). For this method, we reported sera with antibody index (AI) values of 1.0 or higher as positive. The multiplex autoimmune assay method was the BioPlex 2200 ANA screen (Bio-Rad Laboratories, Hercules, CA), which employs dyed magnetic beads to perform measurements of 13 autoimmune antibodies simultaneously in one tube (10, 16, 18, 28). Briefly, 13 different dyed beads were precoated with SSA-52, SSA-60, SSB, Sm, SmRNP, RNP-68, RNP-A, Jo-1, ribosomal protein, Scl-70 (topoisomerase), centromere B, chromatin, and double-stranded DNA (dsdna). Recombinant antigens from Diarect (Freiberg, Germany) were used for SSA-52, RNP-68, RNP-A, Jo-1, Scl-70, and centromere B. Affinity-purified antigens from Arotec (New Zealand) were used for Sm and SSB. Affinity-purified SSA-60, SmRNP, and ribosomal protein were prepared at Helix (West Sacramento, CA). Chromatin and dsdna were prepared at Bio-Rad Laboratories by standard methods (4 6). The bead mixture was combined with patient sample and diluent and allowed to incubate for 20 min at 37 C. After a wash cycle, anti-human immunoglobulin G antibody conjugated to B-phycoerythrin was added to the dyed beads and allowed to incubate for 10 min at 37 C. Following removal of excess conjugate, the mixture passed through the detector, which identified the bead type, based on the fluorescence of the dyed bead. The amount of antibody bound to the bead was determined by the fluorescence of B-phycoerythrin. Raw data were initially measured as relative fluorescence intensity and then converted to FR by normalization using a predyed internal standard bead. A series of calibrators was analyzed with the patient samples to convert the FR into international units per milliliter for anti-dsdna and AI for the other antibodies measured. For anti-dsdna, levels of 10 IU/ml or higher were positive; for the other analytes, AI values of 1.0 or higher were positive. For this method, we reported the screening result as positive when 1 or more of the 13 antibody results was positive. The specificity of this method was established in a recent study of 510 healthy subjects by multiple methods (28). The pattern recognition method was a k nearest neighbor algorithm (39) operating on the numerical results from the multiplex autoimmune assay. This software was developed in-house by one of the authors (S. R. Binder); it is an optional component included with the instrument software. Other researchers have used the same pattern recognition method for many diagnostic purposes, such as identifying volatiles associated with infectious disease (41), tracking tumor growth rates (12), and detecting disease clustering using data from children with high blood lead levels (34). Another pattern recognition method, the artificial neural network, has been utilized for classification of giant cell arteritis (2), and for the prediction of lupus nephritis in patients with SLE (24). Before the knn algorithm is used for prediction, it must be trained using a number of sera from patients with known diagnoses. We used 1,152 samples (Table 1) for this purpose. When the trained algorithm is presented with a new serum sample, it compares the characteristics of the new sample to previously encountered samples with similar characteristics to predict the diagnosis of the unknown sample. The characteristics of the sample were the log-transformed FR (AI) values for the 13 autoimmune antibodies examined by the multiplex assay. The TABLE 2. Results for the test set (n 325) a Total no. of specimens No. of specimens positive by: ELISA Multiplex knn No CTD 190 42 18 7 Nontargeted CTD 90 37 24 10 Targeted CTD 45 42 41 38 SLE 34 32 32 29 Total 325 121 83 55 a Positive results produced by ELISA compared to positive results produced by the multiplex method and to positive disease associations proposed by the knn algorithm. previously encountered samples, which are considered when assigning a pattern to a new serum sample, are known as the k nearest neighbors. The algorithm was developed and tested using the leave one out strategy, where one sample is removed from the training set and then tested as an unknown against the remaining samples (39). Various parameters required for successful performance were chosen with the assistance of a commercial optimization algorithm (ECHIP, Hockessin, DE). Only neighbors showing at least 30% cumulative agreement were considered (neighbors based on geometric distance [from the new sample to the neighbor for each antibody]). We selected 11 as the value for k and 30% as the value for cumulative agreement. The final step in the prediction process is to assign a pattern to the new serum sample based on the diagnoses of its k nearest neighbors. This was accomplished by accepting diagnoses that were found in at least 25% (at least three) of the neighbors. The knn algorithm can identify patterns associated with the following connective tissue diseases ( targeted CTDs ): mixed connective tissue disease (MCTD), SLE, Sjögren s syndrome, scleroderma, and polymyositis. In addition, it can report that no association could be established (insufficient neighbors or no consistent type of neighbors). If a new serum sample was associated with more than one disease, we reported all associations that met our agreement criterion. Serum sample selection. We collected 1,152 sera to use as a training set for the pattern recognition algorithm. As the predictive ability of the pattern recognition algorithm is related to the diversity of the samples in the training set, we collected sera from a number of institutions. All sera were identified by clinical diagnoses, and all other identifying information was stripped from the samples prior to inclusion in the study. Two authors (R. I. Morris and A. L. Metzger) collected sera (n 570) from individuals seen in their private rheumatology practice between 1996 and 2002. Additional sera (n 381), collected between 2000 and 2002, were obtained from The Registry for the Antiphospholipid Syndrome, which has its principal site at the Oklahoma Medical Research Foundation. These samples included autoimmune sera from individuals from different regions in the United States that were obtained, after informed consent, for use in a collaborative, blinded repository and deidentified clinical database. Both of these collections contained patients with SLE, Sjögren s syndrome, myositis, scleroderma, rheumatoid arthritis, and MCTD. In addition, both collections included sera from patients with osteoarthritis, fibromyalgia, antiphospholipid syndrome, and vasculitis, as well as sera from individuals with no known disease. The diagnoses of SLE and rheumatoid arthritis were established according to the ACR criteria (1, 13, 35). Because polymyositis and scleroderma are relatively rare diseases and not all patients with these diseases produce the antibodies of interest (Jo-1 and Scl-70, respectively), additional sera (n 31) from patients with these conditions were acquired from the Oklahoma Medical Research Foundation and from SLR Research (San Diego, CA). Sera (n 24) from patients with juvenile-onset diabetes were acquired from Intergen (Purchase, NY). Finally, since autoantibodies are commonly encountered in individuals without any known autoimmune disease, we included samples from patients with positive ANA screens (IFA titer of 1:40 or greater) but without any known autoimmune disease (n 146), obtained from the Foundation for Blood Research (Scarborough, ME). To test the performance of the diagnostic methods, we used a separate collection of 325 sera; these are referred to collectively as the test set. One author (M. C. Genovese) collected 173 sera as part of a prospective rheumatologic study conducted at Stanford University. Consecutive patients seen in a rheumatology clinic were enrolled in this study after informed consent was obtained. The protocol for this study was approved by the institutional review board at the Stanford University School of Medicine. The diagnoses of SLE, rheumatoid

VOL. 12, 2005 PATTERN RECOGNITION OF AUTOANTIBODY RESULTS 1355 TABLE 3. Output of the knn interpretative algorithm for the samples collected at a rheumatology clinic compared to rheumatologist-assigned diagnoses a Total no. of samples No. of patients with positive antibody results No. of samples with the following knn output b : SLE MCTD Scleroderma SS PMYO NA Total no. of output results No CTD 38 5 0 0 0 0 0 5 5 Nontargeted CTD 90 24 8 0 1 1 1 14 25 SLE 32 30 25 3 3 3 0 3 37 SLE and MCTD 2 2 2 1 0 0 0 0 3 MCTD 4 4 4 2 0 0 0 0 6 Scleroderma 4 3 0 0 3 0 0 0 3 Sjögren s syndrome 2 1 1 0 0 1 0 0 2 Polymyositis 1 1 0 0 0 0 1 0 1 Total 173 70 40 6 7 5 2 22 82 a Note that the total number of output results from the knn interpretative algorithm exceeds the number of patients with positive antibody results, because some samples were associated with more than one disease. b SS, Sjögren s syndrome; PMYO, polymyositis; NA, no association. The values in bold type show results where the knn output and clinical assessment are in concordance. arthritis, scleroderma, Sjögren s syndrome, and polymyositis/dermatomyositis were assigned according to established criteria (1, 13, 33, 35, 36, 38). For each patient, we recorded the known diagnoses and demographic data. We also collected an additional 152 sera from presumably healthy individuals at a blood bank to evaluate the specificity of both antibody detection and computer-assisted pattern recognition; these samples were age and sex matched against the samples collected at the Stanford rheumatology clinic. Table 1 shows the diagnoses of the patients whose sera were collected in the test and training sets. Statistical methods. We assessed the ability of each diagnostic method to discriminate sera in two disease groups. The disease groups were (i) individuals with SLE and (ii) individuals with a set of all of the targeted CTDs. The comparison group was individuals with no relevant disease or one of the nontargeted CTDs. We calculated the sensitivity of each diagnostic method for both disease groups and the specificity of each method for both nondisease comparison groups. The 95% confidence intervals for sensitivity and specificity were calculated using Wilson s score method with a continuity correction (22). We calculated positive and negative likelihood ratios for each diagnostic method using all possible combinations of disease and nondisease comparison groups. The method described by Simel et al. was used to calculate confidence intervals around likelihood ratios (29). In addition, we tested whether any of the methods offer significantly better sensitivity or specificity by using Cochran s Q-test (31). RESULTS The data produced by the test set were analyzed to determine the number of positive specimens obtained by the two immunological techniques and the software approach. The results of this analysis are presented in Table 2. Table 2 shows the number of positive specimens determined by the ELISA method and multiplex method; these data are compared to the number of specimens associated with a targeted CTD, based upon computer-assisted pattern recognition (knn). Table 3 shows the output of the knn algorithm for 173 sera from the fully characterized patients enrolled in the study at the Stanford University Clinic. An appropriate knn result was proposed for 53 of the 70 antibody-positive patients (75.7%), including 27 of the 32 patients (84.4%) with SLE who were positive for at least one antibody (two were antibody negative by both methods). No further statistical analysis was performed due to the small number of patients in each group. Table 4 displays the performance characteristics of the three diagnostic methods for SLE and for all targeted connective tissue diseases. There is a significant difference in the sensitivity of the three methods for SLE (Q 6.0; P 0.049) and for targeted CTDs (Q 7.6; P 0.022). Table 4 shows that the sensitivity of all methods is greater for detecting SLE than for detecting the presence of any targeted CTD. The sensitivity of the ELISA method is greater than the sensitivity of the multiplex method for targeted CTDs, but examination of the confidence intervals reveals that this difference is not significant. The two methods differ chiefly in the use of a HEp-2 cell extract in the ELISA kit, which theoretically permits the detection of many other antibodies that will be missed by the limited number of antigens available in the multiplex method. TABLE 4. Performance of three diagnostic methods in detecting two types of disease conditions a Test Disease Positive LR b Negative LR Sensitivity Specificity ELISA screen TCTD c 2.87 [2.38 3.45] 0.097 [0.032 0.290] 0.935 [0.811 0.983] 0.674 [0.615 0.728] SLE 2.89 [2.39 3.48] 0.087 [0.023 0.336] 0.941 [0.789 0.990] 0.674 [0.615 0.728] Multiplex TCTD 5.92 [4.40 7.96] 0.128 [0.056 0.293] 0.891 [0.756 0.959] 0.849 [0.801 0.888] SLE 6.25 [4.67 8.36] 0.069 [0.018 0.266] 0.941 [0.789 0.990] 0.849 [0.801 0.888] knn TCTD 12.1 [7.70 19.1] 0.187 [0.099 0.351] 0.826 [0.680 0.917] 0.932 [0.894 0.957] SLE 12.5 [7.94 19.8] 0.158 [0.070 0.355] 0.853 [0.682 0.945] 0.932 [0.894 0.957] a Ninety-five percent confidence intervals are shown in brackets. b LR, likelihood ratio. c TCTD, targeted CTD.

1356 BINDER ET AL. CLIN. DIAGN. LAB. IMMUNOL. In the test set reported, this benefit was not observed for any of the SLE patients and was only observed for one scleroderma sample. The knn algorithm has lower sensitivity than both the ELISA and multiplex methods for detection of SLE and for detection of targeted CTDs, but these differences are also not statistically significant. However, it should be noted that the knn algorithm is providing disease-specific information, unlike the other antibody screens. It is more useful in practice to consider likelihood ratios than sensitivity and specificity, because likelihood ratios provide a better indication of the influence of a positive or negative test on clinical decision-making (11). Table 4 lists the positive likelihood ratios and 95 percent confidence intervals of each diagnostic method for SLE and for CTDs when the nondisease comparison group contains nontargeted CTDs. The positive likelihood ratios are consistently larger for the knn method than for the multiplex method, and the positive likelihood ratios for the multiplex method are consistently greater than for the ELISA method. This implies that a positive result from the knn algorithm is more informative in a clinical context than a positive result from either of the other two methods. DISCUSSION The data presented here support the ability of a k nearest neighbor algorithm to classify antibody patterns on the basis of their resemblance to samples from patients with targeted connective tissue diseases or their lack of resemblance to such samples. The algorithm uses case-based reasoning, an algorithm that is easy to understand because it compares results from patients directly to results from other patients. Recently, case-based reasoning has been used to classify patients with early arthritis (40). Computer-assisted diagnosis of rheumatic disorders has been evaluated for over four decades. A review published in 1991 identified 14 different software systems with applications to rheumatology, of which 7 were developed for general internal medicine and 7 were developed specifically for rheumatologic problems (20). Several of the rheumatology-focused systems have been further refined (3, 21, 27). All of these systems rely primarily on clinical symptoms. The approach described here has obvious limitations due to the use of serological data alone, and it is designed to support a diagnosis rather than supply one. It may benefit from the uniformity of data produced by automated instrumentation, since results often vary greatly between laboratories and between methods (9). A comparison of the performance of IFA and a number of ELISA methods for ANA screening was recently presented (8). All of the ELISA methods used purified antigens in combination with HEp-2 cell extracts or recombinant antigens. Results were compared against diagnosis using high-prevalence and low-prevalence populations. The IFA was performed at a 1:160 dilution to minimize false-positive results in accordance with current recommendations (17); as a result, several ELISA methods were more sensitive than IFA. In the study reported here, we show that a multiplex method can produce results comparable to ELISA for disease state samples (Table 2). For patients without known CTDs, there were less positive results, and the use of the knn algorithm as an additional screen further reduced the number of positive samples that might require follow-up testing and referrals. The performance reported here for the multiplex screen relates specifically to the targeted diseases as defined above. Other diseases (such as rheumatoid arthritis, primary biliary cirrhosis, and dermatomyositis) may produce characteristic autoantibodies, but these disease-associated antibodies are not reportable as part of a defined ANA panel in the United States, due to FDA definitions. Further, an array-based technology cannot replace the use of HEp-2 cell-based ANA screens for autoimmune hepatitis or for monitoring patients with juvenile rheumatoid arthritis (32). The test population reported here is small, and additional studies are in progress to evaluate the utility of pattern recognition in a larger prospective study. The samples that were collected at a blood bank were not from clinically characterized patients, and some of the antibody-positive sera may reflect the presence of actual disease. The blood bank study offers an estimate of specificity that can be readily repeated in other laboratories and in other countries (28). Conclusion. The ANA screen has been previously used as a qualitative tool to satisfy one of the ACR criteria. The use of multiple antibody results, coupled with computer-generated pattern interpretation, demonstrates another option that could aid in the identification of patients who may benefit from an early referral to a specialist and in the identification of patients for whom a referral might not be needed. ACKNOWLEDGMENTS We thank Morris Reichlin, Oklahoma Medical Research Foundation, for providing specimens from patients with polymyositis; Kevin Bickford, SLR Inc., for providing specimens from patients with scleroderma; and Thomas Ladue (Foundation for Blood Research) for providing additional IFA-positive sera. We also acknowledge the contribution of John Glossenger (Bio-Rad Laboratories), who programmed software for the knn algorithm. David Buckeridge, Stanford University, assisted with the statistical presentation of the data. REFERENCES 1. Arnett, F. C., S. M. Edworthy, D. A. Bloch, D. J. McShane, J. F. Fries, N. S. Cooper, L. A. Healey, S. R. Kaplan, M. H. Liang, and H. S. Luthra. 1988. The American Rheumatology Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheum. 31:315 324. 2. Astion, M. L., M. H. Wener, R. G. Thomas, G. G. Hunder, and D. A. Bloch. 1994. Application of neural networks to the classification of giant cell arteritis. Arthritis Rheum. 37:760 770. 3. Athreya, B. H., M. L. Cheh, and L. C. Kingsland III. 1998. Computerassisted diagnosis of pediatric rheumatic diseases. Pediatrics 102:1 7. 4. Breneman, J. R., III. 1992. Histone H1 distribution in chromatin and chromosomes. Ph.D. thesis. University of California, Davis. 5. Burlingame, R. W., and R. L. Rubin. 1990. Subnucleosome structures as substrates in enzyme-linked immunosorbent assays. J. Immunol. Methods 134:187 199. 6. Emlen, W., P. Jarusiripipat, and G. Budick. 1990. A new ELISA for the detection of double-stranded DNA antibodies. J. Immunol. Methods 132: 91 101. 7. Feng, Y., X. Ke, R. Ma, Y. Chen, G. Hu, and F. Liu. 2004. Parallel detection of autoantibodies with microarrays in rheumatoid diseases. Clin. Chem. 50:416 422. 8. Fenger, M., A. Wiik, M. Hoier-Madsen, J. J. Lykkegaard, T. Rozenfeld, M. S. Hansen, B. D. Samsoe, and S. Jacobsen. 2004. Detection of antinuclear antibodies by solid-phase immunoassays and immunofluorescence analysis. Clin. Chem. 50:2141 2147. 9. Fritzler, M. J., A. Wiik, E. M. Tan, J. S. Smolen, J. S. McDougal, E. K. Chan, T. P. Gordon, J. A. Hardin, J. R. Kalden, R. G. Lahita, R. N. Maini, W. H. Reeves, N. F. Rothfield, Y. Takasaki, M. Wilson, M. G. Byrd, L. Slivka, and J. A. Koziol. 2003. A critical evaluation of enzyme immunoassay kits for detection of antinuclear autoantibodies of defined specificities. III. Comparative performance characteristics of academic and manufacturers laboratories. J. Rheumatol. 30:2374 2381.

VOL. 12, 2005 PATTERN RECOGNITION OF AUTOANTIBODY RESULTS 1357 10. Fulton, R. J., R. L. McDade, P. L. Smith, L. J. Kienker, and J. R. Kettman, Jr. 1997. Advanced multiplexed analysis with the FlowMetrix system. Clin. Chem. 43:1749 1756. 11. Grimes, D. A., and K. F. Schulz. 2005. Refining clinical diagnosis with likelihood ratios. Lancet 365:1500 1505. 12. Haney, S. M., P. M. Thompson, T. F. Cloughesy, J. R. Alger, and A. W. Toga. 2001. Tracking tumor growth rates in patients with malignant gliomas: a test of two algorithms. Am. J. Neuroradiol. 22:73 82. 13. Hochberg, M. C. 1997. Updating the American College of Rheumatology revised criteria for the evaluation of systemic lupus erythematosus. Arthritis Rheum. 40:1725. 14. Hoffman, I. E. A., I. Peene, E. M. Veys, and F. De Keyser. 2002. Detection of specific antinuclear reactivities in patients with negative anti-nuclear antibody immunofluorescence screening tests. Clin. Chem. 48:2171 2176. 15. Homburger, H. A., Y. D. Cahen, J. Griffiths, and G. L. Jacob. 1998. Detection of antinuclear antibodies: comparative evaluation of enzyme immunoassay and indirect immunofluorescence methods. Arch. Pathol. Lab. Med. 122:993 999. 16. Kaul, R., P. Chen, and S. R. Binder. 2004. Detection of immunoglobulin M antibodies specific for Toxoplasma gondii with increased selectivity for recently acquired infections. J. Clin. Microbiol. 42:5705 5709. 17. Kavanaugh, A., R. Tomar, J. Reveille, D. H. Solomon, and H. A. Homburger. 2000. Guidelines for clinical use of the antinuclear antibody test and tests for specific autoantibodies to nuclear antigens. Arch. Pathol. Lab. Med. 124:71 81. 18. Klutts, J. S., R. S. Liao, W. M. Dunne, and A. M. Gronowski. 2004. Evaluation of a multiplexed bead assay for assessment of Epstein-Barr virus. J. Clin. Microbiol. 42:4996 5000. 19. Martens, T. B., R. Burlingame, C. A. von Muhlen, T. D. Jackowski, C. M. Litwin, and H. R. Hill. 2004. Evaluation of multiplexed fluorescent microsphere immunoassay for detection of autoantibodies to nuclear antigens. Clin. Diagn. Lab. Immunol. 11:1054 1059. 20. Moens, H. J., and J. K. van der Korst. 1991. Computer-assisted diagnosis of rheumatic disorders. Semin. Arthritis Rheum. 21:156 169. 21. Moens, H. J., and J. K. van der Korst. 1992. Development and validation of a computer program using Bayes s theorem to support diagnosis of rheumatic disorders. Ann. Rheum. Dis. 51:266 271. 22. Newcombe, R. G. 1998. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat. Med. 17:857 872. 23. Perilloux, B., A. K. Shett, L. E. Leiva, and A. Gedalia. 2000. Antinuclear antibody (ANA) and ANA profile tests in children with autoimmune disorders: a retrospective study. Clin. Rheumatol. 19:200 203. 24. Rajimeh, R., S. Farsiu, and L. M. Kouhsari. 2002. Prediction of lupus nephritis in patients with systemic lupus erythematosus using artificial neural networks. Lupus 11:485 492. 25. Robinson, W. H., C. DiGennaro, W. Hueber, B. B. Haab, M. Kamachi, E. J. Dean, S. Fournel, D. Fong, M. C. Genovese, H. E. de Vegvar, K. Skriner, D. L. Hirschberg, R. I. Morris, S. Muller, G. J. Pruijn, W. J. van Venrooij, J. S. Smolen, P. O. Brown, L. Steinman, and P. J. Utz. 2002. Autoantigen microarrays for multiplex characterization of autoantibody responses. Nat. Med. 8:295 301. 26. Rouquette, A.-M., C. Desgruelles, and P. Laroche. 2003. Evaluation of the new multiplexed immunoassay, FIDIS, for simultaneous quantitative determination of antinuclear antibodies and comparison with conventional methods. Am. J. Clin. Pathol. 120:676 681. 27. Schewe, S., and M. A. Schreiber. 1993. Stepwise development of a clinical expert system in rheumatology. J. Clin. Investig. 71:139 144. 28. Shovman, O., B. Gilburd, O. Barzilai, E. Shinar, B. Larida, G. Zandman- Goddard, S. R. Binder, and Y. Shoenfeld. 2005. Evaluation of the Bio- Plex 2200 ANA screen. Analysis of 510 healthy subjects: incidence of natural/predictive autoantibodies. Ann. N.Y. Acad. Sci. 1050:380 388. 29. Simel, D. L., G. P. Samsa, and D. B. Matchar. 1991. Likelihood ratios with confidence: sample size estimation for diagnostic test studies. J. Clin. Epidemiol. 44:763 770. 30. Slater, C. A., R. B. Davis, and R. H. Shmerling. 1996. Antinuclear antibody testing. Arch. Intern. Med. 156:1421 1425. 31. Sokal, R. R., and F. J. Rohlf. 1995. Biometry: the principles and practice of statistics in biological research, 3rd ed. W. H. Freeman, New York, N.Y. 32. Solomon, D. H., A. J. Kavanaugh, and P. H. Schur. 2002. Evidence-based guidelines for the use of immunologic tests: antinuclear antibody testing. Arthritis Rheum. 47:434 444. 33. Subcommittee for Scleroderma Classification of the American Rheumatism Association Diagnostic and Therapeutic Criteria Committee. 1980. Preliminary criteria for the classification of systemic sclerosis (scleroderma). Arthitis Rheum. 23:581 590. 34. Swartz, J. B., S. J. Rothenberg, S. Teklehaimanot, and F. Khan. 1999. Comparison of the entropy technique with two other techniques for detecting disease clustering using data from children with high blood lead levels. Am. J. Epidemiol. 149:750 760. 35. Tan, E. M., A. S. Cohen, J. F. Fries, A. T. Masi, D. J. McShane, N. F. Rothfield, J. G. Schaller, N. Talal, and R. J. Winchester. 1982. The 1982 revised criteria for the evaluation of systemic lupus erythematosus. Arthritis Rheum. 25:1271 1277. 36. Tanimoto, K., K. Nakano, and S. Kano. 1995. Classification for polymyositis and dermatomyositis. J. Rheumatol. 22:668 674. 37. Tonuttia, E., D. Bassetti, A. Piazza, D. Visentini, M. Poletto, F. Bassetto, P. Caciagli, D. Villalta, R. Tozzoli, and N. Bizzaro. 2004. Diagnostic accuracy of ELISA methods as an alternative screening test to indirect immunofluorescence for the detection of antinuclear antibodies. Evaluation of five commercial kits. Autoimmunity 37:171 176. 38. Vitali, C., S. Bombardieri, R. Jonsson, H. M. Moutsopoulos, E. L. Alexander, S. E. Carsons, T. E. Daniels, P. C. Fox, R. I. Fox, S. S. Kassan, S. R. Pillemer, N. Talal, M. H. Weisman, and the European Study Group on Classification Criteria for Sjogren s Syndrome. 2002. Classification for Sjögren s syndrome: a revised version of the European criteria proposed by the European-American Consensus Group. Ann. Rheum. Dis. 61:554 558. 39. Webb, A. R. 2002. Statistical pattern recognition, 2nd ed. Wiley, Chichester, United Kingdom. 40. Wyns, B., S. Sette, L. Boullart, D. Baeten, I. E. Hoffman, and F. De Keyser. 2004. Prediction of diagnosis in patients with early arthritis using a combined Kohonen mapping and instance-based evaluation criterion. Artif. Intell. Med. 31:45 55. 41. Zlatkis, A., K. Y. Lee, C. F. Poole, and G. Holzer. 1979. Capillary column gas chromatographic profile analysis of volatile compounds in sera of normal and virus-infected patients. J. Chromatogr. 163:125 133.