Determination of the Optimal Cutoff Value for a Serological Assay: an Example Using the Johne's Absorbed EIA

Transcription

1 JOURNAL OF CLINICAL MICROBIOLOGY, May 1993, p /93/ $02.00/0 Copyright ) 1993, American Society for Microbiology Vol. 31, No. 5 Determination of the Optimal Cutoff Value for a Serological Assay: an Example Using the Johne's Absorbed EIA SALLY E. RIDGE`* AND ANDREW L. VIZARD2 Victorian Institute ofanimal Science, Department ofagriculture, Mickleham Road, Attwood, Victoria 3049,1 and Veterinary Clinical Centre, University of Melbourne, Werribee, Victoria 3030,2 Australia Received 24 September 1992/Accepted 10 February 1993 Traditionally, in order to improve diagnostic accuracy, existing tests have been replaced with newly developed diagnostic tests with superior sensitivity and specificity. However, it is possible to improve existing tests by altering the cutoff value chosen to distinguish infected individuals from uninfected individuals. This paper uses data obtained from an investigation of the operating characteristics of the Johne's Absorbed EIA to demonstrate a method of determining a preferred cutoff value from several potentially useful cutoff settings. A method of determining the financial gain from using the preferred rather than the current cutoff value and a decision analysis method to assist in determining the optimal cutoff value when critical population parameters are not known with certainty are demonstrated. The results of this study indicate that the currently recommended cutoff value for the Johne's Absorbed EIA is only close to optimal when the disease prevalence is very low and false-positive test results are deemed to be very costly. In other situations, there were considerable financial advantages to using cutoff values calculated to maximize the benefit of testing. It is probable that the current cutoff values for other diagnostic tests may not be the most appropriate for every testing situation. This paper offers methods for identifying the cutoff value that maximizes the benefit of medical and veterinary diagnostic tests. No diagnostic test exhibits perfect operating characteristics under all conditions. The traditional approach to improve the accuracy of diagnosis is to develop new tests with superior sensitivity and specificity. An alternative, and rarely used approach, is to improve existing tests by altering the cutoff value that distinguishes infected individuals from uninfected individuals. For any given serological test, sensitivity and specificity are determined by the cutoff value. Ideally, the cutoff value of a test should be chosen to maximize the benefit that accrues from testing a population. This is equally true in human and veterinary medicine. To determine the benefit of testing, the economic and social consequences of misdiagnoses and the prevalence of the disease in the population must be considered. Invariably, cutoff values for diagnostic tests have been determined by arbitrary methods that fail to consider these issues. For example, a survey (by title) of papers published during 1991 in the Journal of Clinical Microbiology revealed 21 articles describing the development, evaluation, or improvement of enzyme-linked immunoassays (ELISA). In 12 (6, 8, 10, 12-14, 19, 20, 23, 24, 29, 32) of the 21 articles, the cutoff value was based on an arbitrary statistic, such as 3 standard deviations above the mean of the negative controls. Four papers (7, 17, 26, 30) used an optical density (OD) value as the cutoff value without any reference to how it was derived. One article (21) chose a cutoff value to optimize both the positive and negative predictive value of the test result without considering the effect of disease prevalence on predictive values, and four (3, 9, 11, 28) used a receiver operating characteristic (ROC) curve or some other method to optimize sensitivity and/or specificity. In short, none of the 21 papers demonstrated that the chosen cutoff value was likely to maximize the benefit of * Corresponding author testing and in all cases there may be opportunity to improve the test by changing the cutoff value. In this study we describe a method to optimize the operating characteristics of a serological test by (i) estimating cutoff values that maximize the economic benefit of using the test for various probabilities of disease, (ii) estimating the cost of continued use of the currently recommended cutoff compared with the cutoffs that maximize the economic benefit of testing as determined in (i), and (iii) describing a method to estimate the cutoff value of a test which maximizes the expected monetary value of testing when the probability of disease is not known with certainty. The serological test that is used as an example is an ELISA for the diagnosis of paratuberculosis in cattle, but the methodology is suggested as being applicable to many medical and veterinary diagnostic tests. MATERIALS AND METHODS Paratuberculosis is a chronic granulomatous enteritis of ruminants, caused by Mycobacterium paratuberculosis. The disease has worldwide distribution and causes considerable production losses due to both clinical and subclinical disease. Control and eradication of the disease from infected farms and regions are severely hindered by the inability of all current diagnostic tests to accurately identify subclinically infected animals (2). An ELISA (the Johne's Absorbed EIA [Commonwealth Serum Laboratories, Parkville, Victoria]) has been recently developed for the diagnosis of M. paratuberculosis infection in cattle (5). The cutoff value for the test that is recommended is 0.1 absorbance unit above the mean OD value for a negative control serum, tested in duplicate, on the same plate as the test samples. No justification or supporting evidence was advanced for the selection of this value. Animals. Serum samples from three groups of cattle were used in this study.

2 VOL. 31, 1993 DETERMINATION OF OPTIMAL CUTOFF FOR SEROLOGICAL ASSAY 1257 The first group of 136 samples were from animals slaughtered under the Victorian Cattle Compensation Scheme between 1986 and 1990 because they exhibited clinical signs consistent with paratuberculosis and for which the diagnosis was confirmed by histopathological examination of ileocecal tissues collected at slaughter. The samples were collected during the clinical phase of disease in these animals. The second group of sera were collected at slaughter from 998 Western Australian cattle. While there were no fecal culture or histopathological details available for these animals, the local animal health authorities in Western Australia believe the State to be free of paratuberculosis and have maintained surveillance and an active program to prevent the introduction or spread of the disease. On this basis, these animals were regarded as paratuberculosis free. A further 164 serum samples were obtained from subclinically infected cattle: 48 from an Australian collection and 116 from samples collected as part of the U.S. National Repository for Paratuberculosis Specimens. A detailed description of these cattle, the determination of their disease status, and the methods of sample collection have been described in earlier papers (22, 25). Briefly, the Australian samples were collected from animals in three endemically paratuberculosis-infected dairy herds. The results of fecal culture, complement fixation test, and in some cases postmortem histopathology were available for individual animals. Samples from animals that had remained in the herd that showed no clinical signs of paratuberculosis but had a positive fecal culture and/or histopathological or bacteriological evidence of infection on necropsy were used in this study. Subsamples of those held in the U.S. National Repository were included in this study if the cattle were clinically normal at the time of sampling but M. paratuberculosis had been cultured from their feces by either conventional or radiometric culture methods. Vaccinated cattle were excluded. Johne's Absorbed ELA. Assays were performed by using the commercial kits according to the manufacturer's directions. Negative, low-positive, and high-positive control sera supplied with each kit were included on every plate. Samples were tested in duplicate. Those with greater than 30% discrepancy between wells or that had an OD value greater than 0.1 were repeated in duplicate. The OD at 450 nm (OD450) values for each sample were obtained, and the mean value of the replicated samples was calculated. Estimation of cutoff values that maximize the benefit of testing. Preferred cutoff values were estimated by the method described by Anderson (1) in which the preferred cutoff value of a test is the value of c that minimizes (1-P) fc' d, (x) d-x + rp f_ -.c d2(x where r is the cost of a false-negative result/the cost of a false-positive result,p is disease prevalence, d1 is the density distribution of test values from uninfected individuals, and d2 is the density distribution of test values from infected individuals. The equation above may also be expressed in terms of sensitivity (Se) and specificity (Sp) such that the preferred cutoff value is the value that minimizes (1 - p)(l - Sp) + rp(l - Se). In this analysis, it was assumed that the population distributions were identical to the sample distributions. An algorithm was developed on a commercial spreadsheet program (Lotus 123, version 2.0) to perform the necessary calculations. The algorithm was limited to the examination &9c of 16 arbitrarily selected cutoff values (0.05, 0.06, 0.07, 0.08, 0.09, 0.10, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.19, 0.22, and 0.25 absorbance units). The benefit of testing was calculated for each of these cutoff values for a given prevalence, proportion of infected animals displaying clinical signs of disease, and relative cost of misdiagnosis (r). The cutoff value that minimized the above equation was identified and defined as the preferred cutoff value for that prevalence, proportion of infected animals displaying clinical signs, and relative cost of misdiagnosis. Calculations were repeated for 1,064 combinations of prevalence and r. Estimating the additional benefit of using the preferred cutoff value instead of the currently recommended cutoff value. The additional benefit from using the preferred cutoff value rather than the currently recommended cutoff value was estimated by subtracting the absolute benefit of testing 100 cows by using the currently recommended cutoff value from the absolute benefit of testing the same herd by using the preferred cutoff value. Absolute benefits were determined by assigning -$80.00 as the value of a false-positive result. This represents the cost of replacement of the animal minus the average price paid for animals with subclinical M. paratuberculosis infection under the Victorian Cattle Compensation Scheme in 1990 to The value of a false-negative result was determined by multiplying -$80.00 by r. These figure were incorporated in the algorithm described above. Calculations were repeated for 1,064 combinations of prevalence and relative cost of misdiagnosis. Estimation of the preferred cutoff value when the probability of disease is not known with certainty. The calculations given above assume that the probability of infection (prevalence) is well defined. The "best-option" cutoff values for the test were estimated by using decision analysis (18) for two circumstances in which the probability of infection was not known with certainty. The first situation was a herd of cattle for which the probability of infection was expected to be low but was unknown. The second situation was a herd for which the prevalence of infection was expected to be high but was undefined. A decision tree was mapped for each example using five cutoff values and a false-negative/falsepositive ratio of 1.0. Disease prevalence and probability values were selected to reflect the situation described. Monetary values were calculated for each cutoff value by using the method described above. RESULTS Estimation of cutoff values that maximize the benefit of testing. The preferred cutoff values for 1,064 combinations of prevalence and relative cost of misdiagnosis are displayed as a decision surface in Fig. 1. Only 5 of the 16 candidate cutoff values (OD450 values of 0.05, 0.06, 0.07, 0.13, and 0.25) were identified as preferred cutoff values for the range of relative costs and prevalences that were tested. None of these values correspond to the usual placement of cutoff values currently recommended (about to 0.15). The preferred cutoff value was extremely sensitive to changes in both the disease prevalence and the relative cost of misdiagnosis. Estimating the additional benefit of using the preferred cutoff value instead of the currently recommended cutoff value. Table 1 summarizes the additional benefit of using the preferred cutoff value for 50 of the 1,064 combinations of prevalence and relative cost of misdiagnosis. The greatest benefit of using the preferred cutoff value rather than the recommended cutoff value occurred when there was high

3 1258 RIDGE AND VIZARD J. CLIN. MICROBIOL :i1.0 0~~~~~~~~~00 0~~~~~ I \. X., I III,,,, I, IZ.,I,, True Prevalence of Paratuberculosis FIG. 1. Decision surface showing the preferred cutoff value for the Johne's Absorbed EIA (1% clinical cases) for prevalence ranging from 0 to 55% and relative cost of misdiagnosis ranging from 0.1 to 10. FN, false negative; FP, false positive. prevalence of disease and when false-negative test results were considered much more costly than false-positive test results. Estimation of the best-option cutoff value when the probability of disease is not known with certainty. The best-option cutoff value for the situation of unknown but probably low prevalence was calculated to be Figure 2 displays this decision tree. The best-option cutoff value when there was an unknown but probably high disease prevalence was an OD450 value of Changes in the relative cost of misdiagnosis significantly altered the outcome of the analysis for each situation examined. DISCUSSION Several methods have been employed to determine the cutoff value for serological assays with continuous or ordinal test result ranges. They can be grouped into five categories: (i) arbitrary methods, (ii) methods to optimize sensitivity or TABLE 1. Additional benefit of using the preferred rather than the recommended cutoff value for the Johne's Absorbed EIA for five relative costs of misdiagnosis and 10 disease prevalences Disease Relative cost of misdiagnosis prevalence a , ,680 1, ,030 1, ,381 2, a Cost in dollars per 100 cattle tested. Cut-off Prevalence Probability Benefit (0.8) ) 0.1) $ (0.05) ) ) ) $ (0.05) (0.05) ) ) ) 0.0 ~<0.38 (0.05) ) (0.1) 0.0 -$ (0.05) (0.05) ) 0.0 X ) $ (0.05) (0.05) FIG. 2. Decision tree for the determination of the optimal cutoff value for the Johne's Absorbed EIA when prevalence of paratuberculosis is not known with certainty but is believed to be low (in a herd of 100 cows).

4 VOL. 31, 1993 DETERMINATION OF OPTIMAL CUTOFF FOR SEROLOGICAL ASSAY 1259 specificity, (iii) methods to optimize test accuracy, (iv) ROC curves, and (v) methods to optimize the predictive value. Each has limitations. Arbitrary methods include the recommendation of a subjective cutoff value or the use of a statistical parameter (such as 3 standard deviations above the mean for negative control) to set the cutoff value. These procedures fail to consider test sensitivity and specificity and the number or costs of misdiagnoses produced by the test settings. In addition, while the statistics employed may have been applicable to the original population used to evaluate the test, they may not have any relevance in the populations to be tested. Determining cutoff values by optimizing either test sensitivity alone or test specificity alone will do so at the expense of the other parameter (15). Such methods also fail to examine the cost incurred from misdiagnoses or the effect of disease prevalence on the frequency of false-positive or false-negative test results. Some researchers have defined the optimal cutoff as the value which maximizes the total number of correct diagnoses in the experimental population. This method fails to recognize that test accuracy is a function of disease prevalence (27) and it will produce a cutoff value that maximizes test accuracy only for populations with the same prevalence as the experimental population. Even if the prevalence of disease is accounted for, this method fails to consider the economic and social costs associated with misdiagnoses. The costs incurred by incorrectly classifying an uninfected individual as infected (false positive) may be different from the costs incurred by incorrectly classifying an infected individual as uninfected (false negative). Therefore, minimizing the total number of misdiagnoses may not minimize the total cost associated with these misdiagnoses. ROC curves graphically illustrate the effect of changing the cutoff value on test sensitivity and specificity (16). If the curve lies above the major diagonal of the graph (bottom left to top right), the test provides useful diagnostic information. ROC curves have been used to select cutoff values for numerous tests by identifying the point which is maximally distant from the diagonal. However, this cutoff value maximizes the benefit of testing only when disease prevalence is 50% and each type of misdiagnosis is equally costly (r = 1) (4) or for a single and unknown r value (of the infinite possible r values) for any other given prevalence. Using predictive values (positive or negative) to set a cutoff value for diagnostic tests maximizes the number of true-positive or true-negative test results. Predictive values are sensitive to changes in disease prevalence (15), so that the cutoff value selected for one population may be inappropriate for another. Once again, no consideration is given to the costs associated with each type of misdiagnosis. The process outlined in this report provides a method for determining the optimal cutoff value for a test that takes into consideration disease prevalence and the costs associated with misdiagnosis. To determine the optimum cutoff value of a test by this method, assumptions must be made about the distribution of test values in the uninfected and infected populations to which the test is to be applied. In this study it was assumed that the population distributions of ELISA OD values were identical to the sample distributions. Optimum cutoff values are highly dependent on the assumptions made regarding the distribution of test values in the infected and uninfected populations (31). There is often insufficient information to define the parent density distributions adequately, especially in the tails of the distribution. Decisions regarding cutoffvalues for diagnostic tests should be reviewed as more data become available. An estimation must also be made about the cost of a false-negative diagnosis relative to the cost of a falsepositive diagnosis. Often this estimation can be made in veterinary medicine on a purely economic basis, but this is not so in medical diagnosis. Weighing the social, physical, and psychological costs of misdiagnosis is not easy. The difficulty in determining the relative cost of misdiagnosis may lead some clinicians to believe that this method is inappropriate for determining the cutoff value for medical tests. However, we contend that consciously or unconsciously, these assessments are being made whenever cutoff points are being set. An advantage of the method outlined is that it brings into the open the hidden assumption of the relative cost of misdiagnosis implicit in any calculation of a preferred cutoff value. The algorithm used to determine the preferred cutoff value was limited to the examination of only 16 possible cutoff values. The preferred cutoff value selected for each combination of prevalence and r was the best of these 16 but may not have been the optimum cutoff value if all possible cutoff values were examined. However, the difference between adjacent possible cutoff values (generally 0.01 OD unit) was close to the precision limits of the ELISA testing system. Since the sample distributions of the uninfected and infected individuals were not uniformly smooth, only five preferred cutoff values were identified. The two extreme preferred cutoff values, 0.05 and 0.25, represent the points that provide nearly perfect sensitivity and perfect specificity, respectively. They have limited value diagnostically. The intermediate points represent cutoff values for which small changes in the number of animals classified as positive resulted in large changes in the test sensitivity or specificity. In the example shown, the preferred cutoff value was similar to the manufacturer's recommended cutoff value when disease prevalence was low and/or when the cost of a false-positive misdiagnosis was far greater than the cost of a false-negative misdiagnosis. This situation could arise, for example, in preexport testing of cattle by exporters. The preferred cutoff value was very different from the manufacturer's recommended cutoff value when disease prevalence was high and the cost of a false-negative result was far greater than the cost of a false-positive diagnosis. This situation would arise if the test were used to control paratuberculosis in a heavily infected herd. Using the preferred cutoff value instead of the manufacturer's recommendation in such situations resulted in large financial gains. For example, it was estimated (Table 1) that in a herd of 100 cows with 30% prevalence of paratuberculosis there was a $2, advantage in using the preferred cutoff value when false negatives were valued at -$ and false positives were valued at -$ The monetary value assigned to the cost of a false-positive test result may vary considerably and can be calculated simplistically, as in this example, or in more detail, to allow for lost genetic material, restricted access to markets, and other sources of loss. There are considerable benefits associated with standardized diagnostic tests, particularly in relationship to export testing, animal health regulation, and disease control. However, as this study demonstrates there are considerable advantages to tailoring cutoff values to suit the specific requirements of the testing situation. For example, the cutoff value for the Johne's Absorbed EIA could be changed to a value in the range 0.07 to 0.13 for almost all routine diagnostic testing and for herds for which a planned eradication

5 1260 RIDGE AND VIZARD program is undertaken. The current cutoff value (0.15) or another somewhat higher value is appropriate for preexport testing by exporters to ensure that very few false-positive diagnoses are made. If a clinician believed that the prevalence of disease in the population being tested was about 25% and that a falsenegative result was as costly as a false-positive result (r = 1), then an examination of the decision surface shown in Fig. 1 indicates that the preferred cutoff value is However, in situations in which the probability of disease is not known with any certainty, it is difficult to use the decision surface shown in Fig. 1 or other normal cost-benefit analysis techniques to determine the preferred cutoff value. Other analytical methods such as decision analysis are available to assist researchers making decisions about test cutoff values under conditions of uncertainty about disease prevalence or cost factors. Even though some of the data used for decision analysis may be quite imprecise, the resultant cutoff values should be more reliable than purely arbitrarily derived figures. The decision analysis results in this study indicate that the best-option cutoff values for two commonly encountered situations (prevalence unknown but probably low and prevalence unknown but probably high) were lower than the manufacturer's recommended cutoff value as well as being quite different from each other. In summary, the results of this study indicate that the currently recommended cutoff value for the Johne's absorbed EIA is close to optimum only when the disease prevalence is very low and false-positive test results are deemed to be very costly. In other situations, there were considerable financial advantages in using cutoff values calculated to maximize the benefit of testing. It is probable that the current cutoff values for other diagnostic tests may not be the most appropriate for every testing situation. This paper offers a method for identifying the cutoff value that maximizes the benefit of testing for any testing situation. REFERENCES 1. Anderson, T. W An introduction to multivariate statistical analysis, p John Wiley & Sons, Inc., New York. 2. Blood, D. C., 0. M. Radostits, and J. A. Henderson Veterinary medicine: a textbook of the diseases of cattle, sheep, pigs, goats, and horses, 6th ed., p Bailliere Tindall, London. 3. Castillo, R. M., P. Grados, C. Carcamo, E. Miranda, T. Montenegro, A. Guevara, and R. H. Gilman Effect of treatment on serum antibody to Hymenolepis nana detected by enzyme-linked immunosorbent assay. J. Clin. Microbiol. 29: Connell, F. A., and T. D. Koepsell Measures of gain in certainty from a diagnostic test. Am. J. Epidemiol. 121: Cox, J. C., D. P. Drane, S. L. Jones, S. E. Ridge, and A. R. Milner Development and evaluation of a rapid enzyme immunoassay test for the diagnosis of Johne's disease in cattle. Aust. Vet. J. 68: Decoster, A., B. Slizewicz, J. Simon, C. Bazin, F. Darcy, G. Vittu, C. Boulanger, Y. Champeau, J. L. Demory, M. Duhamel, and A. Capron Platelia-Toxo IgA, a new kit for early diagnosis of congenital toxoplasmosis by detection of anti-p30 immunoglobin A antibodies. J. Clin. Microbiol. 29: DiPersio, J. R., F. J. Varga, D. L. Conwell, J. A. Kraft, K. J. Kozak, and D. H. Willis Development of a rapid enzyme immunoassay for Clostndium difficile toxin A and its use in the diagnosis of C. difficile-associated disease. J. Clin. Microbiol. 29: Erdman, D. D., L. J. Anderson, D. R. Adams, J. A. Stewart, L. E. Markowitz, and W. J. Bellini Evaluation of monoclonal antibody-based capture enzyme immunoassays for detec- J. CLIN. MICROBIOL. tion of specific antibodies to measles virus. J. Clin. Microbiol. 29: Gilbert, M., J. Kirihara, and J. Mills Enzyme-linked immunoassay for human immunodeficiency virus type 1 envelope glycoprotein 120. J. Clin. Microbiol. 29: Gonik, B., M. Seibel, A. Berkowitz, M. B. Woodin, and K. Mills Comparison of two enzyme-linked immunoassays for detection of herpes simplex virus antigen. J. Clin. Microbiol. 29: Hansen, K., K. Pii, and A.-M. Lebech Improved immunoglobin M serodiagnosis in Lyme borreliosis by using a,u-capture enzyme-linked immunosorbent assay with biotinylated Borrelia burgdorferi flagella. J. Clin. Microbiol. 29: Hartley, T. M., G. E. Malone, R. F. Khabbaz, R. B. Lal, and J. E. Kaplan Evaluation of a recombinant human T-cell lymphotrophic virus type 1 (HTLV-1) p21e antibody detection enzyme immunoassay as a supplementary test in HTLV-I/II antibody testing algorithms. J. Clin. Microbiol. 29: Knowles, D. P., Jr., L. E. Perryman, L. S. Kappmeyer, and S. G. Hennager Detection of equine antibody to Babesia equi merozoite proteins by a monoclonal antibody-based competitive inhibition enzyme-linked immunosorbent assay. J. Clin. Microbiol. 29: Lal, R. B., W. Heneine, D. L. Rudolph, W. B. Present, D. Hotbienz, T. M. Hartley, R. F. Khabbaz, and J. E. Kaplan Synthetic peptide-based immunoassays for distinguishing between human T-cell lymphotropic virus type 1 and type 2 infections in seropositive individuals. J. Clin. Microbiol. 29: Martin, S. W The evaluation of tests. Can. J. Comp. Med. 41: Metz, C. W Basic principles of ROC analysis. Semin. Nucl. Med. 8: Montagna, R. A., L. Papsidero, and B. J. Poiesz Evaluation of a solid-phase immunoassay for the simultaneous detection of antibodies to human immunodeficiency virus type 1 and human lymphotropic virus type 1. J. Clin. Microbiol. 29: Ngategize, P. K., J. B. Kaneene, S. B. Harsh, P. C. Bartlett, and E. L. Mather Decision analysis in animal health programs: merits and limitations. Prev. Vet. Med. 4: Nielsen, R., T. Plambeck, and N. T. Foged Blocking enzyme-linked immunosorbent assay for detection of antibodies to Actinobacillus pleuropneumoniae serotype 2. J. Clin. Microbiol. 29: Olson, J. G., T. W. Scott, L. H. Lorenz, and J. H. Hubbard Enzyme immunoassay for detection of antibodies against eastern equine encephalomyelitis virus in sentinel chickens. J. Clin. Microbiol. 29: Paerragaard, A., G. H. Shand, K. Gaarslev, and F. Epersen Comparison of crossed immunoelectrophoresis, enzymelinked immunosorbent assays, and tube agglutination for serodiagnosis of Yersinia enterocolitica serotype 0:3 infection. J. Clin. Microbiol. 29: Ridge, S. E., I. R. Morgan, D. C. Sockett, M. T. Collins, R. J. Condron, N. W. Skilbeck, and J. J. Webber Comparison of the Johne's absorbed EIA and the complement-fixation test for the diagnosis of Johne's disease in cattle. Aust. Vet. J. 68: Sandin, R. L., C. C. Knapp, G. S. Hall, J. A. Washington, and I. Rutherford Comparison of the Vitek immunodiagnostic assay system with an indirect immunoassay (Toxostat test kit) for detection of immunoglobin G antibodies to Toxoplasma gondii in clinical specimens. J. Clin. Microbiol. 29: Schwarz, T. F., S. Modrow, B. Hottentrager, B. Hoflacher, G. Jager, W. Schartl, R. Sumazki, H. Wolf, J. Middeldorp, M. Roggendorf, and F. Deinhardt New oligopeptide immunoglobin G test for human parvovirus B19 antibodies. J. Clin. Microbiol. 29: Sockett, D. C., D. J. Carr, W. D. Richards, and M. T. Collins A repository of specimens for comparison of diagnostic testing procedures for bovine paratuberculosis. J. Vet. Diagn. Invest. 4:

6 VOL. 31, 1993 DETERMINATION OF OPTIMAL CUTOFF FOR SEROLOGICAL ASSAY Toth, I., T. J. Barrett, M. L. Cohen, H. S. Rumschlag, J. H. Green, and I. K. Wachsmuth Enzyme-linked immunosorbent assay for products of the 60-megadalton plasmid of Escherichia coli serotype 0157:H7. J. Clin. Microbiol. 29: Trajstman, A. C Diagnostic tests, sensitivity, specificity, efficiency and prevalence. Aust. Vet. J. 55: van den Oever, H. L. A., R. J. L. F. Loffeld, and E. E. Stobberingh Usefulness of a new serological test (Bio- Rad) to diagnose Helicobacter pylon-associated gastritis. J. Clin. Microbiol. 29: van Oirschot, J. T Comparative evaluation of an enzyme-linked immunosorbent assay (ELISA) to detect antibodies directed against glycoprotein 1 of pseudorabies virus and a conventional ELISA and neutralization tests. J. Clin. Microbiol. 29: Verstinen, C. P. H. J., H. M. Ly, K. Polman, C. Richter, S. P. Smits, S. Y. Maselle, P. Peerbooms, D. Rienthong, N. Montreewasuwat, S. Koanjanart, D. D. Trach, S. Ku"per, and A. H. J. Kollk Enzyme-linked immunosorbent assay using monoclonal antibodies for identification of mycobacteria from early cultures. J. Clin. Microbiol. 29: Vizard, A. L., G. A. Anderson, and R. B. Gasser Determination of the optimum cut-off value of a diagnostic test. Prev. Vet. Med. 10: Zoller, A., I. Kramer, R. Kappe, and H. G. Sonntag Enzyme immunoassays for invasive Candida infection: reactivity of somatic antigens of Candida albicans. J. Clin. Microbiol. 29: