Vol. 51, No.2, February 1989 Copyright The American Fertility Society

Transcription

1 FERTILITY AND STERILITY Vol. 51, No.2, February 1989 Copyright The American Fertility Society Printed in U.S.A. Analysis of three laboratory tests used in the evaluation of male fertility: Bayes' rule applied to the postcoital test, the in vitro mucus migration test, and the zona-free hamster egg test Francis F. Polansky, M.D. Emmet J. Lamb, M.D. Department of Gynecology and Obstetrics, Stanford University School of Medicine, Stanford, California To introduce the reader to the application of Bayes' rule in the interpretation of laboratory tests, this review will discuss some basic ideas about probability, the symbols we use to express these ideas, and several characteristics of laboratory tests such as sensitivity and specificity. It will explain the use of odds and likelihood ratios in revising estimates of the probability of the occurrence of an event. Finally, in this review, we will demonstrate the application of Bayes' rule to revise clinical estimates of the likelihood of fertility, using as examples three tests that have become widely used as supplements to the standard semen analysis in the clinical investigation of the male factor in human infertility. We address first the postcoital test, secondly, the in vitro mucus penetration test, and finally, the zona-free hamster egg penetration test. This report is not intended to be a comprehensive review of methods for these andrology tests nor of their clinical interpretation. The studies cited were selected because they provided data that could be used in examples of the application of Bayes' rule to problems in clinical infertility practice. Several reviews of the tests themselves have been published in this journal in the past. 1-3 BASIC SCIENCE BACKGROUND Terms and Symbols In this review, we use the following terminology and symbols. The probability of an event occurvol. 51, No.2, February 1989 ring,p(e+), is a number between 0 and 1 indicating how likely the event is to occur. By using statements of probability rather than expressions such as probably or possibly, the clinician expresses uricertainty quantitatively and unambiguously. 4 We chose the symbol E to signify the occurrence of an event in order to reduce the confusion when we later talk of pregnancy or the result of another laboratory test as the event in question. In all instances in which we refer to a positive test (T+), we mean that result that predicts that the event would be expected to occur. For example, in discussing postcoital tests, the event is pregnancy and a positive test is one in which more than a certain number of motile sperm are found. A conditional probability indicates the probability of one event given the fact that a second event has occurred. For instance, p(e+ IT+) denotes the probability of event E given that the event T has occurred. By definition: p(e+) is the probability of presence of an event; p(e-) is the probability of absence of an event; p(e+ IT+) is the probability of an event given that a test is positive; p(e+ IT-) is the probability of event given that a test is negative; p(t+ IE+) is the probability that a test is positive given that an event occurs (sensitivity); p(t+ IE-) is the probability that a test is positive given that an event does not occur (false positive rate); p(t-1 E+) is the probability that a test is negative given that an event occurs Polansky et al. Bayes' rule applied to male fertility tests 215

2 (false negative rate); p(t-1 E-) is the probability that a test is negative given that an event does not occur (specificity). In 1763, the Reverend Thomas Bayes of England established the principles that have resulted in a system of probability revision. 5 He developed a rule for estimating the probability of an event occurring in the future based on knowledge of the prior probability of the occurrence of the event and some new information. One may think of Bayes' rule as a system for modifying historical information on a parameter in light of current data. The application of Bayes' rule most familiar to physicians is the interpretation of clinical laboratory results. Bayes' rule is used in this case for statistical inference from the pretest probability to the post-test probability of a disease. In the interpretation of laboratory tests, p(e+), the pretest probability of an event (as with a disease), termed the prior probability, is a probability that a patient to whom the test result will be applied has the specified disease. The pretest probability for a screening test is determined by the prevalence of a disease in the population screened. For a clinical test, the clinician determines the pretest probability for the individual patient by interpretation of information available from the history, physical examination, and previous laboratory tests. Test result Pretest probability ----L~ --+ Post-test probability Bayes' theorem often provides insights that are not obtained by relying on intuition. It is used in formal studies of clinical decision -making, and is of special importance for computer applications in this field. 6 7 The derivation of the formula from the basic rules of probability is available in standard texts, to which the interested reader is referred. 6 8 Reports of the application of Bayesian analysis in the domain of reproduction are becoming frequent enough to warrant this review. 9 The following are two forms of Bayes' theorem: Probability of an event given a positive test: p(t+ IE+). p(e+) p(e+ IT+) = p(t+ IE+). p(e+) + p(t+ I E-). p(e-) Probability of an event given a negative test: + p(t-1 E+) p(e+) p(e IT ) - p(t-1 E+). p(e+) + p(t-1 E-). p(e-) Characteristics of Laboratory Tests Assume that we use a very accurate method, a gold standard procedure such as biopsy or long- TEST TP FN TP+FN EVENT FP TN FP+TN SENSITIVITY = p (T"!E+) ~ TP+FN SPECIFICITY = p (ri Ei l!!_ FP+TN TP+FP FN+TN Figure 1 Definitions of sensitivity and specificity using a 2 X 2 contingency table. TP, true positive; FP, false positive; TN, true negative; FN, false negative. The marginal totals indicate the following: left column, total number of cases in which the event has occurred; right column, total number of cases in which the event has not occurred; first row, total number of cases with positive tests; second row, total number of cases with negative tests. term follow-up to establish the true state of the patient, diseased/nondiseased, pregnant/nonpregnant, etc. Then we can construct a 2 X 2 contingency table of test results and status, such as that shown in Figure 1. True positives (TP) are patients in whom the disease is present and the test result is positive. True negatives (TN) are patients in whom the disease is absent and the test result is negative. False positives (FP) are patients in whom the disease is absent but the test result is positive. False negatives (FN) are patients in whom the disease is present but the test result is negative. We will use this contingency table to define the terms sensitivity and specificity. Sensitivity (true positive rate, TPR), the probability of a positive test result in a person with the event, p(t+ IE+), is calculated by dividing the number of true positives by the total number of occurrences of the event: TPR = TP/(TP + FN). Specificity (true negative rate, TNR) the probability of a negative test result in a person without the event, p(t-1 E-) is calculated by dividing the number of true negatives by the total number of people without the event: TNR = TN/(TN + FP). These probabilities are defined from the viewpoint of the event, down the columns in Figure 1. An ideal test has both sensitivity and specificity equal 216 Polansky et al. Bayes' rule applied to male fertility tests Fertility and Sterility

3 SENSITIVITY SPECIFICITY Figure 2 Distribution of test results among patients with a hypothetical disease, hicountosis, who have mainly higher values of test results, and in normals, who have mainly lower values of test results. The disease is common in this population, the pretest probability being equal to 0.50, as is illustrated by the equal areas under the two distribution curves. False positives and false negatives for a cutoff point Y are indicated by stippled areas. A second disease with a five times lower pretest probability is indicated by the hatched area. to 1.0, but in reality very few, if any, tests have these utopian characteristics. Often, tests with high sensitivity have medium or low specificity and tests with high specificity have medium or low sensitivity. Tests may be applied in situations in which the pretest probability of the disease is low (many screening tests), intermediate (considerable uncertainty about the patient's true state), or relatively high (patient strongly suspected of having the disease). We will discuss the effect of the pretest probability after we discuss a related concept, the likelihood ratio, since the ideas are more easily shown in this context. One can determine sensitivity and specificity for a test for which the observation is dichotomous, i.e., consists of merely a recording of whether the measured attribute is present or absent, e.g., pregnant or not. Sensitivity and specificity also can be calculated for a test in which a quantitative measurement of response is made: sperm concentration or percent motility. To be able to construct the 2 X 2 contingency table, using a quantitative response, one has to dichotomize or to reduce the number of results to two: a positive and a negative one, e.g., a low sperm count and a normal one. The selected value of the tested variable that differentiates between a positive and a negative test result is called the cutoff point. As shown in Figure 2, there usually is an overlap of values between those obtained in the group in whom the event has occurred, or will occur, and those obtained from those in whom it has not, or will not. In this figure, we illustrate hypothetical data in which the test is.70 sperm concentration and the event is the occurrence of the mythical disease, hicountosis, for which the frequency distribution of sperm counts is shown as E+ in the figure. The frequency distribution of sperm counts in the group without hicountosis is shown as E-. Since the disease is mythical, let us not be concerned with its manifestations. Moreover, we may simply state that the results of the gold standard test are supplied by a wizard. In this figure, setting the cutoff point in the region of overlap of the two symmetrical and equalsized distributions, line Y, strikes one compromise between sensitivity and specificity. A sensitivity of 90% indicates that if we were to apply the test to a group who all actually have the disease hicountosis (E+), we would mislabel % of them as normal. This is shown by the stippled area on the left. In the same way, if we were to apply this test, which has a specificity of 97%, to a group consisting entirely of normal people, we would mislabel 3% of them as diseased. This situation is indicated by the stippled area on the right in Figure 2. When we use a test in which the result values a:.;:e continuous rather than dichotomous, we may move the cutoff point to alter the criteria for a positive test. This approach is helpful both to exclude or to confirm the presence of a disease according to the purpose we have for obtaining the test. Suppose, for the sake of the illustration, that by setting the cutoff point at 0 X 6 /ml, once again line Y, one found that 3% of normal men had values above that point and that % of males actually suffering from hicountosis had values below that point. Moving the cutoff point to a lower count, to line Z, say, to 0 X 6 /ml, to avoid ever missing the diagnosis ofhicountosis, will result in a sensitivity of 1.0 (i.e., all diseased patients have a count greater than the cutoff). At the same time, the specificity will necessarily be lower, meaning that more normal males will be included in the abnormal test group. Setting the cutoff point to a higher count, to line X, 250 X 6 /ml, for example, will increase the specificity to 1.0, meaning that all normal males will have counts less than the cutoff point and be labeled "no disease" (E-). However, the sensitivity will decrease, causing us to misdiagnose some males with hicountosis (E+) as normal. It is evident that each cutoff point defines a corresponding specificity and sensitivity. As sensitivity is increased, specificity necessarily falls, and vice versa. A graphical method that illustrates the consequences of selecting alternative locations for cutoff points within the range of possible test results is Vol. 51, No.2, February 1989 Polansky et al. Bayes' rule applied to male fertility tests 217

4 1.0~ -~~c Tf'R p (T+I~l.5 X y -~~'C FPR p(t+ie! FPR P (T+Ie-, FPR P (T+Ie! Figure 3 Receiver operating characteristic curves, for three hypothetical tests. Each test has a different relationship of the distribution of test results among those with the event, E+, and among those without it, E-. The effect of changes in the location of the cutoff point separating those with a positive test result (T+) and those with a negative result (T-) in each of these situations is described in the text. TPR, true positive rate (sensitivity); FPR, false positive rate (1- specificity). called a receiver operator characteristic curve or ROC curve (Fig. 3). As the cutoff point is moved through the range of possible test results, the ROC curve shows the relationship between the true positive rate (sensitivity) and the false positive rate (1 - specificity). If the frequency distribution of test results is identical in the event and no-event populations, E+ and E-, as shown in Figure 3A, then the test would be worthless. The false positive rate would be the same as the true positive rate for all cutoff points. A straight line running at 45 degrees from point 0, the line of identity, represents this situation. If the distributions overlap (Fig. 3B), as we progressively lower the cutoff so that more test results in both populations are above the cutoff, then the path described on the ROC curve would sweep from the lower left upward and to the right. In general, as the positivity requirement for diagnosis is progressively relaxed, that is, as the cutoff point is moved down in the direction ofe-, the false positive rate is increased, specificity is reduced, and the point on the ROC curve moves toward the right. Conversely, as the positivity requirement is made more stringent, the true positive rate, or sensitivity, is reduced, and the point on the ROC curve moves downward. The further upward and to the left the ROC curve lies, the better is the test. The extreme example, Figure 3C, is one in which the distribution of test results do not overlap at all. A test result that falls into an overlap zone, such as near line Y in Figure 2, can be viewed as an extreme result both for members of thee+ population and for members of the E- population. Physicians often repeat the test when a result is extreme or unexpected. Here they depend on a phenomenon generally known as the regression toward the mean. Repeated measurements that are subject to imprecision tend to move toward the mean of the distribution to which the tested subject actually belongs. In interpreting the results of serial tests, one must keep in mind the possibility of regression toward the mean before inferring that a real trend in the clinical variable has been observed Revising Probability Estimates Although sensitivity and specificity are important characteristics of a test, they are difficult to use intuitively in clinical decision-making. A clinician using a diagnostic test to determine the likelihood of the presence of a disease must address the following two questions: (1) Given a positive test, 218 Polansky et al. Bayes' rule applied to male fertility tests Fertility and Sterility

5 what is the probability that the disease is present, p(e+ IT+)? (2) Given a negative test, what is the probability that the disease is not present, p(e-1 T-)? The clinician begins by estimating the pretest probability, p(e+), that this particular patient has the disease. This estimate is based initially on the physician's knowledge of the prevalence of disease in the group from which the patient is drawn. The estimate is modified by taking into account the results of the medical history and physical examination and the results of any tests done previously. Griner et al. maintain that physicians are more precise in estimating pretest probability of disease in patients than they think. 11 Knowing test results and having information about test characteristics, we can revise the pretest estimate of probability of disease and obtain an estimate of the post-test probability of disease for that particular patient. The following examples illustrate this process of probability revision: Positive Test Result p(e+) l p(e+ IT+) where p(e+ IT+) is the conditional probability of the occurrence of the event if the test is positive. Negative Test Result p(e-).!_.,.. p(e-1 T-) p(e-1 T-) is a conditional probability of the absence of the event if the test is negative. Although it is possible to calculate the probability of the occurrence of the event conditional on a positive test, p(e+ I T+), or the absence of the event conditional on a negative test, p(e-1 T-), directly from the 2 X 2 contingency table constructed from results of a study, the values thus computed will apply only to the group of subjects studied. This post-test probability, p(e+ IT+), is calculated with the formula TP /(TP + FP). The post-test probability, p(e-1 T-), is calculated with the formula TN/(FN + TN). To calculate more meaningful post-test probability estimates, we must start with knowledge of the pretest probability of the disease in the specific patient or in the ultimate target group of subjects to whom the test will be applied. The pretest probability in this target group may be quite different from the pretest probability among all subjects who were used in the initial test evaluation study. The characteristics of a new test are often determined by an investigator studying a group Vol. 51, No.2, February 1989 of subjects known by the use of the gold standard test to have the disease and another group, of controls known by application of the gold standard test to be free of the disease. If the groups are equal in size, as is often true in these studies, the pretest probability in the study population is 0.5, p(e+) = 0.5. Among people who will eventually be tested in the clinical situations, the pretest probability of the disease is often quite different. The shaded area in Figure 2 illustrates the situation in which the number with disease is much smaller than the number who are disease-free, and the pretest probability is low. We can see from analysis of this figure that the post-test probabilities depend, not only on test characteristics at the particular cutoff point selected, but also on the pretest probability of disease. Bayes' formula uses knowledge of the pretest probability and, thus, adjusts for differences in the pretest probability of disease between the study sample used to define test sensitivity and specificity and the pretest probability in the patient or group of patients to be tested. At this point it helps our understanding of how sensitivity and specificity relate to Bayes' rule to reformulate the rule by substituting the words we defined in Figure 1 into the previously given statement of Bayes' rule: sensitivity pretest probability sensitivity pretest probability + (1 -specificity) (1- pretest probability) Odds and Likelihood Ratios It is possible to combine the information contained in statements of a test's sensitivity and specificity to calculate two new measures of the discrimination value of a test which are called the likelihood ratio for a positive test (LR+) and the likelihood ratio for a negative test (LR-). After converting a pretest probability to pretest odds, we can use the likelihood ratio in a simple way to obtain the post-test odds and then, the post-test probability of an event. Before we further define likelihood ratios, let us give an example of their use. Start by changing the pretest probability of the event to pretest odds in favor of the event by the following relationship: (E+) (E+) Odds in favor of event = P(E = p + p -) 1-p(E ) To obtain the post-test odds in favor of the event Polansky et al. Bayes' rule applied to male fertility tests 219

6 for a person with a positive test result, multiply the pretest odds by the likelihood ratio for a positive test (LR+). Similarly, to obtain the post-test odds in favor of the event for a person with a negative test, multiply the pretest odds by LR-. For a positive test result: Post-test odds in favor of event =pretest odds in favor of event likelihood ratio for a positive test For a negative test result: Post-test odds in favor of event = pretest odds in favor of event likelihood ratio for a negative test Finally, transform the post-test odds back to probability using the formula: P ro b a b 1. 1ty. o f event = odds dd s Let us suppose that we estimate that a male has a pretest probability of 0.25 of having the mythical disease hicountosis. Suppose further, that we obtain a result in a diagnostic test for hicountosis for which we know the LR+ has previously been found to be 2.0. Pretest probability, p(e+) = 0.25 dd 0.25 Pretest o s = Post-test odds = pretest odds LR+ Post-test probability, p(e+ IT+) = = As a result of the test, thus, we revise the estimate of the probability that the man has a hicountosis from 0.25 to Having seen the use of likelihood ratios, let us see how they are derived. A likelihood ratio is the ratio of the probability of a test result (i) among those with the event to the probability of the same test result among those without the event. The likelihood ratio for a positive test is the rela- tive likelihood, ranging from 1.0 to infinity, of the occurrence of a positive test result among those with the event and those without it. For example, if LR+ = 2 and the event is the presence of disease, then a diseased person is twice as likely to have a positive test as is a disease-free one. The higher the value of LR+, the bigger the increase in the estimated probability of the event from pretest to posttest. An ideal test, one with both sensitivity and specificity equal to 1.0, would have an LR+ of infinity. To calculate the likelihood ratio for a positive test, use the formula below. The relationships to sensitivity and specificity are evident from the formulas given with Figure 1. LR+ = p(t+ IE+) = sensitivity p(t+ I E-) 1 - specificity The likelihood ratio for a negative test (LR-), the relative likelihood, ranging from 1.0 to 0.0, of the occurrence of a negative test result among those with the event and those without it, is calculated by the formula below. The lower the LR-, the larger the decrease in probability of disease from pretest to post-test. An ideal test, with both sensitivity and specificity equal to 1.0, will have LRequal to 0.0. LR- = p(t-1 E+) = 1 - sensitivity p(t-1 E-) specificity Likelihood ratios can be used to compare the relative merit of any two tests that could be applied in a clinical situation. Test A is more useful than test B, if at the same time the LR+ of test A is higher and the LR- of test A is lower than the corresponding ratios for test B. In other combinations of the ratios, it is not possible to determine unquestionably which test is more useful, and the choice between them may differ with at least two clinical circumstances. The first, the difference in the difficulty of performing or of repeating the test, can usually be reflected to a large extent by the dollar cost, but also may include consideration of comfort, convenience or other factors; the second, the difference in the penalties for false positive and/ or false negative results, is related primarily to the nature of the disease, but may differ greatly in individual cases. Likelihood ratios have the advantage of easy computation of post-test odds. Moreover, like sensitivity and specificity, they completely characterize the diagnostic implications of a given test outcome over the entire spectrum of pretest probabili- 2 Polansky et al. Bayes' rule applied to male fertility tests Fertility and Sterility

7 POST-TEST PROBABILITIES p<eit ) P<elr) PRETEST PROBABILITY p(e+) Figure 4 The relationship between the pretest probability and the post-test probability of the occurrence of an event for selected likelihood ratios ranging from zero to infinity_ ties. Figure 4, which shows the relation between the pretest and the post-test probability of an event for selected likelihood ratios between zero and infinity, graphically shows the application of these concepts. To use this graph, first estimate the pretest probability, then calculate the LR+ and LR-, using the sensitivity and the specificity of the test. Next, select the appropriate curves for LR+ and LR- in Figure 4 and read on the ordinate the posttest probability of the event corresponding to the chosen pretest probability on the abscissa. Note that outcomes associated with likelihood ratio greater than 1.0 result in an increased post-test probability of the event, as compared with the pretest probability over the entire range of pretest probabilities, whereas likelihood ratios less than 1.0 result in a decreased post-test probability of the event as compared with the pretest probability over the entire range of pretest probabilities. If the likelihood ratio equals 1.0, the post-test probability of the event will be the same as the pretest probability. In other words, the test result has no diagnostic information. Therefore, a test with both the LR+ and LRvery close to 1.0 will have very little clinical value under any circumstances. A test may be quite useful even if one of the likelihood ratios is close to 1.0, as long as the other likelihood ratio is not. For example, a test for which LR- is close to 1.0 but LR+ is quite high may be an excellent choice to rule in a diagnosis, i.e., to raise the estimate of p{e+), even though it is not very useful to lower p{e+) and rule out the diagnosis. Conversely, if LR+ is close to 1.0 but LR- is quite low, the test is best used to rule out a diagnosis, i.e., to lower p{e+) rather than to increase p{e+) and rule in the diagnosis. In general, unexpected results of tests cause the most marked change in estimated probability of the occurrence of an event. If the pretest probability of an event is low, to the left in Figure 4, a positive result of a test with a high LR+ will result in a marked increase in the estimated probability of the event. Similarly, if the pretest probability of an event is high, to the right in Figure 4, a negative result of a test with a low LR- will result in a marked decrease in the estimated probability of the event. Fagan reported another way of using likelihood ratios, a nomogram from which the post-test probability of disease can be read for any combination of pretest probability and likelihood ratios. 12 Table llists the sensitivity, specificity, and likelihood ratios for the tests shown in Figure 3. Following in this review are similar tables that illustrate the effect of the selection of alternative cutoff points for andrology tests and the application of Bayes' rule to revise probability estimates for two events, pregnancy or the result of another laboratory test. Postcoital Tests CLINICAL EXAMPLES Let us start with a common question: "Do the results of the postcoital test predict fertility?" The study by Skaf and Kemman 13 of the postcoital test in patients treated with human menopausal gonadotropins is of special interest because all subjects were tested when they had profuse cervical mucus, a study design that removed one troublesome variable, the quality of mucus. Of a total of 50 subjects, 24 conceived during the ovulation induction cycle. Eleven of 24 women who conceived had a positive postcoital test result, defined as more than motile sperm per high power field (hpf). Thus, as shown in row 1 of Table 2, the sensitivity of the test p{t+ IE+) is 11/24 = The specificity, p{t-1 E-), is /26 = Six of 26 women who did not conceive also had a positive result. The likelihood ratio for a positive test (11/24)/(6/26) is 1.98, and the likelihood ratio for a negative test (13/24)/(/ 26) is To determine how a positive or a negative result of a test will change the clinical estimate of the probability of conception, we start by estimating Vol. 51, No.2, February 1989 Polansky et al. Bayes' rule applied to male fertility tests 221

8 Table 1 Test Characteristics for the Three Tests Illustrated in Figure 3 Cutoff Distribution Figure point Sensitivitya Identical 3A X 0.15 y 0.50 z 0.85 Partial 3B X 0.70 Overlap y 0.95 z 1.00 Separate 3C X 0.95 y 1.00 z 1.00 a Sensitivity = p (T+ IE+). b Specificity = p (T-1 E-). Likelihood Likelihood ratio for ratio for Specificityb positive test negative testd CXJ CXJ CXJ Likelihood ratio positive= p (T+ I E+)/p (T+ I E-). d Likelihood ratio negative = p (T-1 /E+)/p (T-1 E-). the pretest probability of the patient being fertile, p(e+). In the Skaf and Kemman study, 13 p(e+) is 24/50 or To see the effect of knowing the result of the postcoital test, first find the curves in Figure 4 that most closely approximate the LR+ and LR- in Table 2: 1.98 and Next, note that, for a pretest probability of0.48, the post-test probability of conception for a positive result, p(e+ IT+), is approximately 0.65, and that for a negative result,p(e+ IT-); the post-test probability is approximately A modest change from the pretest probability of 0.48 to a post-test probability of 0.65 or of 0.40 is unlikely to cause the clinician to alter his or her ideas about diagnosis or therapy. Note in Table 2 that a change of the cutoff point defining the border between a positive and a negative test result improves the ability of this test to discriminate between those who later conceived during treatment and those who remained infertile. A useful rule of thumb is illustrated by the data of Skaf and Kem- manp Moving the cutoff point in the direction of a negative result, the result expected in the group without the event, improves sensitivity, but inevitably reduces specificity. In this example, the event is conception during treatment, and moving the cutoff point from to 5 sperm/hpf logically will increase the sensitivity of the test in detecting those who will conceive. However, change in the likelihood ratios is not so easily predicted. In the case shown in Table 2, improvement in the likelihood ratio for a negative test that accompanies shifting the cutoff point far outweighs the loss in the likelihood ratio for a positive test. Using again a pretest probability of 0.48 in Figure 4, and the likelihood ratios obtained with the cutoff point of 5 motile sperm, LR+ = 1.66 and LR- = 0., we would estimate the probability of being in the group that conceives to be about 0.60 for a positive test result and 0. for a negative test result. Although shifting the cutoff point from 5 to 1 further improves the likelihood ratio for a negative test, it Table 2 Use of the Postcoital Test as a Measure offertilitya Likelihood Likelihood No. of Cutoff point ratio for ratio for Study subjects (sperm/hpf) Sensitivity Specificity positive test negative test Skaf & Kemman Jette & Glass Hull et al a Sensitivity, specificity, and likelihood ratios are defined in the caption of Table 1. T+, Sperm present in numbers per high power field (hpf) exceeding the cutoff points (, 5, or 1) tested; E+, Subject is a member of the group that later became pregnant. 222 Polansky et al. Bayes' rule applied to male fertility tests Fertility and Sterility

9 reduces the likelihood ratio for a positive test almost to the line of identity. The choice of the cutoff point that gives the best combination of likelihood ratios and, hence, the best combination of post-test probabilities for positive and negative test results depends on several factors. These are: (1) the pretest probability of the event in the patient studied; (2) the importance of making the diagnosis as judged by the relative severity of the penalty for making errors in each direction, erroneously making or erroneously missing the diagnosis; (3) the cost of the test, taking into account the difficulty of obtaining it and any restrictions on the opportunity to repeat it. As another example, let us look at data from the study by Jette and Glass (Table 2), who reviewed postcoital test results of patients undergoing an infertility investigation rather than those undergoing gonadotropin therapy. 14 During a follow-up period extending to 4 years, about half of the patients in this study conceived. This study group is similar in composition and in pretest probability p(e+) of subsequent conception to groups of patients to whom most clinicians will apply the postcoital test. A pretest probability of 0.50 is reasonable if we consider patients in any infertility clinic to be the test group and pregnancy within 5 years to be the event. When we use motile sperm/hpf as a cutoff point, the sensitivity, specificity, and likelihood ratios are quite comparable to those calculated from the data of Skaf and Kemman's studyp Data from this study, and data from the life table at 1 year in the Hull et al. study/ 5 both also illustrate the shifts in sensitivity and specificity that can be predicted after changes in the cutoff point. Moving the cutoff point in the direction of the test results expected in the population without conception, from to 5 and to 1 motile sperm/hpf, again increases the sensitivity and decreases the specificity. The choice of a cutoff point that gives the overall best test characteristics will depend on the clinical situation. For example, let us consider a woman who has had a cervical cone biopsy followed by many years of infertility. In this situation, we estimate that the pretest probability of future fertility, p(e+) is low. And, therefore, a test with a high LR+ would be our best choice, since a positive result would greatly raise our estimate of the probability, p(e+ IT+) (Figure 4). This may be achieved by setting the cutoff point for the number of motile sperm per high power field to a high value. A second example could be a young woman who has been attempting to conceive for only a short time and who has nothing in the history or physical examination suggesting abnormalities of the cervical mucus. Our pretest estimate of the probability of fertility is high, and a test with a very low LRis the one that would change our estimate the most. Therefore, moving the cutoff point in the opposite direction, to low numbers of sperm per high power field, would be the preferred maneuver. This results in a low value for p(e+ IT-). The fact that, for all three studies, neither the likelihood ratio for a positive test nor the likelihood ratio for a negative test is far from 1.0 means that knowing the outcome of a postcoital test will not drastically change the estimated probability of fertility in either direction. In Vitro Mucus Penetration Tests The in vitro sperm penetration test avoids two clinical problems in the use of postcoital test: determining the best time for the test in relation to ovulation and deciding on the optimal interval between coitus and the test. The in vitro mucus pen~tration test is of special use in evaluating couples who repeatedly have had several poor postcoital tests despite evidence of intravaginal ejaculation, proper timing, and an adequate semen analysis. The test should help the clinician to decide if the problem lies with the mucus or with the semen. For the test, as commonly done today, a commercially prepared flat capillary tube containing frozen bovine cervical mucus is thawed, and one end is placed in a reservoir of semen for 90 minutes. The distance along the capillary tube traveled by the most advanced sperm, called the sentinel sperm, is measured with a microscope. 16 There is ample evidence that bovine and human cervical mucus share enough physical and chemical properties that bovine mucus can be used for in vitro testing of human sperm penetration It can be obtained in batches of 0 ml, frozen without serious loss of penetrability and, thus, provides a reasonably standard test material. The commercially available Penetrak test (Serono Diagnostics, Norwell, MA) measures in vitro sperm penetration in bovine mucus. Of the 60 males tested in a study reported in the brochure for this test, 38 were semen donors of proven fertility. The others were selected because they were partners in an infertile marriage and, in addition, had a poor postcoital test. The group in which the sentinel sperm migrated more than 2 em along the capillary tube contained a notably higher proportion of fertile donor males; those with 2 em Vol. 51, No.2, February 1989 Polansky et al. Bayes' rule applied to male fertility tests 223

10

11 Table 4 Use of the Zone-Free Hamster Egg Penetration Test as a Measure of Fertility Cutoff point No. of (%of eggs Study subjects penetrated) Rogers et al Aitken et al Karp, et al Margalioth et al Martin et al Overstreet et al Zausner-Guelman et al Cohen et al Sensitivity, specificity, and likelihood ratios are defined in the caption of Table 1. T+, Penetration by one or more sperm Likelihood Likelihood ratio for ratio for Sensitivity Specificity positive test negative test O.o of a percentage of hamster eggs exceeding the cutoff point listed. E+, Subject is a member ofthe fertile (donor) group. the follow-up period. Semen samples from artificial insemination donors or from men who recently fathered a pregnancy served as positive controls. In one of the initial reports of the application of this method to the study of infertile males, Rogers and colleagues examined semen samples from 21 fertile donors and 30 infertile males. 27 Sperm from fertile males penetrated zona-free hamster eggs to a greater degree (14 to 0% of hamster eggs) than did sperm from infertile males (O to %). Since, when using % egg penetration as a cutoff point, there was no overlap at all in results between subjects' and controls', the sensitivity and specificity of the test were both 1.0, the test characteristics of a perfect test. Most other authors whose data are tabulated in Table 4 have also reported data for a cutoff point that provides a sensitivity of 1.0, i.e., no negative test results among men who caused a pregnancy. The sensitivities, specificities, and the likelihood ratios are for the most part better than those for the postcoital test or the in vitro mucus penetration test. Using the data of Table 4 to again illustrate the practical application of likelihood ratios, let us consider a male partner in an infertile marriage whose pretest probability of subsequent fertility we estimate as 0.50, p(e+) = Select a set of likelihood ratios for a positive test and for a negative test from Table 4, for example, those from Martin: LR+ = 2.47 andlr- = FromFigure4 or by calculation, find the post-test probability of subsequent pregnancy. If the hamster test is positive, the post- test probability increases only modestly: p(e+ I T+) = If the hamster test result is negative, however, the post-test probability drops markedly: p(e+ IT-) = Thus, a negative result of the hamster test is of greater clinical value, as it changes the estimated probability of the occurrence of the event much more dramatically. A great variation in methods used makes it difficult to decide which of the three cutoff points shown in the table best differentiates between the fertile and infertile male. Since changes in the laboratory methods used can result in marked differences in the rates of penetration of the hamster egg, cutoff points should be selected after analysis of the sensitivity and specificity in each laboratory or, at least, for each unique combination of laboratory methods. Predicting Hamster Test Results from Semen Analysis Data One can use Bayes' rule and the results of routine semen analysis to predict the outcome of the hamster penetration assay (Table 5). With a cutoff point for sperm concentration of million sperm per ml, for example, the calculated likelihood ratios for a positive test differ only slightly from 1.0, but the likelihood ratios for a negative test differ markedly from 1.0. Predicting In Vitro Fertilization of Human Eggs with Hamster Test Data In some in vitro fertilization (IVF) programs, the male partner is required to have a positive hamster Vol. 51, No.2, February 1989 Polansky et al. Bayes' rule applied to male fertility tests 225

12 Table 5 Use of the Semen Analysis to Predict the Hamster Egg Test Result" Test No. of cutoff Study subjects Test point Rogers Sperm concentration M Motility 50% Normal morphology 50% Berger et al Sperm concentration M Motility 60% Normal morphology 60% Zausner-Guelman 45 Sperm concentration M et al. 33 Motility 50% Aitken et al Sperm concentration M Sensitivity, specificity, and likelihood ratios are defined in the caption of Table 1. T+, Sperm concentration, percent motility, or percent normal morphology exceeding the cutoff point Likelihood Likelihood ratio for ratio for Sensitivity Specificity positive test negative test OC) 0.05 listed; E+, Penetration by one or more sperm of more than % of hamster eggs. egg penetration test. Donor semen may be recommended in the presence of two negative hamster tests. The hamster egg test also has been proposed to differentiate between laboratory failure and inherent reduced fertilizing capacity of the husband's sperm in case of failure of in vitro fertilization of human ova. 40 The six studies summarized in Table 6 were selected because they provide data on three different cutoff points. Only three of the six have both LR+ greater than 1.0 and LR- less than 1.0 for all three cutoff points. The results of the other three studies are opposite to the intuitively expected results. In these instances, for one or more of the cutoff points, either a positive hamster egg penetration test predicts a decrease in the probability of fertilization of human oocytes by IVF or a negative hamster test predicts an increase in the probability. In the study of Wolf and co-workers, 41 for example, the LR- for a cutoff point of 1% or % is infinity, meaning that all males with a negative hamster test fertilized at least one human oocyte in vitro. The results shown in this table illustrate the importance of differences in methodology and the need for each laboratory to choose its own cutoff points. Since the hamster egg penetration test does not reliably predict the results of in vitro fertilization of human eggs, but better predicts the chance of an infertile male causing a pregnancy, one might postulate that the procedures of human in vitro fer- Table 6 Use of the Zone-Free Hamster Egg Penetration Test to Predict the Results ofln Vitro Fertilization of Human Oocytes Test cutoff point Likelihood Likelihood No. of (%of eggs ratio for ratio for Study subjects penetrated) Sensitivity Specificity positive test negative test Wolfe et aly 23 1 Margalioth et al Foreman et al Ausmanas et al Van Uem et al Cohen et al (22.C) 1(4"C) OC) OC) OC) Sensitivity, specificity, and likelihood ratios are defined in the caption of Table 1. T+, Penetration by one or more sperm of a percentage of hamster eggs exceeding the cutoff point listed; E+, Fertilization of one or more human oocytes in vitro. 226 Polansky et al. Bayes' rule applied to male fertility tests Fertility and Sterility

13 tilization may bypass or correct some factors that play a role in human infertility and are measured by the hamster egg test. CONCLUSIONS The examples in this paper, using data for three male factor infertility tests, illustrate the application of Bayes' rule to clinical problems. This type of Bayesian analysis could be applied to many medical problems since the calculations are simple and straightforward, providing that reports in the medicalliterature provide sufficient information for the reader to construct 2 X 2 contingency tables. Intuitive predictions tend to ignore the effect of pretest probabilities. 45 We have therefore emphasized the importance of knowing or estimating the pretest probability of the occurrence of the event of interest-conception, disease, etc., in the patient or in the population of patients tested. We have presented a simple graphic method to modify the estimated probability in light of test results. The Bayesian approach is also useful in determining the cutoff points defining the border between normal and abnormal results for various clinical laboratory tests. This idea was illustrated using examples from the three tests. Because of increasing recognition in our field of the value of quantitative reasoning based on Bayes' rule, it is important that editors encourage investigators to explicitly state in publications the sensitivity and the specificity of the tests used in the studies and to provide sufficient data that the interested reader can perform the calculations at different cutoff points. Acknowledgments. The authors are indebted for helpful discussions to John A. Collins, M.D., James W. Overstreet, M.D., Ph.D., Harold C. Sox, Jr., M.D., and Eugene Washington, M.D. They are also grateful for the help in the initial phases of development of this paper by Hagai Kaneti, M.D., a resident visiting from Hadassah Medical School, Israel. Recommended readings. In addition to several works that have been cited, th!,u the authors suggest the following as sources of further information about the clinical applications of Bayes' rule. Chapter 4 in Sackett is an especially lucid description of the principles. 1. Gale RS and Gambino SR: Beyond Normality: The Predictive Value and Efficiency of Medical Diagnoses. New York, John Wiley & Sons, Sackett DL, Haynes RB, Tugwell P: Clinical Epidemiology, A Basic Science for Clinical Medicine. Boston, Little Brown and Company, Sox HC, Blatt M, Higgins MC, Marton Kl: Medical Decision Making. Stoneham, Butterworths, Inc., REFERENCES 1. Blasco L: Clinical tests of sperm fertilizing ability. Fertil Steril41:177, Bronson R, Cooper G, Rosenfeld D: Sperm antibodies: their role in infertility. Fertil Steril42:171, Rogers BJ: The sperm penetration assay: its usefulness reevaluated. Fertil Steril43:821, Kong A, Barnett GO, Mosteller F, Youtz C: How medical professionals evaluate expressions of probability. N Engl J Med 315:740, Friedman HA: Introduction to statistical inference. Reading, MA, Addison-Wesley, 1963, p lngelfinger JA, Mosteller F; Thibodeau LA, Ware JH: Diagnostic testing: introduction to probability. In Biostatistics in Clinical Medicine. New York, Macmillan Publishers, 1983,p 1 7. Sox Jr HC: Probability theory in the use of diagnostic tests. Ann Intern Med 4:60, Weinstein MC, Fineberg HV, Elstein AS, Frazier HS, Neuhauser D, Neutra RR, McNeil BJ: Clinical decision analysis. Philadelphia, W.B. Saunders Company Sheffield LJ, Sackett DL, Goldsmith CH, Doran TA, Allen LC: A clinical approach to the use of predictive values in the prenatal diagnosis of neural tube defects. Am J Obstet Gynecol145:319, Stempel LE: Eeenie, meenie minie, mo.... What do the data really show? Am J Obstet Gynecol144:745, Griner PF, Mayewski RJ, Mushlin AI, Greenland P: Selection and interpretation of diagnostic tests and procedures. Ann Intern Med 94:553, Fagan TJ: Nomogram for Bayes's theorem. N Engl J Med 293:257, Skaf RA, Kemmann E: Postcoital testing in women during menotropin therapy. Fertil Steril37:514, Jette NT, Glass RH: Prognostic value ofthe postcoital test. Fertil Steril 23:29, Hull MGR, Savage PE, Bromham DR: Prognostic value of the postcoital test: prospective study based on time-specific conception rates. Br J Obstet Gynaecol 89:299, Serono: Penetrak informational insert, Serono Diagnostics, Moghissi KS, Sacco AG, Borin K: Immunologic infertility I. Cervical mucus antibodies and postcoital test. Am J Dbstet Gynecol136:941, Alexander NJ: Evaluation of male infertility with an in vitro cervical mucus penetration test. Fertil Steril 36:1, Bergman A, Amit A, David MP, Homonnai ZT, Paz GF: Penetration of human ejaculated spermatozoa into human and bovine cervical mucus. I. Correlation between penetration values. Fertil Steril 36:363, Gaddum-Rosse P, Blandau RJ, Lee WI: Sperm penetration into cervical mucus in vitro. I. Comparative studies. Fertil Steril33:636, Gaddum-Rosse P, Blandau RJ, Lee WI: Sperm penetration into cervical mucus in vitro. II. Human spermatozoa in bovine mucus. Fertil Steril 33:644, Lee WI, Gaddum-Rosse P, Blandau RJ: Sperm penetration into cervical mucus in vitro. III. Effect of freezing on estrous bovine cervical mucus. Fertil Steril 36:9, 1981 Vol. 51, No.2, February 1989 Polansky et al. Bayes' rule applied to male fertility tests 227