Project: Conditional Probability (the sequel)

Transcription

1 Project: Conditional Probability (the sequel) Police, when stopping people for DWI (Driving While Intoxicated), use a test manufactured by a certain chemical company. One particular test is designed to indicate the presence of THC a in the suspect s system. The manufacturer claims that its portable police test will detect the presence of THC in the blood (i.e. show positive for a driver who has ingested THC in the last 72 hours) 92% of the time. However, the manufacturer admits that 10% of all THC-free drivers also test positive. One of your friends has just told you that he was recently stopped by the police and this roadside drug test for the presence of THC showed positive. He tells you that he hadn t smoked marijuana for at least a year before he was pulled over that night (a national survey indicates that 20% of all drivers have smoked pot during the last 72 hours). Calculate the probability that your friend had THC in his system during the 72 hours preceding the drug test. The purpose of this project is to explore an oft neglected aspect of probability called Bayes Theorem that occurs in everything from eyewitness reliability in court cases to disease testing to SPAM blocking. I believe it s neglected in most probability courses because, when taught traditionally, it scares the bejeezus out of people. I mean, for goodness sake, here s the formula you d have to use, if you wanted to use its formula: P( A B) P( B A) P( A) C C P( B A) P( A) P( B A ) P( A ) We, for sure, will NOT be using that monster b. We also won t be using its nicer tree diagrams: No, no we ll attack this thing more holistically, for sure. We ll use CT s, which, by now, you ve grown comfortable navigating. a THC is Delta-9-tetrahydrocannabinol; the active ingredient in marijuana. b Actually, I think it s kinda awesome but my students have always hated it.

2 Let s look at the situation our friend is in a little more closely. If someone take a drug test, there are four possible scenarios: Tested Positive Tested Negative The Test Worked False Negative False Positive The Test Worked So, you can see that this isn t as simple as it might first seem. Your friend tested positive. This means that he might have properly tested positive (meaning he had smoked pot in the last 72 hours), or he got a false positive reading from the test (meaning that he had no THC in his system, but the test flagged him anyway). Example 1: Find p(your friend smoked pot in the past 72 hours, given that he tested positive). Answer 1: Let s attack it with contingency tables. Begin with a population of, say, 10,000 drivers, each of whom has taken the drug test described above. This means that we have to assign these 10,000 people to the four cells in the contingency table so that 1) each person is in exactly one cell, and 2) the probabilities pertinent to the known information are correct. For starters, we know (from the problem) that 20% of all drivers have smoked pot in the last 72 hours. Thus, of our 10,000 drivers, 2,000 have smoked pot, and 8,000 haven t: Tested Positive Tested Negative Now, we have to adjust the chart to reflect the true and false positives. The test will correctly identify THC in a person s system 92% of the time; that is, of the 2000 folks that have THC in their system, 92%, or 1840, of them, will test positive. The other 160 will get a false negative: Tested Positive 1840 Tested Negative

3 Of those who are clean, the test will wrongly accuse them (a false positive) 10% of the time. That is, of the 8000 clean drivers, 800 will get a false positive reading, while the other 7200 will (correctly) be labeled THC free: Tested Positive Tested Negative We now have all we need to find the chance that our friend is clean. Since his test was positive, we have no need to concern ourselves with the tested negative row (remember given that he tested positive ): Tested Positive Tested Negative So, this means that he is in the smaller subgroup of 2640 people in the chart. The probability that is actually has the drug in his system is This means that the probability that he is clean is p(using, assuming he tested positive) = p(clean, assuming he tested positive) = This means that it s more likely that your buddy had THC in his system when he was stopped. Oops. By creating the contingency tables, we are, in effect, creating a random sample of folks to illustrate the probabilities. In essence, the table is better than any possible random sample: the numbers reflect a perfect alignment of the likelihoods in the problems (of course, this assumes the chances given are correct). Let s try another one! Example 2: Environmentalists have developed a test for determining when the mercury level in fish is above permissible levels set by the FDA and EPA. If the fish s mercury level exceeds this permissible level, the test is 99% effective in determining this. However, if the fish s mercury level is within permissible limits, the test will incorrectly indicate that the level is too high 4% of the time (again, this is a false positive...for the fish). This test is to be administered to fish in a river into which a chemical company has been dumping its wastes. It is estimated (from pilot studies; more on this later) that 30% of the fish in this river contain mercury in excessive amounts. If a fish is caught and tests positive, what is the probability that the fish s mercury level exceeds permissible levels?

4 Answer 2: OK, once again, we assume 10,000 fish in the river c. Based on the pilot study, 3000 are assumed to be laden with excessive mercury, while the others are assumed to be clean (enough): Excessive Mercury Clean Enough Tested Positive Tested Negative Now, for the hidden probabilities. First, of the fish with excessive mercury, 99% will be identified by the test the FDA and EPA uses. That means that, of the 3000 contaminated fish, 2970 will test positive for excessive mercury, while 30 will be off the hook (nyuck, nyuck): Excessive Mercury Tested Positive 2970 Tested Negative 30 Clean Enough 7000 For the remaining clean fish, 96% will (correctly) test negative, while 4% will (incorrectly) test positive: Excessive Mercury Clean Enough Tested Positive Tested Negative Now, we caught a fish, and he tested positive. What s the chance he has too much mercury? p(fish has excessive mercury tests positive) = c You may ask yourself, Why 10,000? What if there are more? Or fewer? Great question! It doesn t matter what number you use, so long as the probabilities are represented, because the ratios will always be the same, regardless of your population...just like the supercounting project.

5 This test appears to be doing a pretty good job at catching contaminated fish. Let s look at a different probability, with the same table: p(fish is clean tests negative) = This test does a stellar job at identifying clean fish! Now that we have these two probabilities, we can also find the chance of a false positive and a false negative...they re just the complements, respectively, of the above two probabilities: p(false positive) = p(fish is clean tests positive) = 0.09 p(false negative) = p(fish has excessive mercury tests negative) = You didn t mind that I used the notation for conditional probability there, did you? Good! So much information can be gleaned from a contingency table. Let s look at a couple more examples, shall we? At this point, if you d like another look at how to break up (or break down) a CT to do these, I recorded a video to help. Here s the link: Example 3: There is a test for Down s syndrome that can be done during the 4 th month of pregnancy for at-risk mothers d. If an at-risk fetus actually has (or will develop) Down s syndrome, the test is accurate 65% of the time, with a 7.2% false positive rate. A pregnant friend of yours (who is not deemed at risk) takes the test, and the test comes back positive. What is the chance that her baby will have Down s syndrome? Assume that the incidence of Down s syndrome in the general population is estimated to be 1 in 900 births (from the CDC). Answer 3: The pregnant friend of whom I speak is actually my neighbor. She was, as you can imagine, an absolute wreck, so, I decided to do a little figuring on my own. Here s the math I showed her. In 10,000 births (in the general population): Baby has Down s Baby Doesn t Have Down s Tested Positive Tested Negative d This test (an amniocentesis) is invasive, requiring the insertion of a needle into the uterus. Thus, it s not pleasant, and shouldn t be done (as you ll see), unless there is evidence to show the mother (or developing baby) is deemed at risk.

6 So, knowing she tested positive: p(baby will have Down s tests positive) = Less than a 1% chance. That means that p(baby will not have Down s tests positive) = Of course, being human, we tend to focus only on the fact that the test came back positive, and not on the fact (in this case, anyway) that the positive result might actually be meaningless. Actually, in this case, a positive test result is a good thing e. And, check this out: p(baby will not have Down s tests negative) = So, let s get this straight...if you re not at risk, and test positive for Down s, there s a 99% chance your baby s OK. And, if you test negative, there s a 99.96% chance your baby s OK. So, why would you even take the test? By the way, my neighbor s baby, Adeline, was born without Down s Syndrome. Follow up: I ve done more research into this matter, and I ve discovered a couple of things: 1) The chance of a fetus developing Down s while in the womb varies with the mother s age. Exponentially, in fact. In the optional spreadsheet, you can examine results of the above experiment with mothers from ages 20 through 49, if you like. 2) The 65% true positive and 7.2% false positive are hardly industry standard. I ve been told they should be 85% and 5%, and I ve also been told (by others) that those are too high. That s why I built the spreadsheet to allow for any sensitivity/specificity you like. Have at it, if you want to! 3) At right is a graph one test includes in its promotional materials. One glance at those curves, an at my spreadsheet, and hopefully you realize they re NO WAY in line with what s actually happening in the world of these tests (bell curves? Really?). e My neighbor should have never even taken this test, as she wasn t at risk. All a positive test result does is make expectant mothers worry, which may cause damage to the developing baby that wouldn t have occurred otherwise.

7 Example 4: According to the CDC, there are about 1,100,000 people living with HIV in the US. However, many of them (about a quarter, according to the CDC), don t know they are carrying HIV. There is an antibody test for HIV that can be taken. This test has a false positive rate of %, and a false negative rate of %. Assuming someone not atrisk tests negative for HIV, using this antibody test, what is the probability that this person does not have HIV? Answer 4: Ready? Using the government s estimate of 300,000,000 Americans, the incidence of HIV is about 37 in 10,000, so, out of 10,000 random Americans, Has HIV Doesn t Have HIV Tested Positive Tested Negative p(person doesn t have HIV tests negative) = Basically, if you test negative, you re clean. So, I think we can tell this person that they should feel pretty good about that negative test result. Finally, a few terms that will help you understand any kind of similar situation you may see in the future, terms that get used quite a bit in speaking of disease and drug tests: false positive and false negative are two we saw above. A false positive is used in conversation to indicate that a test has been given a positive test result in error; however, the false positive rate is defined to be the probability of a false positive. The false negative rate is, thus, the probability of a false negative. Two other important terms that sometimes come up in medical literature are sensitivity and specificity. Sensitivity is the probability of a true positive test result. Specificity is the probability of a true negative test result. As illustrations...in Example 2 above, the disease test s sensitivity is 99%, and its specificity is 96%. Further, its false negative rate is 1% and its false positive rate is 4% (sensitivity and false negative rate, as you probably see, are complementary, as are specificity and false positive rate).

8 Here we go...round each of the probabilities to the hundredths place, if needed (if you re using decimals; whole numbers if you re using percents). I ve given you some answers in red, but you still have to show me how you got the answers (your contingency tables and the fractions you used would qualify as work ). We ll start with some continuations of the above examples! For this first one, you can just use a CT that s already done 1) (2 points) We ll start with a continuation from a previous problem. Look back at Example 4. Find p(has HIV tested positive). Use the CT that s already complete there! 2) (2 points) Look back at Example 1. Find the probability of a false negative; that is, find p( was using THC tested negative). 3) Now, you ll see how the ideas of disease testing can be a little misleading. Here s a personal example: Lyme disease is a pretty terrible tick-borne illness, very prevalent in the Eastern US (not so much out here in Bend, though). See that little guy over there? That s a deer tick, the little sucker (literally) that gives you Lyme Disease. He s shown in his nymph stage, the one that causes most of the trouble, because, really, do you think you ll ever see something that small on you? I had Lyme disease thrice, and never once did I find the tick that gave it to me. Anywho...in the article Laboratory Considerations in the Diagnosis and Management of Lyme Borreliosis (American Journal of Clinical Pathology [1993]), the test for Lyme disease identification was analyzed. It was found that, in the general population, 0.207% of people carry Lyme disease (note: that s 0.207%, which, as a decimal, is ). The test has a sensitivity of 93.7% and a specificity of 97%. Sounds good! Suppose you take the test. Find a) (4 points) (w) p(have disease you test positive) 6% b) (1 point) p(don t have disease you test positive) c) (4 points) (w) p(have disease you test negative) 0% d) (1 point) p(don t have disease you test negative) e) (4 points) Based only on your answers above, how would you feel if you tested positive for this test? Why? 4) (4 points) (w) Gerd Gigerenzer, a cognitive psychologist f at the Max Planck Institute for Human Development in Berlin, asked doctors in Germany and the United States to estimate the probability that a woman in the general population with a positive mammogram actually has breast cancer: (paraphrased slightly) The probability that a woman has breast cancer is 0.8% (that is, 0.008). If a woman has breast cancer, the probability is 90 percent that she will have a positive mammogram. If a woman does not have breast cancer, the probability is 7 percent that she will have a falsely positive mammogram. Imagine a woman who has a positive mammogram. What is the probability that she actually has breast cancer? that is, find p(has breast cancer tested positive) f With a totally rad name.

9 Over 100 doctors were surveyed, and their answers ranged from 1 percent to 90 percent. Of American doctors surveyed, most put the probability at 75%. The correct answer is a little more than 9%. Show me why. 5) (extra 3 points) (w) Have you been wondering where the infection rates of the disease in the above problems came from? Me too! g But then I realized, you can use the math from this project to help! Check it out! Suppose that 90% of the population of a town is inoculated against a certain disease. After a time, a sample of the townspeople is drawn. In the sample, it is shown that 25% of those who were not inoculated came down with the disease, while only 1% of those who were inoculated came down with the disease (ignore false positives or false negatives). How prevalent is the disease in this town (that is, what is the probability that a randomly selected member of this town has the disease)? g I won t be upset if you really didn t.