Bayes theorem (1812) Thomas Bayes ( ) English statistician, philosopher, and Presbyterian minister

Transcription

1 Bayes theorem (1812) Thomas Bayes ( ) English statistician, philosopher, and Presbyterian minister

2 Bayes theorem (simple) Definitions: P(B A) =P(B A)/P(A); P(A B) =P(A B)/P(B) In Science we often want to know: How much faith should I put into hypothesis, given the data? or P(H D) What we usually can calculate is: Assuming that this hypothesis is true, what is the probability of observing this data? or P(D H) Bayes theorem can help: P(H D)=P(D H) P(H)/P(D) The problem is P(H) (so called prior) is often not known

3 Bayes theorem (continued) Works best with exhaustive and mutually exclusive hypotheses: H 1, H 2, H n such that H 1 U H 2 U H 3 U H n =S and H i H j =ᴓ for i j P(H k D)=P(D H k ) P(H k )/P(D) P(H k ) is a prior of hypothesis k. But what is P(D)? P(D)= P(D H 1 ) + P(D H 2 )+ P(D H n )= = P(D H 1 ) P(H 1 ) + P(D H 2 ) P(H 2 ) + P(D H n ) P(H n )

4 An awesome new test has been invented for an early detection of cancer. The probability that it correctly identifies someone with cancer as positive is 95%, and the probability that it correctly identifies someone without cancer as negative is 99%. The incidence of this type of cancer in the general population is A random person in the population takes the test, and the result is positive. What is the probability that he/she has cancer? A. 99% B. 95% C. 30% D. 1% Get your i clickers 4

5 An awesome new test has been invented for an early detection of cancer. The probability that it correctly identifies someone with cancer as positive is 95%, and the probability that it correctly identifies someone without cancer as negative is 99%. The incidence of this type of cancer in the general population is A random person in the population takes the test, and the result is positive. What is the probability that he/she has cancer? A. 99% B. 95% C. 30% D. 1% Get your i clickers 5

6 Let s try to check if it makes sense Consider 1,000,000 people from the street Only 1e6*1e 4=100 will have this rare cancer 95 of them (95%) will be identified by the test Out of 1,000, ,000,000 people without cancer, 1% will get a false positive diagnosis That is 10,000 people!!! The probability that a random person has cancer if diagnosed is 95/(10,000+95) 1%

7

8

9

10

11

12

13 How come? We thought it was a great test.. Let C be the event that the patient has cancer; C patient is cancer free Y/N events that test is Positive/Negative (N=Y ) We know: P(C)=10 4, P(Y C)=0.95, P(N C )=0.99 We need to find P(C Y) Bayes to the rescue: P(C Y)=P(Y C)*P(C)/P(Y) What on earth is P(Y)???

14 What is P(Y)??? Likelihood that a random patient would test Y: P(Y)=P(Y C)+P(Y C )=P(Y C)P(C)+P(Y C )P(C )= 0.95*10 4 +(1 0.99)*( ) 0.01 Hence P(C Y)=P(Y C)*P(C)/P(Y) 10 4 /0.01=0.01=1% But we would like it to be 100%, please!!! Until the false positive discovery rate 1 P(N C ) does not fall below the general population prevalence the result will never be close 100%

15 What is the prior probability in this problem? A. P(C)=population incidence=10 4 B. P(Y)=probability of positive test 0.01 C. P(Y C)=probability pos. given cancer=0.95 D. P(CIY) =probability of cancer if pos E. I don t have a foggiest clue Get your i clickers 15

16 What is the prior probability in this problem? A. P(C)=population incidence=10 4 B. P(Y)=probability of positive test 0.01 C. P(Y C)=probability pos. given cancer=0.95 D. P(CIY) =probability of cancer if pos E. I don t have a foggiest clue Get your i clickers 16

17 What if I am already 50% sure (based on other tests) that a patient has cancer? That changes everything! Now P(C)=P(C )=0.5 P(C Y)=P(Y C)*P(C)/[P(Y C)*P(C)+ P(Y C )*P(C )]= 0.95*0.5/[0.95*0.5+(1 0.99)*0.5]=0.99 Now the doctor can be almost 100% sure. The importance of prior: If prior belief that one has cancer is 10 4 test is useless If prior belief is at least 1% the test is useful

18 (15 points) Prostate cancer is the most common type of cancer found in males. It is checked by PSA test that is notoriously unreliable. The probability that a noncancerous man will have an elevated PSA level is approximately 0.135, with this probability increasing to approximately if the man does have cancer. If, based on other factors, a physician is 70 percent certain that a male has prostate cancer, what is the conditional probabilitythathehasthecancergiventhatthetest indicates an elevated PSA level? A % B. 99% C. 82% D. 71% Get your i clickers 18

19 (15 points) Prostate cancer is the most common type of cancer found in males. It is checked by PSA test that is notoriously unreliable. The probability that a noncancerous man will have an elevated PSA level is approximately 0.135, with this probability increasing to approximately if the man does have cancer. If, based on other factors, a physician is 70 percent certain that a male has prostate cancer, what is the conditional probabilitythathehasthecancergiventhatthetest indicates an elevated PSA level? A % B. 99% C. 82% D. 71% Get your i clickers 19

20

21

22 All this trouble for a lousy 12% gain in confidence? I don t believe you! Let C be the event that the patient has cancer; C patient is cancer free, E events that the PSA test was elevated We know doctor s prior belief: P(C)=0.7 We know test stats: P(E C)=0.268, P(E C )=0.135 We need to find P(C E)=P(E C)*P(C)/P(E) P(E)=P(E C)*P(C)+P(E C )*P(C )= =0.268* *0.3=0.23 P(C E)=0.7*0.268/0.23=0.82=82%

23 What is wrong in this comics? If you are not yet reading XKCD comics you should start

24 Credit: XKCD comics

25 Conditional probabilities continued

26 Let s Make a Deal show with Monty Hall aired on NBC/ABC

27 Monty Hall problem In Monty Hall s game show Let s Make a Deal there are three closed doors. Behind a random one of these doors is a car. Behind two other doors are goats. Monty Hall knows behind which door is the car. After the contestant picks a door, Monty always opens one of the remaining doors with the goat. The contestant is always given the option to switch doors. Is it better to switch or to keep the door you picked? What is the probability of winning the car under switching and non switching strategies?

28 Monty Hall problem. What gives you better chances to win a car? A. Better to switch doors B. Better not to switch doors C. Switching does not matter D. It depends on the phase of the moon E. I don t know Get your i clickers 28

29 Don t feel bad if you guessed wrong When first presented with the Monty Hall problem an overwhelming majority of people assume that each door has an equal probability and conclude that switching does not matter Out of 228 subjects in one study, only 13% chose to switch Paul Erdős, one of the most famous mathematicians in history, remained unconvinced until he was shown a computer simulation confirming the predicted result Pigeons repeatedly exposed to the problem show that they rapidly learn always to switch, unlike humans

30 Solution #1 (intuitive) With Prob=1/3 you guessed the car door right: you loose the car if you switch you win the car if you don t switch With Prob=2/3 you got it wrong and picked a goat door. Then Monty opens another goat door. What is left is the car door. you win the car if you switch you loose the car if you don t switch Since 2/3 >1/3, on average it makes sense to switch

31 Solution #2. Tree & conditional probabilities

32 Solution #2. Tree & conditional probabilities

33 Matlab program skeleton %set Stats large... cars=0; carn=0; %counts successes of switch (cars) and not (carn) strategies. Initialize=0 for i = 1:Stats a = randperm(3); %Monty places two goats and the car at random %a(1) the door for goat #1, a(2) the door for goat #2, a(3) the door for car i= floor(3.*rand)+1; %you randomly select the door! % SWITCH STRATEGY if(i == a(1)) cars=cars+1; %a(2) opened, switch to a(3), car! elseif (i == a(2)) cars = cars + 1 ;%a(1) opened, switch to a(3), car! else cars = cars + 0; %a(1)/a(2) opened, switch to a(2)/a(1), no car! end % NO SWITCH STRATEGY if(i == a(1)) carn = carn +?? elseif (i==a(2)) carn = carn +?? else carn = carn +?? end; end; disp(cars); disp(carn);

34 Matlab program Stats=10000; %set Stats large... cars=0; carn=0; %counts successes of switch (cars) and not (carn) strategies. Initialize=0 for i = 1:Stats a = randperm(3); %Monty places two goats and the car at random %a(1) goat, a(2) goat, a(3) car i= floor(3.*rand)+1; %you select the door! % SWITCH STRATEGY if(i == a(1)) cars=cars+1; %a(2) opened, switch to a(3), car! elseif (i == a(2)) cars = cars + 1 ;%a(1) opened, switch to a(3), car! else cars = cars + 0; %a(1)/a(2) opened, switch to a(2)/a(1), no car! end; % NO SWITCH STRATEGY if(i == a(1)) carn = carn + 0; %a(2) opened, no car elseif (i==a(2)) carn = carn + 0; %a(1) opened, no car else carn = carn + 1; %a(1) or a(2) opened, car! end end; disp('probability to win a car if switched doors='); disp(num2str(cars./stats)); %# of cars with switching disp('probability to win a car if did not switch doors='); disp(num2str(carn./stats)); %# of cars w/o switching

35 Credit: XKCD comics