STAT100 Module 4. Detecting abnormalities. Dr. Matias Salibian-Barrera Winter 2009 / 2010

Transcription

1 STAT100 Module 4 Detecting abnormalities Dr. Matias Salibian-Barrera Winter 2009 / 2010

2 Contact information Subject: STAT100 Subject: Hi Subject: Subject: Question LSK

3 About me

4 Motivation Much effort is devoted to modeling normal events Typical behaviour, average values Means, trends... In some cases, the interesting things are the different or atypical ones In other cases, they are the ones unlikely to have happened

5 Module learning outcomes By the end of this module you will be expected to: determine whether you have enough information to recognize a particular event as atypical or very unlikely to occur ; be able to determine whether an event may be classified as atypical or not; and be able to interpret what the occurrence of such an event would mean (or imply) in practical terms.

6 In today's lecture you can expect... Overview of different aspects of the problem Overview of different types of atypical events Some short detours Motivating example in some detail

7 Different types of atypical events Contradict completely the established model ( Black Swan -type) Contradict an assumption of the established model (if you are just guessing you shouldn't get too many correct answers...)

8 Different types of atypical events Deviate from predictions of the model (number of wins in lottery games) Deviate from historical trends (disease outbreaks)

9 Contradict the model... The Black Swan The Impact of the Highly Improbable by Nassim Taleb The beast in this book is not just the bell curve and the self-deceiving statistician (...) It is the drive to focus on what makes sense to us. (...) Note that I am not (...) collecting selective corroborating evidence (...) Anyone looking for confirmation will find enough of it to deceive himself and no doubt his peers.

10 These are easy to detect But hard to interpret! May have a very large impact Induce questions: What knowledge can be proved? What role does statistical evidence play?

11 Advancing knowledge is not easy Popper Falsifiability Kuhn Scientific revolutions (paradigm shifts) Lakatos tried to reconcile Popper & Kuhn Feyerabend Epistemological anarchism...

12 Although they are important we won't deal with these questions here

13 Different types of atypical events

14

15 Different types of atypical events Unauthorized user Unauthorized activities Accessing unauthorized files

16

17

18 Different types of atypical events Duplicate cards Unusual transactions On-line (real time) fraud detection

19

20

21 Different types of atypical events Unauthorized long-distance calls Probably not a big deal anymore?

22

23

24 Google Flu trends US CANADA Google Official Data

25 Different types of atypical events Unusually large case counts Atypically clustered cases Abnormal infection rate (certain population / time)

26

27

28

29

30

31 Different types of atypical events Atypical outcome Unusual number of winners in one draw Unusual sequence of winning numbers

32

33

34 Different types of atypical events Unusual number of prize claims from one group

35 Contradicting assumptions of the model

36 Contradicting model assumptions Extrasensory perception An experiment to check whether person is just guessing or not If a person was guessing, what is the probability that he / she would correctly guess 10 randomly chosen coin-toss outcomes?

37 Extrasensory perception The experiment: (Coin 1) (Coin 2) (Coin 10) Possible guesses: (Head) (Head) (Head) (Head) (Head) (Tail) (Tail) (Tail) (Tail) 1028 possibilities

38 Only 1 answer out of 1028 is the correct one P( guessing correctly ) = 1 / % A perfect score is very unlikely to be attained by chance A perfect score is highly unexpected and would question our assumption that the person is just guessing...

39 Contradicting assumptions... If the person did not have extrasensory perception, then correctly guessing 10 coins would not be expected to happen So that if we observe such an unexpected outcome (10 correct guesses), we will have to question our assumption that the person did not have extrasensory powers...

40 However, if we only used 3 coins... P( guessing correctly 3 out of 3 ) = 1 / 8 12% A perfect score can still be attained by chance fairly often With only 3 coins we can't distinguish fake from real visionaries Because a perfect score is not unexpected

41 Questioning assumptions AKA Testing of hypotheses Is an important part of science progress Underlies many discoveries Allows science to move forward ( evolve ) Statistics is heavily involved in this process

42 We need to agree on what events can be verified independently Experiments (and the statistical analysis of their results) can be used to check the reasonableness of some assumptions

43 This is not always the case... But astrology cannot be "proven" or falsified by random statistics because astrology is based on the premise that conditions are never random ( ) what if you're testing whether Pisces is less aggressive than Aries and it so happens that Mars is rising during the time of the test? Debunking the Debunkers: Lessons to Be Learned - by Valerie Vaughan

44 Designed vs. observational studies Sometimes you cannot design an experiment Not ethical Smoking and lung cancer MMR vaccine and autism spectrum Not feasible Alzheimer's / dementia disease diagnosis post-mortem

45 Designed vs. observational studies However, you can still use sound statistical methodology to analyse these data The conclusions that can be reached are different

46 Deviations from historical observations

47 Atypical observations / events What makes an event atypical? It is far from what is expected (not just large )

48 Atypical observations / events Need a frame of reference (what is typical ) Need to quantify distances between events Examples of events: Disease counts Credit card transactions Activity in computer systems

49

50 Averages don't tell the whole picture Consider these (artificial) counts: 3, 4, 3, 4, 3, 3, 4, 4, 4, 3 Average = ( )/10 = 3.5 Typical value = 3.5 Does 7 appear to be atypical?

51 DATA POINTS AVERAGE 7

52 Averages don't tell the whole picture Consider these (artificial) counts: 7, 3, 8, 7, 2, 0, 0, 5, 2, 1 Average = ( )/10 = 3.5 Typical value = 3.5 Does 7 appear to be atypical?

53 DATA POINTS AVERAGE 7

54 However... the distance from a particular event to the norm needs to be compared with the typical distances to the norm

55 Atypical observations / events What makes an event atypical? An atypical event is further than usual from what is typical (not just large or far )