Introduction to Medical Computing

Size: px

Start display at page:

Download "Introduction to Medical Computing"

Della Murphy
5 years ago
Views:

1 CS Introduction to Medical Computing Stephen M. Watt

2 Artificial Intelligence in Medicine Notes by Jacquelyne Forgette, March University of Western Ontario CS Stephen M. Watt

3 AIM Artificial Intelligence in Medicine Introduction to Artificial Intelligence Motivate AI for medical use Machine learning techniques Testing Procedures Applications Summary

4 What is it? Why is it great? What has been done?

5 Artificial Intelligence Homo sapiens - man the wise For thousands of years, we have tried to understand how we think. Psychology, philosophy, neuroscience, sociology... Beyond how we think... We want to build intelligent entities. Artificial intelligence S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 2010.

6 Artificial Intelligence What characteristics would an intelligent entity exhibit? What makes humans intelligent?

7 Ex: Sherlock Holmes The Science of Deduction This is what I do: 1. I observe everything. 2. From what I observe, I deduce everything. 3. When I've eliminated the impossible, whatever remains, no matter how mad it might seem, must be the truth.

8 Characteristics suggested in class Deductive reasoning Emotion Self aware Inductive reasoning Path planning Altruistic behaviour Abstract organization of thoughts Plan actions according to goals Recognize objects learning

9 Artificial Human Logic: Deduction if a then b Induction generalize from examples Rationality choose actions that achieve the best outcome Inference deriving logical conclusions from a premise that assumed to be true Planning set goals by looking to the future achieve goals Perception Social contacts warp perception Personality can warp perception Motion and manipulation Manipulating objects Learning Localization Navigation Knowledge representation (commonsense) The amount of useful and useless facts any person has knowledge of is astronomical Natural language processing Understand a question Carry a conversation Social intelligence Social etiquette Social networks (not Facebook) Creativity Find a solution outside of the conventional problem domain Create art and music Computer vision Know what you re seeing S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 2010.

10 Acting Humanly The Turing Test Proposed by Alan Turing in 1950 Operational definition of intelligence Computer will pass the test if a human interrogator cannot tell whether written responses to questions are from a human or a computer. Natural language processing Knowledge representation Reasoning Learning S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 2010.

11 Further Turing Test states that physical simulation of a person is irrelevant to intelligence. Total Turing Test adds that a machine would need Computer vision to perceive objects Robotics to manipulate objects and move about its environment S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 2010.

12 History Work in AI started after WW II, and the name itself was coined in 1956 (Russell, 2010) It was astonishing whenever a computer accomplished anything remotely clever. The novelty of a new idea Reality set in. Promising systems would fail miserably on more difficult problems. Used weak methods based on general-purpose mechanisms. Could not scale to larger problems. S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 2010.

13 History Knowledge based systems. Uses more powerful methods based on domain specific knowledge. If you give the computer domain specific knowledge, it would be able to handle more typical cases in a narrow area of expertise. S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 2010.

14 MYCIN (Early 1970s) Expert system created to diagnose blood infections. It contained 450 rules that were carefully crafted by expert medical professionals, textbooks and case studies. The rules also had certainty factors, that seemed to reflect how doctors naturally assessed the impact of evidence on a diagnosis. It performed as well as some experts and considerably better than junior doctors. Was never used in practice Required around 30 minutes of the user answering questions before it gave an answer. S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 2010.

15 CADUCEUS (1980s) An improvement on MYCIN with a broader field of diagnosis. Expert system for medical diagnosis. Extensive knowledge base. Created by Harry Pople and Jack Meyers

16 History 1980-Present AI became an industry Present AI went scientific. Old theories are expanded, tested, analyzed and rigorously proven Present Re-emergence of intelligent agents. Interested in a general agent, rather than specific tasks Present Availability of large data sets. New applications of AI because of available data, not new techniques. S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 2010.

17 State of the Art AI Robotic Vehicles Speech Recognition Autonomous planning and scheduling Game Playing Spam Filtering Logistics Planning Robotics Machine Translation Many more... S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 2010.

18 State of the Art - Watson Watch more about Watson:

19 Assistance, insight, discovery...

20 Why is AI important for medicine? Automatically do tasks that a human would do in any aspect of the medical profession. Better results Shorter amount of time Do mundane tasks Provide insights Save money! Save patients!

21 Adaptation, pattern discovery...

22 Machine Learning Sub-field of AI that is concerned with learning. Find patterns Adapt to new unforeseen situations Learn from mistakes Draw new conclusions Rule based systems have no learning, so they are only as good as the rules programmers include. Igor Kononenko, 2001

23 Thinking learning and remembering Humans don t just think, we learn from our mistakes and our behaviour changes as the situation changes. Machine learning techniques are used to learn a system that makes good decisions by remembering previous outcomes and generalizing to unknown situations.

24 Adaptation, pattern discovery...

25 Decision Tree Learning Most widely used method for inductive inference. Induction Getting a general idea from specific examples. It approximates discrete valued functions. Capable of learning disjunctive [or] expressions. Robust to noisy data. Method to solve the classification problem Igor Kononenko, 2001 and Mitchell, 1997

26 Make a decision Given this information: Outlook is sunny, Temperature is Hot Humidity is High Wind is Strong Question Should we play tennis? Yes or No? Mitchell, 1997

27 Decision tree Attribute Mitchell, 1997 Tree takes an instance, and classifies it: yes or no.

28 Decision tree Values Mitchell, 1997 Tree takes an instance, and classifies it: yes or no.

29 Decision tree Leafs of the tree indicate the resulting classification. Mitchell, 1997 Tree takes an instance, and classifies it: yes or no.

30 Make a decision Given this information: Outlook is sunny, Attribute Temperature is Hot Humidity is High Wind is Strong An attribute has one value, from a set of values: {Sunny, Overcast, Rain} Question Should we play tennis? Yes or No? Given an instance, we want to classify it into one of two classes, yes or no. Classification problem. Mitchell, 1997

31 Decision tree Given an instance, start at the top of the tree, and work downwards. Outlook is sunny, Temperature is Hot Humidity is High Wind is Strong Classification is NO Tree takes an instance, and classifies it: yes or no. Mitchell, 1997

32 Disjunction of conjunctions (Outlook = Sunny AND Humidity = Normal) OR (Outlook = Overcast) OR (Outlook = Rain AND Wind= Weak) OR AND Mitchell, 1997

33 Decision Trees Must Be Built A tree is constructed from training examples. Once tree is constructed, we are able to classify new instances, that were not in the examples. Effectiveness of the tree depends on the correctness of the new classification.

34 Building a Decision Tree Tree is built by using a set of training examples In this case, each example contains Attribute values <wind=strong,... > Desired result (yes or no)

35 Training Data Day Outlook Temperature Humidity Wind Play Tennis? D1 Sunny Hot High Weak No D2 Sunny Hot High Strong No D3 Overcast Hot High Weak Yes D4 Rain Mild High Weak Yes D5 Rain Cool Normal Weak Yes D6 Rain Cool Normal Strong No D7 Overcast Cool Normal Strong Yes D8 Sunny Mild High Weak No D9 Sunny Cool Normal Weak Yes D10 Rain Mild Normal Weak Yes D11 Sunny Mild Normal Strong Yes D12 Overcast Mild High Strong Yes D13 Overcast Hot Normal Weak Yes D14 Rain Mild High Strong No Mitchell, 1997

36 Building Decision Trees What attribute is the most important? What attribute should be at the top of the tree? From this set of training examples, we want to build a decision tree that will fit the training data. Fits the training data if each example is tested on the tree and the classification result is identical to the training example. Mitchell, 1997

37 Information Theory Entropy is a measure of the (im)purity in a collection of training examples Entropy is 0 if all examples belong to the same class Entropy is 1 if each class contains an equal number of examples If the collection contains unequal number of examples in each class, the entropy will be between 0 and 1. Mitchell, 1997

38 Entropy Entropy is a measure of the (im)purity in a collection of training examples A collection S, and the number of possible classifications c. Where pi is the proportion of the ith classification in S.

39 Entropy Half of the classifications are positive and half are negative. Entropy = 1 The entropy function relative to a Boolean classification, as the proportion of positive examples, P varies between 0 and 1. None of the classifications are positive. Entropy = 0 All of the classifications are positive. Entropy = 0 Mitchell, 1997

40 Example (Tennis) S is a collection of 14 examples of either yes or no for playing tennis on a Saturday. [9+,5-] = 0.940

41 [9+,5-] 9 positive results and 5 negative. Day Outlook Temperature Humidity Wind Play Tennis? D1 Sunny Hot High Weak No D2 Sunny Hot High Strong No D3 Overcast Hot High Weak Yes D4 Rain Mild High Weak Yes D5 Rain Cool Normal Weak Yes D6 Rain Cool Normal Strong No D7 Overcast Cool Normal Strong Yes D8 Sunny Mild High Weak No D9 Sunny Cool Normal Weak Yes D10 Rain Mild Normal Weak Yes D11 Sunny Mild Normal Strong Yes D12 Overcast Mild High Strong Yes D13 Overcast Hot Normal Weak Yes D14 Rain Mild High Strong No

42 Information Gain Information gain is used to measure how well an attribute separates the training examples according to their target classification Where Values(A) is the set of all possible values for the attribute A, and Sv is the subset of the collection S for which attribute A has value v. Mitchell, 1997

43 Example Gain(S, Wind) We want to know how wind strength separates the training data. We calculated this already for the Entropy example Values(A)={Strong, Weak} Because A is wind Mitchell, 1997

44 Need to find out the values of Sweak and Sstrong and their corresponding entropy values.

45 Sweak = [6+,2-] Day Outlook Temperature Humidity Wind Play Tennis? D1 Sunny Hot High Weak No D2 Sunny Hot High Strong No D3 Overcast Hot High Weak Yes D4 Rain Mild High Weak Yes D5 Rain Cool Normal Weak Yes D6 Rain Cool Normal Strong No D7 Overcast Cool Normal Strong Yes D8 Sunny Mild High Weak No D9 Sunny Cool Normal Weak Yes D10 Rain Mild Normal Weak Yes D11 Sunny Mild Normal Strong Yes D12 Overcast Mild High Strong Yes D13 Overcast Hot Normal Weak Yes D14 Rain Mild High Strong No

46 Sstrong = [3+,3-] Day Outlook Temperature Humidity Wind Play Tennis? D1 Sunny Hot High Weak No D2 Sunny Hot High Strong No D3 Overcast Hot High Weak Yes D4 Rain Mild High Weak Yes D5 Rain Cool Normal Weak Yes D6 Rain Cool Normal Strong No D7 Overcast Cool Normal Strong Yes D8 Sunny Mild High Weak No D9 Sunny Cool Normal Weak Yes D10 Rain Mild Normal Weak Yes D11 Sunny Mild Normal Strong Yes D12 Overcast Mild High Strong Yes D13 Overcast Hot Normal Weak Yes D14 Rain Mild High Strong No

47 Entropy of Sweak

48 Calculate the Gain of Wind All the values were equally separated making entropy 1 Sstrong = [3+,3-] Mitchell, 1997

49 Information gain for all 4 attributes Outlook attribute provides the best prediction of the target attribute, PlayTennis, given these training examples. Outlook becomes the root node. Gain(S, Outlook) = Gain(S, Humidity) = Gain(S, Wind) = Gain(S, Temperature) = Mitchell, 1997

50 After first separation What attribute should go here? Mitchell, 1997

51 Sunny = [2+,3-] Day Outlook Temperature Humidity Wind Play Tennis? D1 Sunny Hot High Weak No D2 Sunny Hot High Strong No D3 Overcast Hot High Weak Yes D4 Rain Mild High Weak Yes D5 Rain Cool Normal Weak Yes D6 Rain Cool Normal Strong No D7 Overcast Cool Normal Strong Yes D8 Sunny Mild High Weak No D9 Sunny Cool Normal Weak Yes D10 Rain Mild Normal Weak Yes D11 Sunny Mild Normal Strong Yes D12 Overcast Mild High Strong Yes D13 Overcast Hot Normal Weak Yes D14 Rain Mild High Strong No

52 Overcast = [4+,0-] Day Outlook Temperature Humidity Wind Play Tennis? D1 Sunny Hot High Weak No D2 Sunny Hot High Strong No D3 Overcast Hot High Weak Yes D4 Rain Mild High Weak Yes D5 Rain Cool Normal Weak Yes D6 Rain Cool Normal Strong No D7 Overcast Cool Normal Strong Yes D8 Sunny Mild High Weak No D9 Sunny Cool Normal Weak Yes D10 Rain Mild Normal Weak Yes D11 Sunny Mild Normal Strong Yes D12 Overcast Mild High Strong Yes D13 Overcast Hot Normal Weak Yes D14 Rain Mild High Strong No

53 Rain= [3+,2-] Day Outlook Temperature Humidity Wind Play Tennis? D1 Sunny Hot High Weak No D2 Sunny Hot High Strong No D3 Overcast Hot High Weak Yes D4 Rain Mild High Weak Yes D5 Rain Cool Normal Weak Yes D6 Rain Cool Normal Strong No D7 Overcast Cool Normal Strong Yes D8 Sunny Mild High Weak No D9 Sunny Cool Normal Weak Yes D10 Rain Mild Normal Weak Yes D11 Sunny Mild Normal Strong Yes D12 Overcast Mild High Strong Yes D13 Overcast Hot Normal Weak Yes D14 Rain Mild High Strong No

54 Outlook is Sunny With respect to the fact that the outlook must be sunny Mitchell, 1997

55 Building The Tree Keep calculating Gain and extending the tree until all braches lead to (yes or no) or until there are no more attributes.... Mitchell, 1997

56 Adaptation, pattern discovery...

57 Bayesian Theory Example 1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get a positive mammography. 9.6% of women without breast cancer will also get a positive mammography. Question: A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?

58 Bayesian Theory We know that 1% of women in this age group who get regular screenings have cancer. Now we know that the test was positive, how much does the probability of cancer increase? 1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get a positive mammography. 9.6% of women without breast cancer will also get a positive mammography.

59 Bayesian Theory Question: Question: A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer? -OR- If 10,000 women in this age group undergo a routine screening, about what fraction of women with a positive mammography will actually have breast cancer? 1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get a positive mammography. 9.6% of women without breast cancer will also get a positive mammography.

60 Example with 10,000 women

63 Total of =1030 Women had a positive mammography

65 Bayes Theorem The original proportion of patients with breast cancer is known as the prior probability... P(A) The chance that a patient with breast cancer gets a positive mammography, and the chance that a patient without breast cancer gets a positive mammography, are known as the two conditional probabilities... P(B A) and P(B ~ A) Mitchell, 1997

66 Bayes Theorem Provides ways to calculate posterior probability P(A B) from prior and conditional probabilities. Mitchell, 1997

67 Mathematically P(A) = 0.01 P(~A) = 0.99 P(B A) = 0.8 a woman has cancer a woman doesn't have cancer positive mammography given that she does have cancer. P(B ~ A) = positive mammography given that she doesn't have cancer

68 Calculate new probability

69 Naive Bayes Classifier Bayesian method was developed to tackle the classification problem (remember the decision tree example) Learning task consists of a set of instances x, that are described by a conjunction of attribute values and the target function f(x) can take on any value from some finite set V. (same as decision tree) Mitchell, 1997

70 Naive Bayes Classifier We have a set of training examples Then, a new instance is given by a tuple <a1, a2,..., an> The learner must predict the target value of the given instance, by using the training examples. Mitchell, 1997

71 Naive Bayes Classifier Where denotes the target value output. V = {Yes,No} For each possible classification, calculate the probability that the instance belongs in that target category. Mitchell, 1997

72 Example New instance: <Outlook=sunny, Temperature=cool, Humidity=high, Wind=strong> Now we need to estimate the 10 needed probabilities from the training data. Mitchell, 1997

73 Estimate probabilities from training data P(PlayTennis = yes) = 9/14 = 0.64 P(PlayTennis = no) = 5/14 = 0.36 P(Wind = strong PlayTennis = yes) = 3/9 = 0.33 P(Wind = strong PlayTennis = no) = 3/5 = 0.60 P(Humidity = high PlayTennis = yes) = 3/9 = 0.33 P(Humidity = high PlayTennis = no) = 4/5 = 0.8 P(Temperature = cool PlayTennis = yes) = 3/9 = 0.33 P(Temperature = cool PlayTennis = no) = 1/5 = 0.2 P(Outlook = sunny PlayTennis = yes) = 2/9 = 0.22 P(Outlook = sunny PlayTennis = no) = 3/5 = 0.6 Mitchell, 1997

74 Mitchell, 1997 P(Wind = strong PlayTennis = yes) = 3/9 = 0.33 Day Outlook Temperature Humidity Wind Play Tennis? D1 Sunny Hot High Weak No D2 Sunny Hot High Strong No D3 Overcast Hot High Weak Yes D4 Rain Mild High Weak Yes D5 Rain Cool Normal Weak Yes D6 Rain Cool Normal Strong No D7 Overcast Cool Normal Strong Yes D8 Sunny Mild High Weak No D9 Sunny Cool Normal Weak Yes D10 Rain Mild Normal Weak Yes D11 Sunny Mild Normal Strong Yes D12 Overcast Mild High Strong Yes D13 Overcast Hot Normal Weak Yes D14 Rain Mild High Strong No

75 Mitchell, 1997 Example (cont.)

76 Example (cont.) The bayes classifier assigns the value PlayTennis=no to this new instance Normalization gives the conditional probability that the target is no, given the observed attribute values. Given the instance we are 79.5% sure that we should not play tennis. Mitchell, 1997

78 Over-fitting Generalization is very important to correctly classifying previously unknown instances. Over-fitting exists because the criterion used for training the model is not the same as the criterion used to judge the efficiency the model. Mitchell, 1997

79 Prevent Over-fitting in DT If over-fitting occurs, the system will effectively memorize the training data, and not be able to generalize to new instances. Techniques exist for preventing over-fitting Early stopping stop growing the tree at a certain depth Pruning For decision trees, you want to get rid parts of the tree that don t really matter for the classification. Mitchell, 1997

80 Assessing a Classification System With problems of over-fitting, noisy and incomplete data, it is necessary to assess the effectiveness of a learned system. Can only verify the correctness of a classification, when the correct classification is already known. Cross-validation uses a portion of the training data to validate the method Mitchell, 1997

81 Cross-Validation

83 AIM in clinical practice Acute care systems Decision support systems Educational systems Laboratory systems Medial Imaging Quality assurance and administration

84 Clinical Decision Support System CDSS/CDS Computer software that is designed to provide assistance for physicians and other health professionals in regards to decision making tasks. Clinical DSSs are typically designed to integrate a medical knowledge base, patient data and an inference engine to generate case specific advice.

85 Medical Diagnosis Requirements Good performance Performance of physicians can be used as a lower bounds to measure performance of algorithms Deal with missing data Patient records may lack data Deal with noisy data Data has uncertainty and errors Transparency of diagnostic knowledge Physicians should be able to analyze and understand the generated knowledge Ability to explain decisions Given an unexpected answer, the physician will require additional explanation in order to seriously consider the system s suggestion Reduce the number of tests needed Tests are costly and time consuming, the system should be able to reliably diagnose a patient based on limited amount of data Igor Kononenko, 2001 and Mitchell, 1997

86 Performance Comparison of ML Methods Classifier Performance Transparency Explanation Reduction Missing data Decision Trees Good Very Good Good Good Acceptable Naive Bayes Very Good Good Very Good No Very Good Neural Network Very Good Poor Poor No Acceptable K-NN Very Good Poor Acceptable No Acceptable The Naive Bayes classifier, though simple, is one of the best classification methods for medical diagnosis. Results extrapolated from Igor Kononenko, 2001 and Mitchell, 1997

87 Future Trends in AIM Looking at the reliability of a single prediction instead of overall reliability of the method. Machine learning in complementary medicine, to verify some unexplained phenomena. Evaluate the effect of different t-shirts on the human bio-electromagnetic (BEM) field. Evaluate the effect of cell phones on the human BEM field.

89 Summary AI is the study and design of intelligent agents AIM is the application of AI in any aspect of the medical field Machine Learning techniques are used when learning (adapt to new information) is needed Decision Trees and the Naive Bayes classifier are methods used to solve classification problems

90 Summary Classification Given an instance, need to predict in what class it belongs. An instance consists of set of attributes (Windy, Outlook, Temperature, Humidity...) each with a finite set of values (Windy={Weak, Strong}, Humidity={High, Normal}...) Training data includes a set of instances with their correct classifications.

91 Summary Validation is the stage in which the learned system is tested. AI is used in Acute care systems Decision support systems Educational systems Laboratory systems Medial Imaging Quality assurance and administration

92 T.M. Mitchell. Machine Learning. McGraw-Hill Series in Computer Science. McGraw-Hill, S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall Series in Artificial Intelligence. Prentice Hall, Igor Kononenko. Machine learning for medical diagnosis: history, state of the art and perspective. Aritificial Intelligence in Medicine, 23(1):89 109, 2001 "Decision support systems" March <

Lazy Associative Classification

Lazy Associative Classification By Adriano Veloso,Wagner Meira Jr., Mohammad J. Zaki Presented by: Fariba Mahdavifard Department of Computing Science University of Alberta Decision Tree (Eager) Associative