Biomedical Machine Learning

Size: px

Start display at page:

Download "Biomedical Machine Learning"

Kellie Hodge
5 years ago
Views:

1 Marco F. Ramoni, PhD Children s Hospital Informatics Program Harvard Medical School Dental Informatics and Dental Research Making The Connection NIH, Bethesda, MD June 12-13, 2003

2 Machine Learning Artificial Intelligence: Mechanical simulation/emulation of functions considered proper of human beings: Reasoning: Derive conclusions and represent knowledge; Vision: Processing and recognition of images; Natural Language: Ability to parse natural language; Machine Learning: Automated learning from data. Machine Learning: Became popular with Bioinformatics; The less I know, the more I learn: supervised vs unsupervised; Know the structure of your domain and want to learn; You know the structure and you don t know what to learn; You don t know the structure and you know what to learn; You don t know the structure and you don t know what to learn.

Rev. Thomas Bayes Name: Thomas Bayes. Birth: 1702, London, UK. Death: 1761, Tunbridge, Kent. Buried: Bunhill Fields, London. Job title: Presbyterian minister.

3 Rev. Thomas Bayes Name: Thomas Bayes. Birth: 1702, London, UK. Death: 1761, Tunbridge, Kent. Buried: Bunhill Fields, London. Job title: Presbyterian minister. Publications: 3 (1 posthumous). Claim to fame: Essay Towards Solving a Problem in the Doctrine of Chances, Phil Trans Royal Soc, 1763, by Richard Price. Trivia: Unrelated to Bayesians.

4 Bayesian Phenotype Bayesian: The opposite of Frequentist. Frequentist: Every (at least most) statistician you know. Population: varieties of Bayesians (Good, 1971). Profile: Hates the cookbook style of classical statistics. Tools: Total Probability Theorem and Bayes Theorem. Main Beliefs: Inference is about decisions. Inference is conditional on data (small samples); Inference is updating prior knowledge (even when it is knowledge of not knowing anything); Everything (also an hypothesis) has a probability.

5 Bayesian Networks Qualitative: A dependency graph made by: Node: a variable X, with a set of states {x 1,,x n }. Arc: a dependency of a variable X on its parents Π. Quantitative: The distributions of a variable X combination of states π i of its parents Π. given each A p(a) p(a) Y 0.3 Y 0.3 O 0.7 O 0.7 E p(e) p(e) L 0.8 L 0.8 H 0.2 H 0.2 A E A=Age Age; ; E=Education Education; ; I=Income Income I A E I I p(i A,E) p(i A,E) Y L L 0.9 Y L L 0.9 Y L H 0.1 Y L H 0.1 Y H L 0.5 Y H L 0.5 Y H H 0.5 Y H H 0.5 O L L 0.7 O L L 0.7 O L H 0.3 O L H 0.3 O H L 0.2 O H L 0.2 O H H 0.8 O H H 0.8

6 Learning Bayesian Networks Bayesian networks were developed to encode experts knowledge. Late 80s, people realize that the statistical nature of Bayesian networks allows us to learn them from data. In principle, the process of learning a Bayesian network structure involves: Search strategy to explore the possible structures; Scoring metric to select a structure. Courtesy Enrico Motta

7 Supervised Classification Components: Dataset of features and a training signal. Features: Gene expression levels in different classes. Training signal: The class label. Task: Create model associating feature profiles to class. Drawback: Intractable (combinatorial explosion). Other methods: Neural networks, support vector machines, etc. C G1 G2 G2 G4 G5 G6

8 Naïve Bayes Classifiers Assumption: A simple classifier assumes that the features (genes, symptoms) are conditionally independent given the class. Classifiers: These models are called Naïve Bayes Classifiers. Method: Estimate, for each feature (gene/symptom), the conditional probability distribution of the feature given each class. Toward Unsupervision: Select best predictors (feature selection). C G1 G2 G3 G4 G5 G5

9 Supervised vs Unsupervised Elements: Features (symptoms genes) and a training signal (disease, class). Question: Which function best maps features to class? Goal: Find a good predictive system of class (a system that takes patient symptoms and return a diagnosis). Task: Estimation (except for feature selection, the task of finding the best predictors). Elements: Features (symptoms genes) but no training signal. Question: Which features behave in a related (similar) way across conditions? Goal: Understand interaction (e.g. how genes/symptoms behave similarly under some experimental conditions). Task: Model selection.

10 Feature Selection Task: Identify those genes that best predict the class: Advantage I: Typically increases predictive accuracy. Advantage II: More compact representation. Advantage III: Provide insights into the process. Type of Task: Model selection. Rationale: Since we cannot try all combinations, most different features should be the best at discriminating. C G1 G2 G3 G4 G5 G5 G6

11 Comparative Experiments Healthy cell Tumor cell Sample 1 Sample 2 Sample 3 Sample 4 Samples k=1,,n i genes g=1,,g Gene.DescS1 S2 S3 S4 S5 AFFX-BioC hum_alu_a AFFX-DapX AFFX-LysX AFFX-HUM AFFX-HUM AFFX-HUM AFFX-HUM AFFX-HUM AFFX-HUM AFFX-HUM AFFX-HUM AFFX-HSA y gik

12 Bayesian Differential Analysis Bayesian solution: Compute the probability that the ratio between the expression of a gene in condition A and B is higher than 1. Reproducibility: 8 groups of 4 data sets each, of different sample sizes, selected from 102 samples of normal vs prostate cancer. with P. Sebastiani, BU

13 Slippery Class Survival Class in Ovarian Cancer: Task: Identify patient subclass with favorable prognosis; BADGE: Bayesian Analysis of Differential Gene Expression. Overall Recurrence Survival (DSF) p < with Cannistra and Spentzos, BIDMC

14 Clustering Condition: The class is so slippery that we don t have a shred of information about it. Goal: Create a classification. Method: Group samples in clusters according to their similarity (eg correlation). Example: Find a classification for 20,000 rock samples from Mars. Example II: Oral cancer (NIDCR): Identify phenotypes using their genomic profile. C? G1 G2 G3 G4 G5 G5

15 Bayesian Clustering Problem: How do we decide that N patients/genes are sufficiently similar become a cluster on their own? Similarity: Profiles are similar when they are generated by the same stochastic process. Example: EKGs are similar but not identical series generated by the a set of physiological process. Clusters: Cluster profiles on the basis of their similarity is to group profiles generated by the same process. Bayesian solution: The most probable set of generating processes responsible for the observed profiles. Strategy: Compute posterior probability p(m D) of each clustering model given the data and take the highest.

16 Modeling Dynamics Time: Control happens along time. Example: Development of normal/cleft palate. Correlation: Assumes data to be independent: same score on permuted data. The past dictates the future. Information: Is information enough to discriminate? Method: Cluster Analysis of Gene Expression Dynamics (CAGED) Correlation=0.57 Correlation=0.72

17 The Clustering Model We need to merge these separated models in a single global model of the processes. We add a variable indexing the series and we build a single conditional probability distribution. C X y y = Xβ + ε

18 CAGED We use a dataset of 512 genes collected to study the response to fibroblast to serum (Iyer et al. 1999). Back in 1999, 238 out of 517 genes were unknown. We relabeled the genes according to the current state of the art and less than 20 are left unknown. There are 19 repeated genes in the dataset: Original model puts 4 of these in different clusters; We put 1 of these in two different clusters. Ramoni Sebastiani Kohane, PNAS (2002)

19 Decoding Control Clustering: Group samples but tells us nothing about the control mechanisms: it provides boxes, not chains of command. Class: It s so slippery that it does not even exist. Dependencies: Bayesian networks are built to represent control: a set of links means that a variable depends on a set of variables. Learning: Automated learning a Bayesian network structure: Search strategy to explore the possible structures; Scoring metric to select a structure. Challenges: Limitations of current methods: Distributions: Only discrete and normal distributions; Dependencies: Only linear (for normal distribution). Solution: Generalized Gamma Networks (GGNs). with P. Sebastiani, BU

20 Structure Learning SNPs: Single nucleotide polymorphisms - change of a single base in the DNA - work as markers of genomic regions. Diabetes network: 13 genes and one phenotype (IDDM) in triads.... ATGCGATACTCGACTCGA ATGCGATACGCGACTCGA... Data by J Hirshorn, WI and CH

21 Finding The Right SNPs Background: Oral mucositis (OM) is a a dose-limiting, toxicity experienced by cancer patients receiving radiotherapy or chemotherapy. Objective: Identify a set of sentinel polymorphisms able to predict individual risk of OM in hematopoietic stem cell transplant (HSCT) recipients as result of antineoplastic therapy. Subjects: Allo (70) and auto HSCT (50) recipients and donors (70). Candidate Genes: Identify a set of genes and regulatory regions. Candidate SNPs: Identify coding SNPs in candidate genes and SNPs in regulatory regions of interest. Challenge: It is likely to be a complex trait. with S Sonis, BWH and HSDM

22 Differential Analysis Example: Take the genes differentially expressed in the prostate cancer database and learn the structure of each condition (normal vs tumor) : collagene is independent of all other nodes, and becomes a child of 914 (oncogene with transcription regulation functions). Normal Samples Tumor Samples

23 Phenomic Landscapes with D Shen, Harvard/MIT HST

BADGE Bayesian Analysis of Differential Gene Expression http://genomethods.org/badge Take Home Toys CAGED Cluster Analysis of Gene Expression Dynamics http://genomethods.

24 BADGE Bayesian Analysis of Differential Gene Expression Take Home Toys CAGED Cluster Analysis of Gene Expression Dynamics Bayesware Discoverer A program to learn Bayesian networks from data Contact: Marco Ramoni, Children s Hospital Informatics program marco_ramoni@harvard.edu

UNLOCKING VALUE WITH DATA SCIENCE BAYES APPROACH: MAKING DATA WORK HARDER

UNLOCKING VALUE WITH DATA SCIENCE BAYES APPROACH: MAKING DATA WORK HARDER 2016 DELIVERING VALUE WITH DATA SCIENCE BAYES APPROACH - MAKING DATA WORK HARDER The Ipsos MORI Data Science team increasingly