SCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA Sistemi di Elaborazione dell Informazione Introduzione Ruggero Donida Labati Dipartimento di Informatica via Bramante 65, 26013 Crema (CR), Italy http://homes.di.unimi.it/donida ruggero.donida@unimi.it
Outline del corso Introduzione all intelligenza computazionale. Analisi continua dei dati: regressione (regressione polinomiale, regressione lineare Bayesiana, reti neurali artificiali). Analisi discreta dei dati: classificazione (metodi di apprendimento supervisionato, k-nearest neighbors, reti neurali artificiali), clusterizzazione (metodi di apprendimento non supervisionato, reti neurali artificiali). Riduzione della dimensionalità (principal component analysis, independent component analysis, metodi di selezione delle caratteristiche). Applicazioni in ambiente Matlab.
Modalità d esame Discussione di un articolo scientifico che utilizza tecniche di intelligenza computazionale nell ambito della fisica medica. L articolo scientifico deve essere concordato con il docente. Progetto da concordare con il docente.
Sistemi di elaborazione dell informazione
Esempi applicativi (1/3) Will respond to treatment Patient Blood sample Mass spectrometry Protormics profile Model Will not respond to treatment Cannot make decision Patient Blood sample Cell image Model Leukemia Helty patient
Esempi applicativi (2/3) Model Will survive for 3.5 year Patient Biopsy Gene expression profile Patient Biomarkers, clinical and demographic data Model Otpimal dosage is X/week
Esempi applicativi (3/3)
Introduzione all intelligenza computazionale
Computational intelligence Two Big Questions: How does a human mind work, and Can non-humans display an intelligent behavior? These questions are still unanswered (Essential English Dictionary, Collins, London, 2008) Intelligence is the ability to understand and learn things. 2 Intelligence is the ability to think and understand instead of doing things by instinct or automatically. Summarizing, intelligence: ability to learn and understand, to solve problems and to make decisions
Goal of computational intelligence The goal of computational intelligence (CI) is to make machines do things that would require intelligence if done by humans. One of the most significant papers on machine intelligence: Computing Machinery and Intelligence, written by Alan Turing Is there thought without experience? Is there mind without communication? Is there language without living? Is there intelligence without life? That is: Can machines think?
Turing Imitation Game (1/3) First phase: The interrogator, a man and a woman are each placed in separate rooms. The interrogator has to discover who is the man and who is the woman by questioning them. The man should deceive the interrogator that he is the woman, while the woman has to convince the interrogator that she is the woman.
Turing Imitation Game (2/3) Second phase: The man is replaced by a computer programmed to deceive the interrogator as the man did. If the computer can fool the interrogator as often as the man did, we may say this computer has passed the intelligent behaviour test.
Turing Imitation Game (3/3) The Turing test: By maintaining communication between the human and the machine via terminals, gives us an objective standard view on intelligence. The test itself is quite independent from the details of the experiment It can be conducted as a two- phase game, or even as a single-phase game (the interrogator needs to choose between the human and the machine) Turing believed that by the end of the 20th century it would be possible to program a digital computer to play the imitation game It took a bit more: on June 2014, a chatbox of the University of Reading (UK) fooled 33% of the judges into thinking it was Eugene Goostman, a 13-year-old Ukrainian boy
Regression Model Will survive for 3.5 year Patient Biopsy Gene expression profile Patient Biomarkers, clinical and demographic data Model Otpimal dosage is X/week
Regression examples Supervised learning
Regression: some approaches Polynomial curve fitting Statistical regression Feedforward neural networks Radial basis functions neural networks
Regression: polynomial curve fitting
Regression: feedforward neural networks
Classification Identifying to which of a set of categories a new observation belongs, on the basis of a training set of data Supervised learning
Classification examples Some our works Acute Limphoblastic Leucemia "healty cell" & "lymphoblast Wildfires "Smoke frame" & "not smoke frame" Wood 21 classes
Classification: pattern recognition
y Classification: 2D example Output space A=0 B=1 C=2 D=3 3 2.5 2 1.5 1 0.5 0-0.5 x
Classification: some approaches Linear classifier Quadratic classifier K-nearest neighbors Feedforward neural networks Support vector machines Deep learning
Deep learning
Dimensionality reduction The process of reducing the number of random variables under consideration, via obtaining a set of principal variables Feature selection and feature extraction Supervised or unsupervided
Dimensionality reduction: examples
Dimensionality reduction: some approaches Principal component analysis Linear discriminant analysis Indipendent component analysis Forward feature selection Backward feature selection
Clustering Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters).
Clustering: example
Clustering: some approaches K-means Self organized maps Fuzzy c-means
Other CI techniques: fuzzy logic (1/2) Fuzzy logic is a form of many-valued logic in which the truth values of variables may be any real number between 0 and 1. By contrast, in Boolean logic, the truth values of variables may only be the integer values 0 or 1. Fuzzy logic has been employed to handle the concept of partial truth, where the truth value may range between completely true and completely false. Furthermore, when linguistic variables are used, these degrees may be managed by specific (membership) functions.
Other CI techniques: fuzzy logic (2/2)
Other CI techniques: evolutionary algorithms
Suggested lectures I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, 2016, Book in preparation for MIT Press, http://www.deeplearningbook.org M.l Negnevitsky, Artificial Intelligence: A Guide to Intelligent Systems,2nd ed., Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2005. C. M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, Inc., New York, NY, USA, 1995.
Regression The following slides are from: C. M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, Inc., New York, NY, USA, 1995
Polynomial Curve Fitting
Sum-of-Squares Error Function
0 th Order Polynomial
1 st Order Polynomial
3 rd Order Polynomial
9 th Order Polynomial
Over-fitting Root-Mean-Square (RMS) Error:
Polynomial Coefficients
9 th Order Polynomial Data Set Size:
9 th Order Polynomial Data Set Size:
Regularization Penalize large coefficient values
Regularization:
Regularization:
Regularization: vs.
Polynomial Coefficients
Apples and Oranges Probability Theory
Probability Theory Marginal Probability Joint Probability Conditional Probability
Probability Theory Sum Rule Product Rule
The Rules of Probability Sum Rule Product Rule
Bayes Theorem posterior likelihood prior
The example p(b = r) = 4/10 p(b = b) = 6/10 p(f = a B = r) = 1/4 p(f = o B = r) = 3/4 p(f = a B = b) = 3/4 p(f = o B = b) = 1/4
Probability Densities
Expectations Conditional Expectation (discrete) Approximate Expectation (discrete and continuous)
Variances and Covariances
The Gaussian Distribution
Gaussian Mean and Variance
The Multivariate Gaussian
Gaussian Parameter Estimation Likelihood function
Maximum (Log) Likelihood
Maximum Likelihood Determine by minimizing sum-of-squares error,.
Curve Fitting Re-visited
Predictive Distribution
Bayesian Curve Fitting
Bayesian Predictive Distribution
Introduction to Matlab https://homes.di.unimi.it/donida/introductiontomatlab.zip