Applied Machine Learning in Biomedicine. Enrico Grisan

Transcription

1 Applied Machine Learning in Biomedicine Enrico Grisan

2 Algorithm s objective cost Formal objective for algorithms: - minimize a cost function - maximize an objective function Proving convergence: - does objective monotonically improve? Considering alternatives: - does another algorithm score better?

3 Loss function

4 Choosing a loss function Motivated by the application 0-1 error, achieving a tolerance, business cost Computational convenience: Differentiability, convexity Beware of loss dominated by artifacts: Outliers Unbalanced classes

5 A step into linear regression

6 A step into linear regression

7 Vector form for RSS

8 Least squares estimation

9 Geometry of least squares

10 Least square estimation % ww = Dx1 weights % X = NxD test cases % Y = Nx1 ww = X\Y;

11 Least square estimation (2)

12 Least square estimation (2) 2. Update

13 The importance of the step

14 Least squares classifier Why not using linear least squares to fit regressors on binary targets? % fit yy = ww*xx % ww = Dx1 weights % xx = NxD test cases % yy = Nx1 ww = xx\yy;

15 Least squares classifier

16 Least squares classifier

17 Least squares classifier Why not using linear least squares to fit regressors on binary targets? % fit yy = ww*xx % ww = Dx1 weights % xx = NxD test cases % yy = Nx1 ww = xx\yy;

18 Least squares in practice (1) Prostate cancer study (Stamey, 1989) Mapping clinical measure with PSA marker

19 Least squares in practice (2) 1) Normalize all data

20 Least squares in practice (3) 2) Split the data into train and test set 3) Fit the regression on the training set 4) Estimate results on the test set

21 Least squares in practice (5)

22 Least squares in practice (6)

23 Least squares in practice (6b)

24 Multiclass linear classifier One versus all K class, K>2 Building K-1 binary classifers

25 Multiclass linear classifier One versus one K class, K>2 Building K(K-1)/2 binary classifers

26 Multiclass linear classifier Least squares approach

27 Linear regression (with features) X = [ones(n,1), xx, xx.^2, xx.^3, xx.^4, xx.^5, xx.^6]; W = X\yy; % FunctionInterpolation Xnew = [ones(n,1), xnew, xnew.^2, xnew.^3, xnew.^4, xnew.^5, xnew.^6]; Ynew=Xnew*W;

28 Neighbour-based regression Take height from the nearest input

29 Kernel smoothing Weight points in proportion to a kernel

30 Kernel smoothing

31 Over fitting We can make the empirical loss zero:

32 Generalization Want to do well on future, unknown data: x?

33 Expected error Training error Generalization error Expected test error

34 Validation set validation p=2 p=5 p=9 train

35 Mean squared error Learning curves p, polynomial order

36 Using validation set Validation set used to estimate test error for fitted model Can overfit the validation set Tracking a validation set is also used during fitting of a single model - ad hoc - depends on optimizer - sometimes fast - sometimes can work annoyingly well

37 Model selection and assessment Model selection: Estimating the performance of different models in order to choose the best one Model assessment: having chosen a final model, estimating its prediction error on new data Train Validation Test

38 Cross validation We want a procedure for estimating at least the average generalization error even when the data is scarce and we can not set aside a validation set Data Fold 1 - Train Fold 2 - Train Fold 3 - Train Fold 4 - Train Fold 5 - Test Fold 1 - Train Fold 2 - Train Fold 3 - Test Fold 4 - Train Fold 4 - Train

39 K-Fold Cross Validation

40 Choosing K K=N leave one out Unbiased estimator for the prediction error High variance (training set are very similar) Small-sized training set may overestimate the prediction error Rule of thumb: K=5 or K=10

41 Cross validation scenario

42 Cross validation scenario

43 P1. Cleveland Heart Disease Data from V.A. Medical Center, Long Beach and Cleveland Clinic Foundation, patients, 14 attributes per patient Predict heart disease (possibly in a scale 1-4)

44 P2. HIV cleavage site Knowledge of the mechanism of HIV protease cleavage specificity is critical to the design of specific and effective HIV inhibitors. Searching for an accurate, robust, and rapid method to correctly predict the cleavage sites in proteins is crucial when searching for possible HIV inhibitors. Scope is to predict if a sequence of aminoacids will constitute a cleavage site Rögnvaldsson, You and Garwicz (2015) "State of the art prediction of HIV-1 protease cleavage sites", Bioinformatics, vol 31 (8), pp Kontijevskis, Wikberg and Komorowski (2007) "Computational Proteomics Analysis of HIV-1 Protease Interactome". Proteins: Structure, Function, and Bioinformatics, 68, You, Garwicz and Rögnvaldsson (2005) "Comprehensive Bioinformatic Analysis of the Specificity of Human Immunodeficiency Virus Type 1 Protease". Journal of Virology, 79,