Derivative-Free Optimization for Hyper-Parameter Tuning in Machine Learning Problems
|
|
- Marian Lucas
- 5 years ago
- Views:
Transcription
1 Derivative-Free Optimization for Hyper-Parameter Tuning in Machine Learning Problems Hiva Ghanbari Jointed work with Prof. Katya Scheinberg Industrial and Systems Engineering Department Lehigh University ICCOPT Conference, Tokyo, Japan August 2016 ICCOPT 2016 Derivative Free AUC Optimization (1 of 30)
2 Outline Introduction AUC Optimization via DFO-TR Parameter Tuning of Cost-Sensitive LR Summary ICCOPT 2016 Derivative Free AUC Optimization (2 of 30)
3 Outline Introduction AUC Optimization via DFO-TR Parameter Tuning of Cost-Sensitive LR Summary ICCOPT 2016 Derivative Free AUC Optimization (3 of 30)
4 Learning From Imbalanced Data Sets Many real-world machine learning problems are dealing with imbalanced learning data (a) Balanced data set (b) Imbalanced data set ICCOPT 2016 Derivative Free AUC Optimization (4 of 30)
5 Learning From Imbalanced Data Sets ICCOPT 2016 Derivative Free AUC Optimization (5 of 30)
6 Learning From Imbalanced Data Sets Rare positive example, as the minority class, but numerous negative ones, as the majority class In many applications the minority class is the more interesting and important one The rare class has a much higher misclassification cost compared to the majority class ICCOPT 2016 Derivative Free AUC Optimization (6 of 30)
7 Learning From Imbalanced Data Sets Most common methods for dealing with imbalance data sets Methods Advantages Drawbacks Independent of underlying May remove significant Undersampling classifier patterns Independent of underlying classifier Time consuming Oversampling Can be easily implemented May lead to over-fitting Cost sensitive Minimizes the cost of misclassification The misclassification costs are unknown ICCOPT 2016 Derivative Free AUC Optimization (7 of 30)
8 Ranking Quality of Classifier f(x) Consider linear classifier f(x) = w T x + b, where { +1, if f(x i ) > 0, y i = 1, otherwise. Consider function f s(x) = 1 1+e f(x), where f(x) = wt x + b. 1 f s(x) f(x) = w T x + b ICCOPT 2016 Derivative Free AUC Optimization (8 of 30)
9 Ranking Quality of Classifier f(x) Consider linear classifier f(x) = w T x + b, where { +1, if f(x i ) > 0, y i = 1, otherwise. Consider function f s(x) = 1 1+e f(x), where f(x) = wt x + b. 1 f s(x) f(x) = w T x + b In perfect classification, all positive examples are ranked higher than the negative ones f s(x) ICCOPT 2016 Derivative Free AUC Optimization (8 of 30)
10 Receiver Operating Characteristic(ROC) Curve Sorted outputs based on descending value of f s(x) f s(x) Confusion Matrix of a binary classification problem, for a specific threshold to decide when negative turns into positive example +ve (Predicted Class) -ve (Predicted Class) +ve (Actual Class) True Positive (TP) False Negative (FN) -ve (Actual Class) False Positive (FP) True Negative (TN) Various thresholds result in different True Positive Rate = T P T P +F N False Positive Rate = F P F P +T N. and Definition 1 ROC curve presents the tradeoff between the true positive rate and the false positive rate, for all possible thresholds. ICCOPT 2016 Derivative Free AUC Optimization (9 of 30)
11 Area Under ROC Curve (AUC) How we can compare ROC curves? ICCOPT 2016 Derivative Free AUC Optimization (10 of 30)
12 Area Under ROC Curve (AUC) How we can compare ROC curves? Higher AUC = Better classifier ICCOPT 2016 Derivative Free AUC Optimization (10 of 30)
13 Measuring AUC How can we measure the value of AUC of a classifier? Lemma 2 Consider linear classifier f(x) = w T x + b, which has classified positive set S + := {x + i : i = 1,..., N + } from negative set S := {x j : j = 1,..., N }. Then, the associated AUC can be obtained by WMW result where F AUC (w) = I w[w T x i > w T x j ] = N+ N i=1 j=1 Iw[wT x + i > w T x j ]. N + N { +1, if w T x + i > w T x j, 0, otherwise. ICCOPT 2016 Derivative Free AUC Optimization (11 of 30)
14 Probabilistic Interpretation of AUC We can measure the ranking quality of the classifier f(x) through a probability value. F AUC (w) = P (w T X + > w T X ). where X + and X are randomly sampled from S + and S. If two sets S + and S are randomly drawn from distributions D 1 and D 2, then E(F AUC (w)) = P (w T ˆX+ > w T ˆX ), (1) where ˆX + and ˆX are randomly chosen from distributions D 1 and D 2, respectively. ICCOPT 2016 Derivative Free AUC Optimization (12 of 30)
15 Ranking Loss maximizing AUC function F AUC (w) is equivalent to minimizing ranking loss 1 F AUC (w) N+ N i=1 j=1 I[wT x + i wt x j 0] 1 F AUC (w) = 1 +, N + N where as is shown below I[w T x + i wt x j 0] = { +1, if w T x + i wt x j 0, 0, otherwise. I( ) 1 0 w T (x + i x j ) ICCOPT 2016 Derivative Free AUC Optimization (13 of 30)
16 Surrogate Convex Losses The main challenge of minimizing ranking loss 1 F AUC (w) is in its non-smoothness and discontinuousness owned from indicator function I( ). ICCOPT 2016 Derivative Free AUC Optimization (14 of 30)
17 Surrogate Convex Losses The main challenge of minimizing ranking loss 1 F AUC (w) is in its non-smoothness and discontinuousness owned from indicator function I( ). Exponential Loss: I exp(w, x + i x j ) = e wt (x + i x j ) w T (x + i x j ) ICCOPT 2016 Derivative Free AUC Optimization (14 of 30)
18 Surrogate Convex Losses The main challenge of minimizing ranking loss 1 F AUC (w) is in its non-smoothness and discontinuousness owned from indicator function I( ). Logistic Loss: w T (x + i x j ) I log (w, x + i x j ) = log(1 + e wt (x + i x j ) ), ICCOPT 2016 Derivative Free AUC Optimization (15 of 30)
19 Surrogate Convex Losses The main challenge of minimizing ranking loss 1 F AUC (w) is in its non-smoothness and discontinuousness owned from indicator function I(.). Hinge Loss: I hinge (w, x + i x j ) = max{0, 1 wt (x + i x j )} w T (x + i x j ) ICCOPT 2016 Derivative Free AUC Optimization (16 of 30)
20 Outline Introduction AUC Optimization via DFO-TR Parameter Tuning of Cost-Sensitive LR Summary ICCOPT 2016 Derivative Free AUC Optimization (17 of 30)
21 Algorithmic Framework of DFO-TR Algorithm 1 Trust Region Based Derivative Free Optimization 0. Initializations. Define interpolation set X and compute corresponding function values F X. 1. Build the model. Build Q k (x) as a quadratic approximation of function f. 2. Minimize the model within the trust region. Find ˆx k such that Q k (ˆx k ) := min x Bk Q k (x), where B k = {x : x x k k }. Compute function f(ˆx k ) and the ratio ρ k = f(x k) f(ˆx k ) Q k (x k ) Q k (ˆx k ). 3. Update the interpolation set. 4. Update the trust region radius. ICCOPT 2016 Derivative Free AUC Optimization (18 of 30)
22 AUC, a Smooth Function of w in Expectation Theorem 3 If two d dimensional random vectors ˆX 1 and ˆX 2 have a joint multivariate normal distribution, such that ( ) ( ) ( ) ˆX1 µ1 Σ11 Σ N (µ, Σ), where, µ =, and Σ = 12, ˆX 2 µ 2 Σ 21 Σ 22 then the marginal distributions of ˆX1 and ˆX 2 are normal distributions with the following properties ˆX 1 N (µ 1, Σ 11 ), ˆX2 N (µ 2, Σ 22 ). ICCOPT 2016 Derivative Free AUC Optimization (19 of 30)
23 AUC, a Smooth Function of w in Expectation Theorem 4 Consider two random vectors ˆX 1 and ˆX 2, then for any vector w R d, we have Z = w T ( ˆX 1 ˆX 2 ) N (µ Z, σ 2 Z ), where µ Z = w T (µ 1 µ 2 ), σ 2 Z = wt (Σ 11 + Σ 22 Σ 12 Σ 21 ) w. Theorem 5 If two random vectors ˆX 1 and ˆX 2, respectively from the positive and the negative classes, have a joint normal distribution, then the expected value of AUC function can be defined as ( ) µz E(F AUC (w)) = φ, σ Z where φ is the cumulative function of the standard normal distribution, so that φ(x) = e 1 2 x2 /2π, for x R. ICCOPT 2016 Derivative Free AUC Optimization (20 of 30)
24 Computational Result (DFO-TR vs. Hinge) Performance of algorithms based on the average value of AUC and number of function evaluations, Algorithm sonar(d = 60, N = 208) fourclass(d = 2, N = 862) AUC fevals AUC fevals Hinge ± ± DFO-TR ± ± Algorithm svmguide1(d = 4, N = 3089) magic04(d = 10, N = 4020) AUC fevals AUC fevals Hinge ± ± DFO-TR ± ± Algorithm svmguide3(d = 22, N = 1243) shuttle(d = 9, N = 3500) AUC fevals AUC fevals Hinge ± ± DFO-TR ± ± Algorithm segment(d = 19, N = 2310) ijcnn1(d = 22, N = 4691) AUC fevals AUC fevals Hinge ± ± DFO-TR ± ± Algorithm letter(d = 16, N = 5000) poker(d = 10, N = 5010) AUC fevals AUC fevals Hinge ± ± DFO-TR ± ± ICCOPT 2016 Derivative Free AUC Optimization (21 of 30)
25 Computational Result (Stochastic DFO-TR) Stochastic vs Deterministic DFO-TR ICCOPT 2016 Derivative Free AUC Optimization (22 of 30)
26 Outline Introduction AUC Optimization via DFO-TR Parameter Tuning of Cost-Sensitive LR Summary ICCOPT 2016 Derivative Free AUC Optimization (23 of 30)
27 Cost-Sensitive Logistic Regression Problem Biasing the classifier toward the minority class where F (w) := 1 N N C(y i ) log(1 + exp( y i w T x i )) + λ 2 w 2, i=1 C(y i ) = { C +, if y i > 0, C, otherwise. ICCOPT 2016 Derivative Free AUC Optimization (24 of 30)
28 Cost-Sensitive Logistic Regression Problem Biasing the classifier toward the minority class where GOAL F (w) := 1 N N C(y i ) log(1 + exp( y i w T x i )) + λ 2 w 2, i=1 C(y i ) = { C +, if y i > 0, C, otherwise. Find the best value of costs C + and C and regularization parameter λ, to achieve a linear classifier f(x) = w T x + b with maximum value of AUC. ICCOPT 2016 Derivative Free AUC Optimization (24 of 30)
29 AUC as a Function of {C +, C, λ} AUC is a function of w F AUC (w) = N+ N i=1 j=1 Iw[wT x i > w T x j ]. N + N The vector w is a function of the parameters {C +, C, λ}, so that F (w) := 1 N N C(y i ) log(1 + exp( y i w T x i )) + λ 2 w 2, i=1 ICCOPT 2016 Derivative Free AUC Optimization (25 of 30)
30 AUC as a Function of {C +, C, λ} AUC is a function of w F AUC (w) = N+ N i=1 j=1 Iw[wT x i > w T x j ]. N + N The vector w is a function of the parameters {C +, C, λ}, so that F (w) := 1 N N C(y i ) log(1 + exp( y i w T x i )) + λ 2 w 2, i=1 F AUC (w) is a non-smooth function of the parameters {C +, C, λ}. ICCOPT 2016 Derivative Free AUC Optimization (25 of 30)
31 w a Smooth Function of {C +, C, λ} Theorem 6 In the cost sensitive logistic regression, the vector w is a smooth function of non-zero parameters C +,C, and λ if matrix M and added matrix M v have the same rank (one special case which satisfies this condition is when the matrix M is a full rank matrix) N + exp(y i w T x i ) M = (1 + exp(y i w T x i )) 2 (x ix T i ), i=1 v = 1 C + N + i=1 y i x i 1 + exp(y i w T x i ). (2) Similar condition is required when M and v are constructed based on the negative class. ICCOPT 2016 Derivative Free AUC Optimization (26 of 30)
32 Computational Result In this section we compare the following algorithms, Cost-Sensitive Logistic Regression based on Class Distribution Ratio Cost-Sensitive Logistic Regression based on Optimized Ratio Algorithm sonar fourclass svmguide1 magic04 CSLR-CDR CSLR-OR Algorithm diabetes german a9a svmguide3 CSLR-CDR CSLR-OR Algorithm connect-4 shuttle HAPT segment CSLR-CDR CSLR-OR Algorithm mnist ijcnn1 satimage vowel CSLR-CDR CSLR-OR Algorithm poker letter w1a w8a CSLR-CDR CSLR-OR performance of algorithms based on average value of AUC ICCOPT 2016 Derivative Free AUC Optimization (27 of 30)
33 Outline Introduction AUC Optimization via DFO-TR Parameter Tuning of Cost-Sensitive LR Summary ICCOPT 2016 Derivative Free AUC Optimization (28 of 30)
34 Summary and Future Study The main challenge of optimizing AUC is in its non-smoothness and discontinuousness Directly Optimizing AUC function via trust region based derivative free optimization Tuning parameters of cost-sensitive logistic regression to obtain a classifier with maximum AUC value ICCOPT 2016 Derivative Free AUC Optimization (29 of 30)
35 References A. R. Conn, K. Scheinberg, and L. N.Vicente, Introduction To Derivative-Free Optimization, SIAM J, M. Menickelly R. Chen and K. Scheinberg, Stochastic optimization using a trust-region method and random models, C. X. Ling, J. Huang, and H. Zhang, AUC: a Statistically Consistent and more Discriminating Measure than Accuracy, J.A. Hanley, B.J. McNeil, The meaning and use of the area under a receiver operating characteristic (ROC) curve, Radiology, 143:29?36, ICCOPT 2016 Derivative Free AUC Optimization (30 of 30)
A scored AUC Metric for Classifier Evaluation and Selection
A scored AUC Metric for Classifier Evaluation and Selection Shaomin Wu SHAOMIN.WU@READING.AC.UK School of Construction Management and Engineering, The University of Reading, Reading RG6 6AW, UK Peter Flach
More informationMETHODS FOR DETECTING CERVICAL CANCER
Chapter III METHODS FOR DETECTING CERVICAL CANCER 3.1 INTRODUCTION The successful detection of cervical cancer in a variety of tissues has been reported by many researchers and baseline figures for the
More informationMachine learning II. Juhan Ernits ITI8600
Machine learning II Juhan Ernits ITI8600 Hand written digit recognition 64 Example 2: Face recogition Classification, regression or unsupervised? How many classes? Example 2: Face recognition Classification,
More informationReview: Logistic regression, Gaussian naïve Bayes, linear regression, and their connections
Review: Logistic regression, Gaussian naïve Bayes, linear regression, and their connections New: Bias-variance decomposition, biasvariance tradeoff, overfitting, regularization, and feature selection Yi
More informationStatistics, Probability and Diagnostic Medicine
Statistics, Probability and Diagnostic Medicine Jennifer Le-Rademacher, PhD Sponsored by the Clinical and Translational Science Institute (CTSI) and the Department of Population Health / Division of Biostatistics
More informationOutlier Analysis. Lijun Zhang
Outlier Analysis Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Introduction Extreme Value Analysis Probabilistic Models Clustering for Outlier Detection Distance-Based Outlier Detection Density-Based
More informationReview. Imagine the following table being obtained as a random. Decision Test Diseased Not Diseased Positive TP FP Negative FN TN
Outline 1. Review sensitivity and specificity 2. Define an ROC curve 3. Define AUC 4. Non-parametric tests for whether or not the test is informative 5. Introduce the binormal ROC model 6. Discuss non-parametric
More informationVarious performance measures in Binary classification An Overview of ROC study
Various performance measures in Binary classification An Overview of ROC study Suresh Babu. Nellore Department of Statistics, S.V. University, Tirupati, India E-mail: sureshbabu.nellore@gmail.com Abstract
More informationKnowledge Discovery and Data Mining. Testing. Performance Measures. Notes. Lecture 15 - ROC, AUC & Lift. Tom Kelsey. Notes
Knowledge Discovery and Data Mining Lecture 15 - ROC, AUC & Lift Tom Kelsey School of Computer Science University of St Andrews http://tom.home.cs.st-andrews.ac.uk twk@st-andrews.ac.uk Tom Kelsey ID5059-17-AUC
More informationYeast Cells Classification Machine Learning Approach to Discriminate Saccharomyces cerevisiae Yeast Cells Using Sophisticated Image Features.
Yeast Cells Classification Machine Learning Approach to Discriminate Saccharomyces cerevisiae Yeast Cells Using Sophisticated Image Features. Mohamed Tleis Supervisor: Fons J. Verbeek Leiden University
More informationEfficient AUC Optimization for Information Ranking Applications
Efficient AUC Optimization for Information Ranking Applications Sean J. Welleck IBM, USA swelleck@us.ibm.com Abstract. Adequate evaluation of an information retrieval system to estimate future performance
More informationStatistical methods for the meta-analysis of full ROC curves
Statistical methods for the meta-analysis of full ROC curves Oliver Kuss (joint work with Stefan Hirt and Annika Hoyer) German Diabetes Center, Leibniz Institute for Diabetes Research at Heinrich Heine
More informationComparing Multifunctionality and Association Information when Classifying Oncogenes and Tumor Suppressor Genes
000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
More informationEstimation of Area under the ROC Curve Using Exponential and Weibull Distributions
XI Biennial Conference of the International Biometric Society (Indian Region) on Computational Statistics and Bio-Sciences, March 8-9, 22 43 Estimation of Area under the ROC Curve Using Exponential and
More informationStatistical methods for the meta-analysis of full ROC curves
Statistical methods for the meta-analysis of full ROC curves Oliver Kuss (joint work with Stefan Hirt and Annika Hoyer) German Diabetes Center, Leibniz Institute for Diabetes Research at Heinrich Heine
More informationINTRODUCTION TO MACHINE LEARNING. Decision tree learning
INTRODUCTION TO MACHINE LEARNING Decision tree learning Task of classification Automatically assign class to observations with features Observation: vector of features, with a class Automatically assign
More informationUNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2014
UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2014 Exam policy: This exam allows two one-page, two-sided cheat sheets (i.e. 4 sides); No other materials. Time: 2 hours. Be sure to write
More informationIntroduction to Computational Neuroscience
Introduction to Computational Neuroscience Lecture 5: Data analysis II Lesson Title 1 Introduction 2 Structure and Function of the NS 3 Windows to the Brain 4 Data analysis 5 Data analysis II 6 Single
More informationFeature selection methods for early predictive biomarker discovery using untargeted metabolomic data
Feature selection methods for early predictive biomarker discovery using untargeted metabolomic data Dhouha Grissa, Mélanie Pétéra, Marion Brandolini, Amedeo Napoli, Blandine Comte and Estelle Pujos-Guillot
More informationPERFORMANCE MEASURES
PERFORMANCE MEASURES Of predictive systems DATA TYPES Binary Data point Value A FALSE B TRUE C TRUE D FALSE E FALSE F TRUE G FALSE Real Value Data Point Value a 32.3 b.2 b 2. d. e 33 f.65 g 72.8 ACCURACY
More informationReceiver operating characteristic
Receiver operating characteristic From Wikipedia, the free encyclopedia In signal detection theory, a receiver operating characteristic (ROC), or simply ROC curve, is a graphical plot of the sensitivity,
More informationDetection of Mild Cognitive Impairment using Image Differences and Clinical Features
Detection of Mild Cognitive Impairment using Image Differences and Clinical Features L I N L I S C H O O L O F C O M P U T I N G C L E M S O N U N I V E R S I T Y Copyright notice Many of the images in
More informationClassical Psychophysical Methods (cont.)
Classical Psychophysical Methods (cont.) 1 Outline Method of Adjustment Method of Limits Method of Constant Stimuli Probit Analysis 2 Method of Constant Stimuli A set of equally spaced levels of the stimulus
More informationSaichiu Nelson Tong ALL RIGHTS RESERVED
2017 Saichiu Nelson Tong ALL RIGHTS RESERVED MODELING CLASSIFICATION NETWORK OF ELECTROENCEPHALOGRAPHIC ARTIFACTS AND SIGNALS ASSOCIATED WITH DEEP BRAIN STIMULATION By SAICHIU NELSON TONG A thesis submitted
More informationData complexity measures for analyzing the effect of SMOTE over microarrays
ESANN 216 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges (Belgium), 27-29 April 216, i6doc.com publ., ISBN 978-2878727-8. Data complexity
More informationPredictive performance and discrimination in unbalanced classification
MASTER Predictive performance and discrimination in unbalanced classification van der Zon, S.B. Award date: 2016 Link to publication Disclaimer This document contains a student thesis (bachelor's or master's),
More informationROI DETECTION AND VESSEL SEGMENTATION IN RETINAL IMAGE
ROI DETECTION AND VESSEL SEGMENTATION IN RETINAL IMAGE F. Sabaz a, *, U. Atila a a Karabuk University, Dept. of Computer Engineering, 78050, Karabuk, Turkey - (furkansabaz, umitatila)@karabuk.edu.tr KEY
More informationWeek 2 Video 3. Diagnostic Metrics
Week 2 Video 3 Diagnostic Metrics Different Methods, Different Measures Today we ll continue our focus on classifiers Later this week we ll discuss regressors And other methods will get worked in later
More informationROC Curves (Old Version)
Chapter 545 ROC Curves (Old Version) Introduction This procedure generates both binormal and empirical (nonparametric) ROC curves. It computes comparative measures such as the whole, and partial, area
More informationBehavioral Data Mining. Lecture 4 Measurement
Behavioral Data Mining Lecture 4 Measurement Outline Hypothesis testing Parametric statistical tests Non-parametric tests Precision-Recall plots ROC plots Hardware update Icluster machines are ready for
More informationAn Empirical and Formal Analysis of Decision Trees for Ranking
An Empirical and Formal Analysis of Decision Trees for Ranking Eyke Hüllermeier Department of Mathematics and Computer Science Marburg University 35032 Marburg, Germany eyke@mathematik.uni-marburg.de Stijn
More informationImproving k Nearest Neighbor with Exemplar Generalization for Imbalanced Classification
Improving k Nearest Neighbor with Exemplar Generalization for Imbalanced Classification Yuxuan Li and Xiuzhen Zhang School of Computer Science and Information Technology, RMIT University, Melbourne, Australia
More informationSelection and Combination of Markers for Prediction
Selection and Combination of Markers for Prediction NACC Data and Methods Meeting September, 2010 Baojiang Chen, PhD Sarah Monsell, MS Xiao-Hua Andrew Zhou, PhD Overview 1. Research motivation 2. Describe
More informationCOMPARATIVE STUDY ON FEATURE EXTRACTION METHOD FOR BREAST CANCER CLASSIFICATION
COMPARATIVE STUDY ON FEATURE EXTRACTION METHOD FOR BREAST CANCER CLASSIFICATION 1 R.NITHYA, 2 B.SANTHI 1 Asstt Prof., School of Computing, SASTRA University, Thanjavur, Tamilnadu, India-613402 2 Prof.,
More informationAn Improved Patient-Specific Mortality Risk Prediction in ICU in a Random Forest Classification Framework
An Improved Patient-Specific Mortality Risk Prediction in ICU in a Random Forest Classification Framework Soumya GHOSE, Jhimli MITRA 1, Sankalp KHANNA 1 and Jason DOWLING 1 1. The Australian e-health and
More informationCOMPUTERIZED SYSTEM DESIGN FOR THE DETECTION AND DIAGNOSIS OF LUNG NODULES IN CT IMAGES 1
ISSN 258-8739 3 st August 28, Volume 3, Issue 2, JSEIS, CAOMEI Copyright 26-28 COMPUTERIZED SYSTEM DESIGN FOR THE DETECTION AND DIAGNOSIS OF LUNG NODULES IN CT IMAGES ALI ABDRHMAN UKASHA, 2 EMHMED SAAID
More information1 Introduction. st0020. The Stata Journal (2002) 2, Number 3, pp
The Stata Journal (22) 2, Number 3, pp. 28 289 Comparative assessment of three common algorithms for estimating the variance of the area under the nonparametric receiver operating characteristic curve
More informationBiomarker adaptive designs in clinical trials
Review Article Biomarker adaptive designs in clinical trials James J. Chen 1, Tzu-Pin Lu 1,2, Dung-Tsa Chen 3, Sue-Jane Wang 4 1 Division of Bioinformatics and Biostatistics, National Center for Toxicological
More informationEnhanced Detection of Lung Cancer using Hybrid Method of Image Segmentation
Enhanced Detection of Lung Cancer using Hybrid Method of Image Segmentation L Uma Maheshwari Department of ECE, Stanley College of Engineering and Technology for Women, Hyderabad - 500001, India. Udayini
More informationBREAST CANCER EPIDEMIOLOGY MODEL:
BREAST CANCER EPIDEMIOLOGY MODEL: Calibrating Simulations via Optimization Michael C. Ferris, Geng Deng, Dennis G. Fryback, Vipat Kuruchittham University of Wisconsin 1 University of Wisconsin Breast Cancer
More informationAccurate Diabetes Risk Stratification Using Machine Learning: Role of Missing Value and Outliers
Journal of Medical Systems (2018) 42: 92 https://doi.org/10.1007/s10916-018-0940-7 Accurate Diabetes Risk Stratification Using Machine Learning: Role of Missing Value and Outliers Md. Maniruzzaman 1,2
More informationModifying ROC Curves to Incorporate Predicted Probabilities
Modifying ROC Curves to Incorporate Predicted Probabilities C. Ferri, P. Flach 2, J. Hernández-Orallo, A. Senad Departament de Sistemes Informàtics i Computació Universitat Politècnica de València Spain
More informationUsing AUC and Accuracy in Evaluating Learning Algorithms
1 Using AUC and Accuracy in Evaluating Learning Algorithms Jin Huang Charles X. Ling Department of Computer Science The University of Western Ontario London, Ontario, Canada N6A 5B7 fjhuang, clingg@csd.uwo.ca
More informationUNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Midterm, 2016
UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Midterm, 2016 Exam policy: This exam allows one one-page, two-sided cheat sheet; No other materials. Time: 80 minutes. Be sure to write your name and
More informationPart [1.0] Introduction to Development and Evaluation of Dynamic Predictions
Part [1.0] Introduction to Development and Evaluation of Dynamic Predictions A Bansal & PJ Heagerty Department of Biostatistics University of Washington 1 Biomarkers The Instructor(s) Patrick Heagerty
More informationZheng Yao Sr. Statistical Programmer
ROC CURVE ANALYSIS USING SAS Zheng Yao Sr. Statistical Programmer Outline Background Examples: Accuracy assessment Compare ROC curves Cut-off point selection Summary 2 Outline Background Examples: Accuracy
More informationBrain Tumor segmentation and classification using Fcm and support vector machine
Brain Tumor segmentation and classification using Fcm and support vector machine Gaurav Gupta 1, Vinay singh 2 1 PG student,m.tech Electronics and Communication,Department of Electronics, Galgotia College
More informationINTERNATIONAL JOURNAL OF ENGINEERING SCIENCES & RESEARCH TECHNOLOGY
IJESRT INTERNATIONAL JOURNAL OF ENGINEERING SCIENCES & RESEARCH TECHNOLOGY A Medical Decision Support System based on Genetic Algorithm and Least Square Support Vector Machine for Diabetes Disease Diagnosis
More informationCorrecting AUC for Measurement Error
Correcting AUC for Measurement Error The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters. Citation Published Version Accessed Citable
More informationEfficacy of the Extended Principal Orthogonal Decomposition Method on DNA Microarray Data in Cancer Detection
202 4th International onference on Bioinformatics and Biomedical Technology IPBEE vol.29 (202) (202) IASIT Press, Singapore Efficacy of the Extended Principal Orthogonal Decomposition on DA Microarray
More informationComputer-aided diagnosis of subtle signs of breast cancer: Architectural distortion in prior mammograms
Computer-aided diagnosis of subtle signs of breast cancer: Architectural distortion in prior mammograms Rangaraj M. Rangayyan Department of Electrical and Computer Engineering University of Calgary, Calgary,
More informationWhen Overlapping Unexpectedly Alters the Class Imbalance Effects
When Overlapping Unexpectedly Alters the Class Imbalance Effects V. García 1,2, R.A. Mollineda 2,J.S.Sánchez 2,R.Alejo 1,2, and J.M. Sotoca 2 1 Lab. Reconocimiento de Patrones, Instituto Tecnológico de
More informationLarge-Scale Statistical Modelling via Machine Learning Classifiers
J. Stat. Appl. Pro. 2, No. 3, 203-222 (2013) 203 Journal of Statistics Applications & Probability An International Journal http://dx.doi.org/10.12785/jsap/020303 Large-Scale Statistical Modelling via Machine
More informationEmpirical assessment of univariate and bivariate meta-analyses for comparing the accuracy of diagnostic tests
Empirical assessment of univariate and bivariate meta-analyses for comparing the accuracy of diagnostic tests Yemisi Takwoingi, Richard Riley and Jon Deeks Outline Rationale Methods Findings Summary Motivating
More informationDepartment of Epidemiology, Rollins School of Public Health, Emory University, Atlanta GA, USA.
A More Intuitive Interpretation of the Area Under the ROC Curve A. Cecile J.W. Janssens, PhD Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta GA, USA. Corresponding
More informationAn Improved Algorithm To Predict Recurrence Of Breast Cancer
An Improved Algorithm To Predict Recurrence Of Breast Cancer Umang Agrawal 1, Ass. Prof. Ishan K Rajani 2 1 M.E Computer Engineer, Silver Oak College of Engineering & Technology, Gujarat, India. 2 Assistant
More informationPart [2.1]: Evaluation of Markers for Treatment Selection Linking Clinical and Statistical Goals
Part [2.1]: Evaluation of Markers for Treatment Selection Linking Clinical and Statistical Goals Patrick J. Heagerty Department of Biostatistics University of Washington 174 Biomarkers Session Outline
More informationWinner s Report: KDD CUP Breast Cancer Identification
Winner s Report: KDD CUP Breast Cancer Identification ABSTRACT Claudia Perlich, Prem Melville, Yan Liu, Grzegorz Świrszcz, Richard Lawrence IBM T.J. Watson Research Center Yorktown Heights, NY 10598 {perlich,pmelvil,liuya}@us.ibm.com
More informationIdentifikation von Risikofaktoren in der koronaren Herzchirurgie
Identifikation von Risikofaktoren in der koronaren Herzchirurgie Julia Schiffner 1 Erhard Godehardt 2 Stefanie Hillebrand 1 Alexander Albert 2 Artur Lichtenberg 2 Claus Weihs 1 1 Fakultät Statistik, Technische
More informationENSEMBLE CLASSIFIER APPROACH IN BREAST CANCER DETECTION AND MALIGNANCY GRADING- A REVIEW
ENSEMBLE CLASSIFIER APPROACH IN BREAST CANCER DETECTION AND MALIGNANCY GRADING- A REVIEW Deepti Ameta 1 1 Department of Computer Engineering, Banasthali University, Banasthali, India ABSTRACT The diagnosed
More informationA Vision-based Affective Computing System. Jieyu Zhao Ningbo University, China
A Vision-based Affective Computing System Jieyu Zhao Ningbo University, China Outline Affective Computing A Dynamic 3D Morphable Model Facial Expression Recognition Probabilistic Graphical Models Some
More informationComparing Two ROC Curves Independent Groups Design
Chapter 548 Comparing Two ROC Curves Independent Groups Design Introduction This procedure is used to compare two ROC curves generated from data from two independent groups. In addition to producing a
More informationEVALUATION AND COMPUTATION OF DIAGNOSTIC TESTS: A SIMPLE ALTERNATIVE
EVALUATION AND COMPUTATION OF DIAGNOSTIC TESTS: A SIMPLE ALTERNATIVE NAHID SULTANA SUMI, M. ATAHARUL ISLAM, AND MD. AKHTAR HOSSAIN Abstract. Methods of evaluating and comparing the performance of diagnostic
More informationA Review on Sleep Apnea Detection from ECG Signal
A Review on Sleep Apnea Detection from ECG Signal Soumya Gopal 1, Aswathy Devi T. 2 1 M.Tech Signal Processing Student, Department of ECE, LBSITW, Kerala, India 2 Assistant Professor, Department of ECE,
More informationFusing Simultaneous EEG-fMRI by Linking Multivariate Classifiers
Fusing Simultaneous EEG-fMRI by Linking Multivariate Classifiers Bryan R. Conroy 1, Jordan Muraskin 1, and Paul Sajda 1 Dept. of Biomedical Engineering, Columbia University, Columbia NY, USA Abstract.
More informationSensitivity, Specificity, and Relatives
Sensitivity, Specificity, and Relatives Brani Vidakovic ISyE 6421/ BMED 6700 Vidakovic, B. Se Sp and Relatives January 17, 2017 1 / 26 Overview Today: Vidakovic, B. Se Sp and Relatives January 17, 2017
More informationComputer Models for Medical Diagnosis and Prognostication
Computer Models for Medical Diagnosis and Prognostication Lucila Ohno-Machado, MD, PhD Division of Biomedical Informatics Clinical pattern recognition and predictive models Evaluation of binary classifiers
More informationEXTRACTION OF RETINAL BLOOD VESSELS USING IMAGE PROCESSING TECHNIQUES
EXTRACTION OF RETINAL BLOOD VESSELS USING IMAGE PROCESSING TECHNIQUES T.HARI BABU 1, Y.RATNA KUMAR 2 1 (PG Scholar, Dept. of Electronics and Communication Engineering, College of Engineering(A), Andhra
More informationDiagnosis of Breast Cancer Using Ensemble of Data Mining Classification Methods
International Journal of Bioinformatics and Biomedical Engineering Vol. 1, No. 3, 2015, pp. 318-322 http://www.aiscience.org/journal/ijbbe ISSN: 2381-7399 (Print); ISSN: 2381-7402 (Online) Diagnosis of
More informationVU Biostatistics and Experimental Design PLA.216
VU Biostatistics and Experimental Design PLA.216 Julia Feichtinger Postdoctoral Researcher Institute of Computational Biotechnology Graz University of Technology Outline for Today About this course Background
More informationData Analytics for Optimal Detection of Metastatic Prostate Cancer
Data Analytics for Optimal Detection of Metastatic Prostate Cancer Selin Merdan, Christine Barnett, Brian T. Denton Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor,
More informationDetection of Glaucoma and Diabetic Retinopathy from Fundus Images by Bloodvessel Segmentation
International Journal of Engineering and Advanced Technology (IJEAT) ISSN: 2249 8958, Volume-5, Issue-5, June 2016 Detection of Glaucoma and Diabetic Retinopathy from Fundus Images by Bloodvessel Segmentation
More informationMethods for Predicting Type 2 Diabetes
Methods for Predicting Type 2 Diabetes CS229 Final Project December 2015 Duyun Chen 1, Yaxuan Yang 2, and Junrui Zhang 3 Abstract Diabetes Mellitus type 2 (T2DM) is the most common form of diabetes [WHO
More informationPrequential AUC: properties of the area under the ROC curve for data streams with concept drift
Knowl Inf Syst (2017) 52:531 562 DOI 10.1007/s10115-017-1022-8 REGULAR PAPER Prequential AUC: properties of the area under the ROC curve for data streams with concept drift Dariusz Brzezinski 1 Jerzy Stefanowski
More informationNoncompartmental Analysis (NCA) in PK, PK-based Design
Noncompartmental Analysis (NCA) in PK, PK-based Design Helmut Schütz BEBAC Consultancy Services for Bioequivalence and Bioavailability Studies 17 Vienna, Austria helmut.schuetz@bebac.at Bioequivalence
More informationarxiv: v1 [cs.lg] 3 Jan 2018
arxiv:1801.01204v1 [cs.lg] 3 Jan 2018 Predicting Chronic Disease Hospitalizations from Electronic Health Records: An Interpretable Classification Approach Theodora S. Brisimi, Tingting Xu, Taiyao Wang,
More informationLOGISTIC REGRESSION VERSUS NEURAL
Monografías del Seminario Matemático García de Galdeano 33, 245 252 (2006) LOGISTIC REGRESSION VERSUS NEURAL NETWORKS FOR MEDICAL DATA Luis Mariano Esteban Escaño, Gerardo Sanz Saiz, Francisco Javier López
More informationPersonalized Colorectal Cancer Survivability Prediction with Machine Learning Methods*
Personalized Colorectal Cancer Survivability Prediction with Machine Learning Methods* 1 st Samuel Li Princeton University Princeton, NJ seli@princeton.edu 2 nd Talayeh Razzaghi New Mexico State University
More informationSISCR Module 7 Part I: Introduction Basic Concepts for Binary Biomarkers (Classifiers) and Continuous Biomarkers
SISCR Module 7 Part I: Introduction Basic Concepts for Binary Biomarkers (Classifiers) and Continuous Biomarkers Kathleen Kerr, Ph.D. Associate Professor Department of Biostatistics University of Washington
More informationarxiv: v1 [cs.cv] 26 Mar 2016
Classification of Large-Scale Fundus Image Data Sets: A Cloud-Computing Framework Sohini Roychowdhury 1 arxiv:1603.08071v1 [cs.cv] 26 Mar 2016 Abstract Large medical image data sets with high dimensionality
More informationPredictive Models for Healthcare Analytics
Predictive Models for Healthcare Analytics A Case on Retrospective Clinical Study Mengling Mornin Feng mfeng@mit.edu mornin@gmail.com 1 Learning Objectives After the lecture, students should be able to:
More informationIntroduction to diagnostic accuracy meta-analysis. Yemisi Takwoingi October 2015
Introduction to diagnostic accuracy meta-analysis Yemisi Takwoingi October 2015 Learning objectives To appreciate the concept underlying DTA meta-analytic approaches To know the Moses-Littenberg SROC method
More informationAnalysis of Diabetic Dataset and Developing Prediction Model by using Hive and R
Indian Journal of Science and Technology, Vol 9(47), DOI: 10.17485/ijst/2016/v9i47/106496, December 2016 ISSN (Print) : 0974-6846 ISSN (Online) : 0974-5645 Analysis of Diabetic Dataset and Developing Prediction
More informationResearch on Classification of Diseases of Clinical Imbalanced Data in Traditional Chinese Medicine
Int'l Conf. Bioinformatics and Computational Biology BIOCOMP'17 57 Research on Classification of Diseases of Clinical Imbalanced Data in Traditional Chinese Medicine Zhu-Qiang Pan School of Computer Science,
More informationAssigning B cell Maturity in Pediatric Leukemia Gabi Fragiadakis 1, Jamie Irvine 2 1 Microbiology and Immunology, 2 Computer Science
Assigning B cell Maturity in Pediatric Leukemia Gabi Fragiadakis 1, Jamie Irvine 2 1 Microbiology and Immunology, 2 Computer Science Abstract One method for analyzing pediatric B cell leukemia is to categorize
More informationA HMM-based Pre-training Approach for Sequential Data
A HMM-based Pre-training Approach for Sequential Data Luca Pasa 1, Alberto Testolin 2, Alessandro Sperduti 1 1- Department of Mathematics 2- Department of Developmental Psychology and Socialisation University
More informationPII: S (96) THE USE OF THE AREA UNDER THE ROC CURVE IN THE EVALUATION OF MACHINE LEARNING ALGORITHMS
Pergamon Pattern Recognition, Vol. 30, No. 7, pp. 1145-1159, 1997 1997 Pattern Recognition Society. Published by Elsevier Science Ltd Printed in Great Britain. All rights reserved 0031-3203/97 $17.00+.00
More informationClassification of benign and malignant masses in breast mammograms
Classification of benign and malignant masses in breast mammograms A. Šerifović-Trbalić*, A. Trbalić**, D. Demirović*, N. Prljača* and P.C. Cattin*** * Faculty of Electrical Engineering, University of
More informationSensitivity, specicity, ROC
Sensitivity, specicity, ROC Thomas Alexander Gerds Department of Biostatistics, University of Copenhagen 1 / 53 Epilog: disease prevalence The prevalence is the proportion of cases in the population today.
More informationWhite Paper Estimating Complex Phenotype Prevalence Using Predictive Models
White Paper 23-12 Estimating Complex Phenotype Prevalence Using Predictive Models Authors: Nicholas A. Furlotte Aaron Kleinman Robin Smith David Hinds Created: September 25 th, 2015 September 25th, 2015
More informationError Detection based on neural signals
Error Detection based on neural signals Nir Even- Chen and Igor Berman, Electrical Engineering, Stanford Introduction Brain computer interface (BCI) is a direct communication pathway between the brain
More informationModule Overview. What is a Marker? Part 1 Overview
SISCR Module 7 Part I: Introduction Basic Concepts for Binary Classification Tools and Continuous Biomarkers Kathleen Kerr, Ph.D. Associate Professor Department of Biostatistics University of Washington
More informationLecture 10: Learning Optimal Personalized Treatment Rules Under Risk Constraint
Lecture 10: Learning Optimal Personalized Treatment Rules Under Risk Constraint Introduction Consider Both Efficacy and Safety Outcomes Clinician: Complete picture of treatment decision making involves
More informationClassification with microarray data
Classification with microarray data Aron Charles Eklund eklund@cbs.dtu.dk DNA Microarray Analysis - #27612 January 8, 2010 The rest of today Now: What is classification, and why do we do it? How to develop
More informationMultivariate Bioequivalence
Multivariate Bioequivalence S h i t a l A g a w a n e, S a n j u k t a R o y P h U S E 2013 S t r e a m : S t a t i s t i c s a n d P h a r m a c o k i n e t i c s S P 0 4 PhUSE 2013 Disclaimer Any views
More informationClassification. Methods Course: Gene Expression Data Analysis -Day Five. Rainer Spang
Classification Methods Course: Gene Expression Data Analysis -Day Five Rainer Spang Ms. Smith DNA Chip of Ms. Smith Expression profile of Ms. Smith Ms. Smith 30.000 properties of Ms. Smith The expression
More informationSelection of Linking Items
Selection of Linking Items Subset of items that maximally reflect the scale information function Denote the scale information as Linear programming solver (in R, lp_solve 5.5) min(y) Subject to θ, θs,
More informationModelling and Application of Logistic Regression and Artificial Neural Networks Models
Modelling and Application of Logistic Regression and Artificial Neural Networks Models Norhazlina Suhaimi a, Adriana Ismail b, Nurul Adyani Ghazali c a,c School of Ocean Engineering, Universiti Malaysia
More informationBayesian methods in health economics
Bayesian methods in health economics Gianluca Baio University College London Department of Statistical Science g.baio@ucl.ac.uk Seminar Series of the Master in Advanced Artificial Intelligence Madrid,
More information