Overview of statistical evaluation techniques adopted by publicly available MVPA toolboxes
|
|
|
- Nathaniel Allison
- 6 years ago
- Views:
Transcription
1 Overview of statistical evaluation techniques adopted by publicly available MVPA toolboxes Yaroslav O. Halchenko Center for Cognitive Neuroscience, Psychological and Brain Sciences Department, Dartmouth College PyMVPA, NeuroDebian, DataLad, duecredit OHBM 2015, Honolulu HI
2 Disclaimer #1: I do PyMVPA
3 Many thanks to Andre Marquand (PROBID, Matlab) Francisco Pereira (searchmight, Matlab) Gael Varoquaux (nilearn/scikit-learn, Python) Jessica Schrouff (PRoNTo, Matlab) Martin Hebart (TDT, Matlab) Mitsuaki Tsukamoto (BDTB, Matlab) Nick Oosterhof (CoSMoMVPA, Matlab) Nikolaus Kriegeskorte (RSA, Matlab) [will not talk about]
4 Many thanks to Andre Marquand (PROBID, Matlab) Francisco Pereira (searchmight, Matlab) Gael Varoquaux (nilearn/scikit-learn, Python) Jessica Schrouff (PRoNTo, Matlab) Martin Hebart (TDT, Matlab) Mitsuaki Tsukamoto (BDTB, Matlab) Nick Oosterhof (CoSMoMVPA, Matlab) Nikolaus Kriegeskorte (RSA, Matlab) [will not talk about] Q: who uses any of the aforementioned toolkits (including PyMVPA)?
5 Many thanks to Andre Marquand (PROBID, Matlab) Francisco Pereira (searchmight, Matlab) Gael Varoquaux (nilearn/scikit-learn, Python) Jessica Schrouff (PRoNTo, Matlab) Martin Hebart (TDT, Matlab) Mitsuaki Tsukamoto (BDTB, Matlab) Nick Oosterhof (CoSMoMVPA, Matlab) Nikolaus Kriegeskorte (RSA, Matlab) [will not talk about] Q: who uses any of the aforementioned toolkits (including PyMVPA)? Q: who uses some other (not your own) toolkit?
6 Many thanks to Andre Marquand (PROBID, Matlab) Francisco Pereira (searchmight, Matlab) Gael Varoquaux (nilearn/scikit-learn, Python) Jessica Schrouff (PRoNTo, Matlab) Martin Hebart (TDT, Matlab) Mitsuaki Tsukamoto (BDTB, Matlab) Nick Oosterhof (CoSMoMVPA, Matlab) Nikolaus Kriegeskorte (RSA, Matlab) [will not talk about] Q: who uses any of the aforementioned toolkits (including PyMVPA)? Q: who uses some other (not your own) toolkit? Q: who writes ad-hoc code instead?
7 Disclaimer #2: This review is not comprehensive I will note where and what functionality is available. I will not state that some functionality is not available in a particular toolbox I will not provide overview of interfaces (e.g. scripting vs. GUI), but rather will reference the functionality available I will not talk about sensitivities analyses although many toolboxes allow for some
8 We (as in neuroimaging) are special
9 We (as in neuroimaging) are special Machine-learning folks construct the best predictive model given a large array of samples characterize the model by accuracy of classification on some canonical datasets
10 We (as in neuroimaging) are special Machine-learning folks construct the best predictive model given a large array of samples characterize the model by accuracy of classification on some canonical datasets We (Neuroimaging) folks construct a model good enough to state that data contain information of interest use summary statistic computed over obtained accuracies to support claim of presence of the signal of interest in new dataset
11 Additional support measures Gut feeling (
12 Additional support measures Gut feeling ( Priors (expertise, publications, NeuroSynth.org)
13 Additional support measures Gut feeling ( Priors (expertise, publications, NeuroSynth.org) Are results trustworthy?
14 Additional support measures Gut feeling ( Priors (expertise, publications, NeuroSynth.org) Are results trustworthy? Stable Reproducible Not random
15 Beliefs see Dance of the p Values Wright, D. B. (2009). Ten statisticians and their impacts for psychologists. Perspectives on Psychological Science, 4(6):
16 Statistical significance testing can improve the level of trust in observed results
17 Factors... affecting level of trust negatively Software bugs [Do you trust your tools?] Experimental design bugs Analysis bugs I: double dipping Analysis bugs II: exploitation
18 Factors... affecting level of trust negatively Software bugs [Do you trust your tools?] Experimental design bugs Analysis bugs I: double dipping Analysis bugs II: exploitation positively Statistical significance of the results
19 Factors... affecting level of trust negatively Software bugs [Do you trust your tools?] Experimental design bugs Analysis bugs I: double dipping Analysis bugs II: exploitation positively Statistical significance of the results All of the above is not MVPA-specific, but With great power comes great responsibility (Uncle Ben)
20 Under assumption of bug-free implementation, how can existing toolboxes help to improve the level of trust in our MVPA results?
21 Experimental design bugs Major manifestations Imbalances Trial order effects
22 Experimental design bugs Scrutinize design (per subject) Review labeling stats: PyMVPA (dataset.summary(), includes trial order stats), PRoNTo Decode the design: TDT (not based on trial-order) Remove overlaps: PRoNTo
23 Experimental design bugs Scrutinize design (per subject) Review labeling stats: PyMVPA (dataset.summary(), includes trial order stats), PRoNTo Decode the design: TDT (not based on trial-order) Remove overlaps: PRoNTo Avoid inbalance Mean the trials. Don t!: introduces spurious signal Sub-sample: PyMVPA, CoSMoMVPA (disallows imbalance, and allows re-balancing), TDT, BDTB Metrics other than overall accuracy AUC: scikit-learn, PyMVPA, TDT balanced accuracy/mean of per-class accuracies: PRoNTo, PROBID, TDT
24 Analysis bugs I: double dipping (DD), circular analysis Cross-validation constructs Split according to natural confounds (e.g. runs/sessions): all Flexible (PyMVPA, nilearn, CoSMoMVPA, TDT) or more restricted (PROBID, searchmight, BDTB) forbidding double-dipping Combined with pre-processing, such as feature selection or tranformation (e.g. PCA): PyMVPA, scikit-learn, CoSMoMVPA, TDT, PRoNTo
25 Analysis bugs I: double dipping (DD), circular analysis Cross-validation constructs Split according to natural confounds (e.g. runs/sessions): all Flexible (PyMVPA, nilearn, CoSMoMVPA, TDT) or more restricted (PROBID, searchmight, BDTB) forbidding double-dipping Combined with pre-processing, such as feature selection or tranformation (e.g. PCA): PyMVPA, scikit-learn, CoSMoMVPA, TDT, PRoNTo Nested cross-validation Parameter selection: scikit-learn, PRoNTo, TDT, (PyMVPA on example, convenience coming) Recursive feature selection/elimination: PyMVPA, PROBID, scikit-learn, TDT
26 Analysis bugs II: exploitation of the models Highest Expected Chance Accuracy % Chance Accuracies for N examples and M classifiers 10 Examples 20 Examples 30 Examples 40 Examples Number of Classifiers Figure 1. The highest expected chance accuracy as a function of the number of examples and classifiers. Each line represents a different number of examples. T he x-axis is the number of classifiers and the y-axis is the accuracy. Palatucci, M. and Carlson, A. (2008). On the chance accuracies of large collections of classifiers. In Proceedings of the 25th International Conference on Machine Learning
27 Analysis bugs II: exploitation of the models Prevention mechanisms Nested CV model selection (scikit-learn, PRoNTo, TDT, PyMVPA) Some toolboxes restrict variety of available classifiers to mitigate Some expose as many as possible to demonstrate it: PyMVPA: clfswh comes with >= 36 of ready-to-be-abused clfs (including a few Random ones)
28 Analysis bugs II: exploitation of the models Prevention mechanisms Nested CV model selection (scikit-learn, PRoNTo, TDT, PyMVPA) Some toolboxes restrict variety of available classifiers to mitigate Some expose as many as possible to demonstrate it: PyMVPA: clfswh comes with >= 36 of ready-to-be-abused clfs (including a few Random ones) Recommendations Establish the best pipeline on an independent sample/study Verify absent results on random/unrelated data
29 Significance estimation H0 distribution estimation (randomization approaches) Dummy classifiers (PyMVPA, nilearn/scikit-learn, PRoNTo) Random, from another experiment(s), outside of the brain data MC permutation (PyMVPA, nilearn/scikit-learn, CoSMoMVPA, PROBID (2 class), TDT, PRoNTo) a must #1: within each run (we seems to be in clear) a must #2: permutation for all CV folds at once maintaining temporal structure. PyMVPA: maintaining target labeling in test portion only labels reassignment (strategy= uattrs ) reassignment of labeling across sessions/chunks (strategy= chunks ) All can do semi-parametric; PyMVPA can also perform semi-parametric estimation
30 Significance estimation: searchlights Make it feasible + multiple comparison problems Simple classifiers == fast: GNB/M1NN searchmight, PyMVPA Spatial sub-sampling (Björnsdotter et al., 2011): PyMVPA Per-subject randomization + bootstrap (Stelzer et al., 2013): PyMVPA (cluster-level with some minor mods + FDR correction on cluster level p s) CoSMoMVPA (cluster-level based, with TFCE correction) Flipping around chance-level of actual performance metrics to simulate chance distribution of the mean (CoSMoMVPA)
31 Summary With great power comes great responsibility Uncle Ben Significance testing should provide support but not the ultimate verdict MVPA is/can be more sensitive to experimental design flaws Avoid common pitfalls: good randomization of trial orders and scrutiny of the design and results is a must Existing MVPA toolboxes provide a complementary spectrum of methodologies to help avoiding pitfalls and provide statistical assessments of the results
32 Thank you! References Björnsdotter, M., Rylander, K., and Wessberg, J. (2011). A monte carlo method for locally multivariate brain mapping. NeuroImage, 56(2): Palatucci, M. and Carlson, A. (2008). On the chance accuracies of large collections of classifiers. In Proceedings of the 25th International Conference on Machine Learning. Stelzer, J., Chen, Y., and Turner, R. (2013). Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): Random permutations and cluster size control. NeuroImage, 65(0): Wright, D. B. (2009). Ten statisticians and their impacts for psychologists. Perspectives on Psychological Science, 4(6):
Introduction to MVPA. Alexandra Woolgar 16/03/10
Introduction to MVPA Alexandra Woolgar 16/03/10 MVP...what? Multi-Voxel Pattern Analysis (MultiVariate Pattern Analysis) * Overview Why bother? Different approaches Basics of designing experiments and
Brain mapping 2.0: Putting together fmri datasets to map cognitive ontologies to the brain. Bertrand Thirion
Brain mapping 2.0: Putting together fmri datasets to map cognitive ontologies to the brain Bertrand Thirion bertrand.thirion@inria.fr 1 Outline Introduction: statistical inference in neuroimaging and big
Network-based pattern recognition models for neuroimaging
Network-based pattern recognition models for neuroimaging Maria J. Rosa Centre for Neuroimaging Sciences, Institute of Psychiatry King s College London, UK Outline Introduction Pattern recognition Network-based
Identification of Neuroimaging Biomarkers
Identification of Neuroimaging Biomarkers Dan Goodwin, Tom Bleymaier, Shipra Bhal Advisor: Dr. Amit Etkin M.D./PhD, Stanford Psychiatry Department Abstract We present a supervised learning approach to
Supplemental Material
1 Supplemental Material Golomb, J.D, and Kanwisher, N. (2012). Higher-level visual cortex represents retinotopic, not spatiotopic, object location. Cerebral Cortex. Contents: - Supplemental Figures S1-S3
Inferential Brain Mapping
Inferential Brain Mapping Cyril Pernet Centre for Clinical Brain Sciences (CCBS) Neuroimaging Sciences Edinburgh Biennial SPM course 2019 @CyrilRPernet Overview Statistical inference is the process of
Finding the Augmented Neural Pathways to Math Processing: fmri Pattern Classification of Turner Syndrome Brains
Finding the Augmented Neural Pathways to Math Processing: fmri Pattern Classification of Turner Syndrome Brains Gary Tang {garytang} @ {stanford} {edu} Abstract The use of statistical and machine learning
Functional Elements and Networks in fmri
Functional Elements and Networks in fmri Jarkko Ylipaavalniemi 1, Eerika Savia 1,2, Ricardo Vigário 1 and Samuel Kaski 1,2 1- Helsinki University of Technology - Adaptive Informatics Research Centre 2-
Advanced Data Modelling & Inference
Edinburgh 2015: biennial SPM course Advanced Data Modelling & Inference Cyril Pernet Centre for Clinical Brain Sciences (CCBS) Neuroimaging Sciences Modelling? Y = XB + e here is my model B might be biased
Explore the brain with Nilearn
Explore the brain with Nilearn Darya Chyzhyk Parietal team, INRIA, Paris-Saclay 2017 Content Concept MRI Data Decoding Functional connectivity Joblib Visualization Documentation Concept Concept How to
the body and the front interior of a sedan. It features a projected LCD instrument cluster controlled
Supplementary Material Driving Simulator and Environment Specifications The driving simulator was manufactured by DriveSafety and included the front three quarters of the body and the front interior of
NILab. NeuroInformatics Laboratory
NILab NeuroInformatics Laboratory NILab Premise Office Building, Mattarello Center for Information Technology Fondazione Bruno Kessler www.fbk.eu Center for Mind/Brain Sciences Università degli Studi di
Rajeev Raizada: Statement of research interests
Rajeev Raizada: Statement of research interests Overall goal: explore how the structure of neural representations gives rise to behavioural abilities and disabilities There tends to be a split in the field
Classification of Honest and Deceitful Memory in an fmri Paradigm CS 229 Final Project Tyler Boyd Meredith
12/14/12 Classification of Honest and Deceitful Memory in an fmri Paradigm CS 229 Final Project Tyler Boyd Meredith Introduction Background and Motivation In the past decade, it has become popular to use
Massachusetts Institute of Technology and University of Cambridge Sir Henry Wellcome postdoctoral fellow
Kamila Maria Jozwik jozwik.kamila@gmail.com +16174016378 jozwik.kamila skype Education Massachusetts Institute of Technology and Sir Henry Wellcome postdoctoral fellow 2018 - Free University Berlin Humboldt
Reliability of Ordination Analyses
Reliability of Ordination Analyses Objectives: Discuss Reliability Define Consistency and Accuracy Discuss Validation Methods Opening Thoughts Inference Space: What is it? Inference space can be defined
Classification on Brain Functional Magnetic Resonance Imaging: Dimensionality, Sample Size, Subject Variability and Noise
Classification on Brain Functional Magnetic Resonance Imaging: Dimensionality, Sample Size, Subject Variability and Noise Jean Honorio CSAIL, MIT Cambridge, MA 039, USA jhonorio@csail.mit.edu Abstract
Classification and Statistical Analysis of Auditory FMRI Data Using Linear Discriminative Analysis and Quadratic Discriminative Analysis
International Journal of Innovative Research in Computer Science & Technology (IJIRCST) ISSN: 2347-5552, Volume-2, Issue-6, November-2014 Classification and Statistical Analysis of Auditory FMRI Data Using
MSc Neuroimaging for Clinical & Cognitive Neuroscience
MSc Neuroimaging for Clinical & Cognitive Neuroscience School of Psychological Sciences Faculty of Medical & Human Sciences Module Information *Please note that this is a sample guide to modules. The exact
Representational similarity analysis
School of Psychology Representational similarity analysis Dr Ian Charest Representational similarity analysis representational dissimilarity matrices (RDMs) stimulus (e.g. images, sounds, other experimental
Alison M. Mattek Department of Psychology, Stanford University 420 Serra Mall, Stanford, CA 94305
Alison M. Mattek Department of Psychology, Stanford University 420 Serra Mall, Stanford, CA 94305 EDUCATION & EMPLOYMENT Email: amattek@stanford.edu 2017-present Postdoctoral Fellow, Stanford University
Voxel-based Lesion-Symptom Mapping. Céline R. Gillebert
Voxel-based Lesion-Symptom Mapping Céline R. Gillebert Paul Broca (1861) Mr. Tan no productive speech single repetitive syllable tan Broca s area: speech production Broca s aphasia: problems with fluency,
Combining machine learning and matching techniques to improve causal inference in program evaluation
bs_bs_banner Journal of Evaluation in Clinical Practice ISSN1365-2753 Combining machine learning and matching techniques to improve causal inference in program evaluation Ariel Linden DrPH 1,2 and Paul
Applying Machine Learning Methods in Medical Research Studies
Applying Machine Learning Methods in Medical Research Studies Daniel Stahl Department of Biostatistics and Health Informatics Psychiatry, Psychology & Neuroscience (IoPPN), King s College London daniel.r.stahl@kcl.ac.uk
Multilevel data analysis
Multilevel data analysis TUTORIAL 1 1 1 Multilevel PLS-DA 1 PLS-DA 1 1 Frequency Frequency 1 1 3 5 number of misclassifications (NMC) 1 3 5 number of misclassifications (NMC) Copyright Biosystems Data
Using AI to interpret BI: machine learning for decoding and characterization of brain activity patterns
Using AI to interpret BI: machine learning for decoding and characterization of brain activity patterns Malin Björnsdotter, Simon Beckmann, Erik Ziegler & Johan Wessberg Abstract The appealing properties
arxiv: v1 [stat.ml] 21 Sep 2017
![arxiv: v1 [stat.ml] 21 Sep 2017](/thumbs/76/74489199.jpg "arxiv: v1 [stat.ml] 21 Sep 2017")
Yet Another ADNI Machine Learning Paper? Paving The Way Towards Fully-reproducible Research on Classification of Alzheimer s Disease Jorge Samper-González 1,2, Ninon Burgos 1,2, Sabrina Fontanella 1,2,
Two-Way Independent ANOVA
Two-Way Independent ANOVA Analysis of Variance (ANOVA) a common and robust statistical test that you can use to compare the mean scores collected from different conditions or groups in an experiment. There
CIS : Computational Reproducibility
CIS 602-01: Computational Reproducibility Introduction Dr. David Koop About Me 3rd Year at UMass Dartmouth Research Interests: - Visualization - Computational Provenance & Reproducibility - Data Science
NeuroImage 61 (2012) Contents lists available at SciVerse ScienceDirect. NeuroImage. journal homepage:
NeuroImage 61 (2012) 1113 1119 Contents lists available at SciVerse ScienceDirect NeuroImage journal homepage: www.elsevier.com/locate/ynimg The advantage of brief fmri acquisition runs for multi-voxel
Interpreting fmri Decoding Weights: Additional Considerations
Interpreting fmri Decoding Weights: Additional Considerations P.K. Douglas Modeling and Simulation Department,UCF Department of Psychiatry and Biobehavioral Medicine, UCLA Los Angeles, CA 90024 pdouglas@ist.ucf.edu
In this second module in the clinical trials series, we will focus on design considerations for Phase III clinical trials. Phase III clinical trials
In this second module in the clinical trials series, we will focus on design considerations for Phase III clinical trials. Phase III clinical trials are comparative, large scale studies that typically
Postdoctoral and research scientist positions
Postdoctoral and research scientist positions Postdoctoral Positions 1. Brain Injury in Contact Sports 2. Imaging Genetics 3. Multi-parametric Tumor Quantification 4. Deep Learning for Neuroimaging 5.
3. Model evaluation & selection
Foundations of Machine Learning CentraleSupélec Fall 2016 3. Model evaluation & selection Chloé-Agathe Azencot Centre for Computational Biology, Mines ParisTech chloe-agathe.azencott@mines-paristech.fr
Chapter 5. Summary and Conclusions! 131
! Chapter 5 Summary and Conclusions! 131 Chapter 5!!!! Summary of the main findings The present thesis investigated the sensory representation of natural sounds in the human auditory cortex. Specifically,
Analysis A step in the research process that involves describing and then making inferences based on a set of data.
1 Appendix 1:. Definitions of important terms. Additionality The difference between the value of an outcome after the implementation of a policy, and its value in a counterfactual scenario in which the
Inference with Difference-in-Differences Revisited
Inference with Difference-in-Differences Revisited M. Brewer, T- F. Crossley and R. Joyce Journal of Econometric Methods, 2018 presented by Federico Curci February 22nd, 2018 Brewer, Crossley and Joyce
Comparing functional connectivity based predictive models across datasets
Comparing functional connectivity based predictive models across datasets Kamalaker Dadi, Alexandre Abraham, Mehdi Rahim, Bertrand Thirion, Gaël Varoquaux To cite this version: Kamalaker Dadi, Alexandre
Evolutionary Programming
Evolutionary Programming Searching Problem Spaces William Power April 24, 2016 1 Evolutionary Programming Can we solve problems by mi:micing the evolutionary process? Evolutionary programming is a methodology
Attention Selectively Reshapes the Geometry of Distributed Semantic Representation
Cerebral Cortex, August 2017;27: 4277 4291 doi: 10.1093/cercor/bhx138 Advance Access Publication Date: 7 June 2017 Original Article ORIGINAL ARTICLE Attention Selectively Reshapes the Geometry of Distributed
Using confusion matrices to estimate mutual information between two categorical measurements
2013 3rd International Workshop on Pattern Recognition in Neuroimaging Using confusion matrices to estimate mutual information between two categorical measurements Dirk B. Walther (bernhardt-walther.1@osu.edu)
Introduction to Computational Neuroscience
Introduction to Computational Neuroscience Lecture 10: Brain-Computer Interfaces Ilya Kuzovkin So Far Stimulus So Far So Far Stimulus What are the neuroimaging techniques you know about? Stimulus So Far
Statistical Analysis Methods for the fmri Data
Basic and Clinical Summer 2011, Volume 2, Number 4 Statistical Analysis Methods for the fmri Data Mehdi Behroozi 1, Mohammad Reza Daliri 1, Huseyin Boyaci 2 1. Biomedical Engineering Department, Faculty
Spatial Normalisation, Atlases, & Functional Variability
Spatial Normalisation, Atlases, & Functional Variability Jörn Diedrichsen Institute of Cognitive Neuroscience, University College London Overview Cerebellar normalisation Anatomical reference High-resolution
Shared Response Model Tutorial! What works? How can it help you? Po-Hsuan (Cameron) Chen and Hejia Hasson Lab! Feb 15, 2017!
Shared Response Model Tutorial! What works? How can it help you? Po-Hsuan (Cameron) Chen and Hejia Zhang!!! @ Hasson Lab! Feb 15, 2017! Outline! SRM Theory! SRM on fmri! Hands-on SRM with BrainIak! SRM
Progress in Risk Science and Causality
Progress in Risk Science and Causality Tony Cox, tcoxdenver@aol.com AAPCA March 27, 2017 1 Vision for causal analytics Represent understanding of how the world works by an explicit causal model. Learn,
Journal of Serendipitous and Unexpected Results
Journal of Serendipitous and Unexpected Results Neural Correlates of Interspecies Perspective Taking in the Post-Mortem Atlantic Salmon: An Argument For Proper Multiple Comparisons Correction Craig M.
Prediction of Successful Memory Encoding from fmri Data
Prediction of Successful Memory Encoding from fmri Data S.K. Balci 1, M.R. Sabuncu 1, J. Yoo 2, S.S. Ghosh 3, S. Whitfield-Gabrieli 2, J.D.E. Gabrieli 2 and P. Golland 1 1 CSAIL, MIT, Cambridge, MA, USA
Distractor-resistant Short-Term Memory Is Supported by Transient Changes in Neural Stimulus Representations
Distractor-resistant Short-Term Memory Is Supported by Transient Changes in Neural Stimulus Representations Jan Derrfuss 1,2, Matthias Ekman 1, Michael Hanke 3,4, Marc Tittgemeyer 5, and Christian J. Fiebach
Functional connectivity in fmri
Functional connectivity in fmri Cyril Pernet, PhD Language and Categorization Laboratory, Brain Research Imaging Centre, University of Edinburgh Studying networks fmri can be used for studying both, functional
RANDOMIZATION. Outline of Talk
RANDOMIZATION Basic Ideas and Insights Marvin Zelen Harvard University Graybill Conference Ft.Collins, Colorado June 11, 2008 Outline of Talk Introduction Randomized vs. Observational Studies Conditioning
What s wrong with models of ventral temporal cortex?
What s wrong with models of ventral temporal cortex? UCLA, July 2016 Jim Haxby Center for Cognitive Neuroscience, Dartmouth College Center for Mind/Brain Sciences (CIMeC), University of Trento Neural representation
Using a multi variate pattern analysis (MVPA) approach to decode FMRI responses to fear and anxiety.
University of Louisville ThinkIR: The University of Louisville's Institutional Repository Electronic Theses and Dissertations 5-2017 Using a multi variate pattern analysis (MVPA) approach to decode FMRI
Novel techniques for normal tissue toxicity modelling. Laura Cella Institute of Biostructures and Bioimaging National Research Council of Italy
Novel techniques for normal tissue toxicity modelling Laura Cella Institute of Biostructures and Bioimaging National Research Council of Italy Disclosure: I have no conflict of interest An honorarium is
Using population decoding to understand neural content and coding
Using population decoding to understand neural content and coding 1 Motivation We have some great theory about how the brain works We run an experiment and make neural recordings We get a bunch of data
Reporting Checklist for Nature Neuroscience
Corresponding Author: Manuscript Number: Manuscript Type: Scott A. Small NNA49290 Article Reporting Checklist for Nature Neuroscience # Main Figures: 5 # Supplementary Figures: 3 # Supplementary Tables:
Panel: Towards reproducible brain imaging research
WIN Conference, 24th January 2017 Panel: Towards reproducible brain imaging research Camille Maumet, Alexander Bowring, Thomas Nichols Neuroimaging Statistics, IDH, University of Warwick, UK WIN Conference,
Data-driven Structured Noise Removal (FIX)
Hamburg, June 8, 2014 Educational Course The Art and Pitfalls of fmri Preprocessing Data-driven Structured Noise Removal (FIX) Ludovica Griffanti! FMRIB Centre, University of Oxford, Oxford, United Kingdom
Experimental design for Cognitive fmri
Experimental design for Cognitive fmri Alexa Morcom Edinburgh SPM course 2017 Thanks to Rik Henson, Thomas Wolbers, Jody Culham, and the SPM authors for slides Overview Categorical designs Factorial designs
STATS8: Introduction to Biostatistics. Overview. Babak Shahbaba Department of Statistics, UCI
STATS8: Introduction to Biostatistics Overview Babak Shahbaba Department of Statistics, UCI The role of statistical analysis in science This course discusses some biostatistical methods, which involve
Methods of MR Fat Quantification and their Pros and Cons
Methods of MR Fat Quantification and their Pros and Cons Michael Middleton, MD PhD UCSD Department of Radiology International Workshop on NASH Biomarkers Washington, DC April 29-30, 2016 1 Disclosures
Center Speller ( )
Center Speller (008-2015) Matthias Treder, Benjamin Blankertz Technische Universität Berlin, Berlin, Germany September 5, 2016 1 Introduction The aim of this study was to develop a visual speller that
Genomics Research. May 31, Malvika Pillai
Genomics Research May 31, 2018 Malvika Pillai Outline for Research Discussion Why Informatics Read journal articles! Example Paper Presentation Research Pipeline How To Read A Paper If I m aiming to just
Hacking the experimental method to avoid Cognitive biases
Hacking the experimental method to avoid Cognitive biases Cyril Pernet, PhD Centre for Clinical Brain Sciences Neuroimaging Sciences, University of Edinburgh 3 rd March 2017 - Warwick - 10.6084/m9.figshare.4696429
Integrative analysis of survival-associated gene sets in breast cancer
Varn et al. BMC Medical Genomics (2015) 8:11 DOI 10.1186/s12920-015-0086-0 RESEARCH ARTICLE Open Access Integrative analysis of survival-associated gene sets in breast cancer Frederick S Varn 1, Matthew
Behavioral Data Mining. Lecture 4 Measurement
Behavioral Data Mining Lecture 4 Measurement Outline Hypothesis testing Parametric statistical tests Non-parametric tests Precision-Recall plots ROC plots Hardware update Icluster machines are ready for
ARTIFICIAL INTELLIGENCE FOR DIGITAL PATHOLOGY. Kyunghyun Paeng, Co-founder and Research Scientist, Lunit Inc.
ARTIFICIAL INTELLIGENCE FOR DIGITAL PATHOLOGY Kyunghyun Paeng, Co-founder and Research Scientist, Lunit Inc. 1. BACKGROUND: DIGITAL PATHOLOGY 2. APPLICATIONS AGENDA BREAST CANCER PROSTATE CANCER 3. DEMONSTRATIONS
NeuroImage 57 (2011) Contents lists available at ScienceDirect. NeuroImage. journal homepage:
NeuroImage 57 (2011) 113 123 Contents lists available at ScienceDirect NeuroImage journal homepage: www.elsevier.com/locate/ynimg Multi-voxel pattern analysis of fmri data predicts clinical symptom severity
Population Inference post Model Selection in Neuroscience
Population Inference post Model Selection in Neuroscience Genevera I. Allen Dobelman Family Junior Chair, Department of Statistics and Electrical and Computer Engineering, Rice University, Department of
Ayan Sengupta 1,2, Stefan Pollmann 2,3, Michael Hanke 3,4
RESEARCH NOTE Spatial band-pass filtering aids decoding musical genres from auditory cortex 7T fmri [version 2; referees: 2 approved] Ayan Sengupta 1,2, Stefan Pollmann 2,3, Michael Hanke 3,4 1Sir Peter
Improved Transparency in Key Operational Decisions in Real World Evidence
PharmaSUG 2018 - Paper RW-06 Improved Transparency in Key Operational Decisions in Real World Evidence Rebecca Levin, Irene Cosmatos, Jamie Reifsnyder United BioSource Corp. ABSTRACT The joint International
Everything you never wanted to know about circular analysis but were afraid to ask
Everything you never wanted to know about circular analysis but were afraid to ask Nikolaus Kriegeskorte MRC Cognition and Brain Sciences Unit, Cambridge, UK Martin A. Lindquist Department of Statistics,
Gene expression correlates of clinical prostate cancer behavior
Gene expression correlates of clinical prostate cancer behavior Cancer Cell 2002 1: 203-209. Singh D, Febbo P, Ross K, Jackson D, Manola J, Ladd C, Tamayo P, Renshaw A, D Amico A, Richie J, Lander E, Loda
Pattern-information analysis: from stimulus decoding to computational-model testing
Pattern-information analysis: from stimulus decoding to computational-model testing Nikolaus Kriegeskorte Medical Research Council, Cognition and Brain Sciences Unit, 15 Chaucer Road, Cambridge, CB2 7EF,
Addressing Content Validity of PRO Measures: The Unique Case of Rare Diseases
Disclaimer: Presentation slides from the Rare Disease Workshop Series are posted by the Kakkis EveryLife Foundation, for educational purposes only. They are for use by drug development professionals and
Predicting Breast Cancer Survival Using Treatment and Patient Factors
Predicting Breast Cancer Survival Using Treatment and Patient Factors William Chen wchen808@stanford.edu Henry Wang hwang9@stanford.edu 1. Introduction Breast cancer is the leading type of cancer in women
Ecological Statistics
A Primer of Ecological Statistics Second Edition Nicholas J. Gotelli University of Vermont Aaron M. Ellison Harvard Forest Sinauer Associates, Inc. Publishers Sunderland, Massachusetts U.S.A. Brief Contents
Supplementary Materials for: Competition Between Items in Working Memory Leads to Forgetting
Supplementary Materials for: Competition Between Items in Working Memory Leads to Forgetting Jarrod A. Lewis-Peacock 1 and Kenneth A. Norman 2 1 Department of Psychology and Imaging Research Center University
Method Comparison for Interrater Reliability of an Image Processing Technique in Epilepsy Subjects
22nd International Congress on Modelling and Simulation, Hobart, Tasmania, Australia, 3 to 8 December 2017 mssanz.org.au/modsim2017 Method Comparison for Interrater Reliability of an Image Processing Technique
Assurance Cases for Model-based Development of Medical Devices. Anaheed Ayoub, BaekGyu Kim, Insup Lee, Oleg Sokolsky. Outline
Assurance Cases for Model-based Development of Medical Devices Anaheed Ayoub, BaekGyu Kim, Insup Lee, Oleg Sokolsky Outline Introduction State of the art in regulatory activities Evidence-based certification
Identification of Tissue Independent Cancer Driver Genes
Identification of Tissue Independent Cancer Driver Genes Alexandros Manolakos, Idoia Ochoa, Kartik Venkat Supervisor: Olivier Gevaert Abstract Identification of genomic patterns in tumors is an important
PTHP 7101 Research 1 Chapter Assignments
PTHP 7101 Research 1 Chapter Assignments INSTRUCTIONS: Go over the questions/pointers pertaining to the chapters and turn in a hard copy of your answers at the beginning of class (on the day that it is
Machine learning approaches for clinical neuroimaging data
Machine learning approaches for clinical neuroimaging data Janaina Mourao-Miranda Jane Rondina Maria Joao Rosa London, UK 28th September 2012 Machine Learning & Neuroimaging Lab We are interested in developing
Automatic Lung Cancer Detection Using Volumetric CT Imaging Features
Automatic Lung Cancer Detection Using Volumetric CT Imaging Features A Research Project Report Submitted To Computer Science Department Brown University By Dronika Solanki (B01159827) Abstract Lung cancer
Big Image-Omics Data Analytics for Clinical Outcome Prediction
Big Image-Omics Data Analytics for Clinical Outcome Prediction Junzhou Huang, Ph.D. Associate Professor Dept. Computer Science & Engineering University of Texas at Arlington Dept. CSE, UT Arlington Scalable
MS&E 226: Small Data
MS&E 226: Small Data Lecture 10: Introduction to inference (v2) Ramesh Johari ramesh.johari@stanford.edu 1 / 17 What is inference? 2 / 17 Where did our data come from? Recall our sample is: Y, the vector
Reporting Checklist for Nature Neuroscience
Corresponding Author: Manuscript Number: Manuscript Type: Alex Pouget NN-A46249B Article Reporting Checklist for Nature Neuroscience # Main Figures: 7 # Supplementary Figures: 3 # Supplementary Tables:
Designing Experiments... Or how many times and ways can I screw that up?!?
www.geo.uzh.ch/microsite/icacogvis/ Designing Experiments... Or how many times and ways can I screw that up?!? Amy L. Griffin AutoCarto 2012, Columbus, OH Outline When do I need to run an experiment and
PROGRAMME Day 1. Friday 26 th October Session (presentation summaries attached) Speakers
PROGRAMME Day 1 Friday 26 th October Session (presentation summaries attached) Speakers 09:00 09:15 Summer School 2018 Welcome Julia Green 09:15 10:15 Session 1 Introduction (1.0hrs) 1.1 What a Quantitative
Assessing and tuning brain decoders: cross-validation, caveats, and guidelines
Assessing and tuning brain decoders: cross-validation, caveats, and guidelines Gaël Varoquaux, Pradeep Reddy Raamana, Denis Engemann, Andrés Hoyos-Idrobo, Yannick Schwartz, Bertrand Thirion To cite this
Biomedical Machine Learning
Marco F. Ramoni, PhD Children s Hospital Informatics Program Harvard Medical School Dental Informatics and Dental Research Making The Connection NIH, Bethesda, MD June 12-13, 2003 Machine Learning Artificial
Preliminary MEG decoding results Leyla Isik, Ethan M. Meyers, Joel Z. Leibo, and Tomaso Poggio
Computer Science and Artificial Intelligence Laboratory Technical Report MIT-CSAIL-TR-2012-010 CBCL-307 April 20, 2012 Preliminary MEG decoding results Leyla Isik, Ethan M. Meyers, Joel Z. Leibo, and Tomaso
Word Length Processing via Region-to-Region Connectivity
Word Length Processing via Region-to-Region Connectivity Mariya Toneva Machine Learning Department, Neural Computation Carnegie Mellon University Data Analysis Project DAP committee: Tom Mitchell, Robert
SUPPLEMENTARY INFORMATION. Predicting visual stimuli based on activity in auditory cortices
SUPPLEMENTARY INFORMATION Predicting visual stimuli based on activity in auditory cortices Kaspar Meyer, Jonas T. Kaplan, Ryan Essex, Cecelia Webber, Hanna Damasio & Antonio Damasio Brain and Creativity
Supplementary Material for Computer keyboard interaction as an indicator of early Parkinson s disease
Supplementary Material for Computer keyboard interaction as an indicator of early Parkinson s disease L. Giancardo 1,,*, A. Sánchez-Ferro 1-5,, T. Arroyo-Gallego 1,6, I. Butterworth 1, C. S. Mendoza 1,
Testing independent component patterns by inter-subject or inter-session consistency
HUMAN NEUROSCIENCE ORIGINAL RESEARCH ARTICLE published: 22 March 2013 doi: 10.3389/fnhum.2013.00094 by inter-subject or inter-session consistency Aapo Hyvärinen 1,2 * and Pavan Ramkumar 3 1 Department
investigate. educate. inform.
investigate. educate. inform. Research Design What drives your research design? The battle between Qualitative and Quantitative is over Think before you leap What SHOULD drive your research design. Advanced
CHAPTER VI RESEARCH METHODOLOGY
CHAPTER VI RESEARCH METHODOLOGY 6.1 Research Design Research is an organized, systematic, data based, critical, objective, scientific inquiry or investigation into a specific problem, undertaken with the
Statistical Considerations: Study Designs and Challenges in the Development and Validation of Cancer Biomarkers
MD-TIP Workshop at UVA Campus, 2011 Statistical Considerations: Study Designs and Challenges in the Development and Validation of Cancer Biomarkers Meijuan Li, PhD Acting Team Leader Diagnostic Devices