Population Inference post Model Selection in Neuroscience

Transcription

1 Population Inference post Model Selection in Neuroscience Genevera I. Allen Dobelman Family Junior Chair, Department of Statistics and Electrical and Computer Engineering, Rice University, Department of Pediatrics-Neurology, Baylor College of Medicine, Jan and Dan Duncan Neurological Research Institute, Texas Children s Hospital. October 2, 2016 G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

2 Functional Neuroimaging & Neural Recordings Micro-Scale (Neurons): Calcium-Florescence Imaging Electrophysiology Microscopy Macro-Scale (Brain): Functional MRI (fmri) EEG / MEG Electrocorticography G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

3 Functional Neuroimaging & Neural Recordings Data: For each subject i, X i is p T for p brain regions (or neurons) and T time points. Subject-level data summarized via Model Selection. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

4 Functional Neuroimaging & Neural Recordings Objective: Population Inference. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

5 1 Motivating Case Studies 2 PIMS 3 Empirical Studies G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

6 Motivating Case Study: Neuron Tuning Calcium Florescence Imaging: G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

7 Motivating Case Study: Neuron Tuning Study Design: WT vs. MECP2 mice (two genotypes). Motor Cortex imaged once a week for 8 weeks. Mice ran on treadmill at 5 different speeds during imaging. Neuron Tuning: Neurons who fire more during certain stimuli (tuned to stimuli). Objective: Compare the proportion and strength of tuned neurons between two genotypes longitudinally. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

8 Motivating Case Study: Neuron Tuning Model Selection (Lasso) used to assess neuron tuning. Challenge: Population inference on neuron tuning across genotypes. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

9 Motivating Case Study: Functional Connectivity How does the brain communicate (connect) at a systems level? Functional Connectivity: Estimated relationships between brain regions (usually from fmri data). G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

10 Motivating Case Study: Functional Connectivity G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

11 Motivating Case Study: Functional Connectivity G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

12 Inference for Functional Connectivity: Existing Procedures Step 0: Standard fmri pre-processing & Parcellation. Step 1: Estimate brain networks for each subject. Step 2: Summarize network via network metrics for each subject. Step 3: Use standard statistical inferential methods to test population effects. Brain Connectivity Toolbox G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

13 Inference for Functional Connectivity: Existing Procedures Step 0: Standard fmri pre-processing & Parcellation. Parcellation: reduces fmri volume to regions of interest (ROIs). Anatomical or Functionally derived Atlases. Step 1: Estimate brain networks for each subject. Step 2: Summarize network via network metrics for each subject. Step 3: Use standard statistical inferential methods to test population effects. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

14 Inference for Functional Connectivity: Existing Procedures Step 0: Standard fmri pre-processing & Parcellation. Step 1: Estimate brain networks for each subject. Correlation Networks. Markov Networks (partial correlations). Causal / Directed Networks. Dynamic Networks. Step 2: Summarize network via network metrics for each subject. Step 3: Use standard statistical inferential methods to test population effects. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

15 Inference for Functional Connectivity: Existing Procedures Step 0: Standard fmri pre-processing & Parcellation. Step 1: Estimate brain networks for each subject. Step 2: Summarize network via network metrics for each subject. Step 3: Use standard statistical inferential methods to test population effects. T-tests, linear regression models, permutation null distributions, multiple testing, etc. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

16 A Simulation Test Simulation: Markov Networks - Inference on edges - Two-Group Population. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

17 A Simulation Test Simulation: Markov Networks - Inference on edges - Two-Group Population. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

18 A Simulation Test Simulation: Markov Networks - Inference on edges - Two-Group Population. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

19 1 Motivating Case Studies 2 PIMS 3 Empirical Studies G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

20 PIMS: Population Inference post Model Selection G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

21 PIMS: Population Inference post Model Selection PIMS A two-level problem with: Subject Level: Selection procedure used to estimate subject-level parameters. Population Level: Inference conducted on population-level parameters. Similarities (and differences) from: Post Selection Inference. Classical Hierarchical Models. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

22 PIMS: Population Inference post Model Selection Failure to account for subject-level model selection: Biased estimates of ˆβ (population-level parameters). Support tests. Parameter tests. Under-estimated variance of ˆβ. Results in: Uncontrolled Type I Error. Loss of Statistical Power. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

23 Our Solution General Selection & Inference Procedure: R 3 * Resampling: Resample (bootstrap) subject-level data and perform selection on bootstrapped samples. * Randomized Model Selection: Use randomized model selection on resampled subject-level data. * Random Effects Models: Conduct population inference via GLMMs with subject-level random effects. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

24 Our Solution R 3 Algorithm Summary: 1 Take S bootstrap samples for each subject. For each sample, X s i, perform randomized model selection: ˆθ s i Randomized Model Selection(X s i ) 2 Fit GLMM: g(e(ˆθ ( s) i D, Z)) = D i,s β + Z i,s α for i = 1,... n, s = 1,... S. D denotes the fixed population effects of interest. Z denotes the random subject effects. g() GLM link function. 3 Population Inference: H 0 : β = 0 v.s. H A : β 0. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

25 1 Motivating Case Studies 2 PIMS 3 Empirical Studies G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

26 Simulation Studies Simulation: Markov Networks - Inference on edges - Two-Group Population. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

27 Simulation Studies Simulation: Markov Networks - Inference on Subnetwork Density - Continuous Covariate (i.e. symptom severity) in Population. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

28 Simulation Studies Simulation: Markov Networks - Inference on Subnetwork Density - Continuous Covariate (i.e. symptom severity) in Population. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

29 Case Study: Color-Sequence Synesthesia G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

30 Case Study: Color-Sequence Synesthesia Brain regions that process color have more connections to those that process numbers and letters in synesthetes than controls. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

31 Case Study: Color-Sequence Synesthesia Are there different node clustering patterns between the two groups? G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

32 Summary Population Inference post Model Selection (PIMS) New type of PSI problem. Arises often in neuroscience. Failure to account for model selection leads to uncontrolled Type I error and poor statistical power. General selection and inference procedure: R 3. Future Directions Develop selection and inference procedures for specific types of PIMS problems. Theory: Valid inference with PIMS problems. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

33 Software MoNet: Markov Network Toolbox (Matlab) for Functional Connectivity Available from gallen/ or G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

34 Acknowledgments Manajari Narayan, PhD, Rice Electrical Engineering (Now at Stanford). Michael Weylandt, PhD Candidate, Rice Statistics. Steffie Tomson, BCM. David Eagleman, BCM. Huda Zoghbi, BCM. Stelios Smirnakis, BCM. Andreas Tolias, BCM. G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

35 References M. Narayan and G. I. Allen, Mixed Effects Models for Resampled Network Statistics Improves Statistical Power to Find Differences in Multi-Subject Functional Connectivity, Frontiers in Neuroscience, 10:108, M. Narayan, G. I. Allen & S. Tomson, Two Sample Inference for Populations of Graphical Models with Applications to Functional Connectivity, (Submitted) arxiv: , S. Tomson, M. Narayan, G. I. Allen, & D. Eagleman, Neural Networks of Synesthesia, Journal of Neuroscience, 33:35, , M. Narayan & G. I. Allen, Randomized Approach to Differential Inference in Multi-Subject Functional Connectivity, In IEEE International Workshop on Pattern Recognition in Neuroimaging, M. Narayan & G. I. Allen, Population Inference for Node Level Differences in Functional Connectivity, In IEEE International Workshop on Pattern Recognition in Neuroimaging, S. Tomson, M. Schreiner, M. Narayan, T. Rosser, N. Enrique, A. J. Silva, G. I. Allen, S. Y. Bookheimer, and C. Bearden, Resting state functional MRI reveals abnormal network connectivity in Neurofibromatosis 1, Human Brain Mapping, 36:11, , G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18

36 Thank You! G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, / 18