The Human Connectome Project multimodal cortical parcellation: new avenues for brain research.

The Human Connectome Project multimodal cortical parcellation: new avenues for brain research. Dr Emma. C. Robinson emma.robinson05@imperial.ac.uk, Biomedical Engineering

Overview A simple model of the human brain Modelling global properties of brain organisation from MRI The Human Connectome Projects s A Multi-modal Parcellation of the Human Cerebral Cortex Comparing patterns of brain connectivity against behavioural/cognitive/genetic markers Future Challenges

A simple model of the human brain A relatively small number of regions Each region has consistent connectivity to each layer of the cortex Each region has a specialised set of functions c/o The MGH Human Connectome project Gallery c/o The WU-MINN Human Connectome Project (Nature)

A simple model of the human brain Important for: - Models of cognition - Study of the mechanisms behind conditions such as Autism or Schizophrenia - Design of Artificial Intelligence systems. c/o The MGH Human Connectome project Gallery Brain network c/o The WU-MINN Human Connectome Project (Nature, in Press)

The Scale of the Human Brain ~100 billion cells ~100 trillion connections micrometre scale

Magnetic Resonance Imaging In vivo and non invasive Multi-modality: - Structural imaging - diffusion weighted imaging - approximates neural pathways - functional imaging - approximates brain activations

Modelling brain organisation from MRI Clustering algorithms: K-means ICA Spectral clustering Matrix factorisation Parisot, Sarah, et al. IPMI, 2015. O'Donnell, Lauren J., and Carl-Fredrik Westin. TMI 26.11 (2007): 1562-1575.

Modelling brain organisation from MRI Different data driven parcellations of the adult human brain: Arslan, S., Ktena, S.I., Makropoulos, A., Robinson, EC., Rueckert, D., Parisot, S., 2017, Human brain mapping: A systematic comparison of brain parcellation methods for the human cerebral cortex, NeuroImage. (In Press)

Modelling brain organisation from MRI Limitations of data driven approaches Individual imaging data sets are very noisy: Subject to physiological and imaging artefacts Low resolution Indirect Modelling error Disagreement between modalities Cortical folding patterns & functional activations do not agree Fischl, Bruce, et al. "Cortical folding patterns and predicting cytoarchitecture." Cerebral cortex 18.8 (2008): 1973-1980. No ground truth!!

The HCP Multi-modal Parcellation Expert manual annotations of 180 functionally specialised regions on group average data 97 entirely new areas 83 areas previously reported by histological studies Region 55b as identified across modalities and (h) as reported from histology

The HCP Multi-modal Parcellation Made possible by comparing data across subjects AND modalities Manual annotation of sub-regions of the visual cortex Image boundaries compared against cyto-architectonic maps

Improving SNR through spatial normalisation Map/deform data to a common space where same structures or functional activations are found at each location e.g. smooth warping of a structural MRI volume until subject A looks more like subject B A B

Cortical Surface Processing HCP data is projected to the cortical surface for two reasons: 1. Surface based smoothing improves SNR 2. Surface-based registration improves alignment of cortical folds

Multi-modal Surface Matching (MSM) Spherical framework for cortical surface registration Use low resolution control point grids to constrain the deformation Optimised using discrete methods Modular Robinson, Emma C., et al. "MSM: a new flexible framework for multimodal surface matching." Neuroimage 100 (2014): 414-426.

MSM framework c 1 and c 2 represent cliques (groups of control point nodes)* *Robinson, Emma C., et al. Multimodal surface matching with higher order smoothness constraints (in revision)

Driving alignment using multi-modal features Curvature Myelin* Glasser, 2011. J. Neurosci, 31 11597-11616 *reflects patterns of cellular organisation Task/rest fmri Structural Connectivity Sotiropoulos et al NeuroImage 2016

Smith, Stephen M., et al. "Functional connectomics from resting-state fmri." Trends in cognitive sciences 17.12 (2013): 666-682. MSMall: Alignment driven multivariate feature vectors myelin (M) and rfmri (R) and visuotopic (V) Improves alignment of task fmri feature sets M R { V {

The HCP Multi-modal Parcellation Regional boundaries found by looking for imaging gradients in group average data Looking for patterns common across multiple modalities Informed by the literature where available

The HCP Multi-modal Parcellation Regional boundaries found by looking for imaging gradients in group average data Looking for patterns common across multiple modalities Informed by the literature where available

The HCP Multi-modal Parcellation: propagating the result to individuals Single subject parcellations were then obtained by training MLP classifiers Binary classifications Group average data propagated to training subjects used to train classifier ONLY where subject data closely agrees with group Hacker, Carl D., et al. "Resting state network estimation in individual subjects." Neuroimage 82 (2013): 616-633.

The HCP Multi-modal Parcellation: propagating the result to individuals Output from Classifier for 4 example datasets Group Average Classifier results for 4 subjects

The HCP Multi-modal Parcellation: accurate detection of regions across test subjects Top = Training Set; Bottom = Test Set Darker orange indicates regions that were not detected in all subjects (or were detected by with very low surface areas)

The HCP Multi-modal Parcellation: high consistency in group average parcellation between training and test sets Top = manual annotation; Bottom = overlap of training and test set classifier results Blue borders= Train set; Red borders= Test set; Purple=overlap

The HCP Multi-modal Parcellation: Advantages: Consistent with known patterns of cellular organisation (cyto-architecture) Consistent with patterns of functional organisation Generalisable to new subjects Independently validated on 210 test subjects Provides standardised reference framework aids in the clarity and efficiency of communicating results

http://fdeligianni.site/basics.html Network Modelling Estimating connectivity networks: Functional connectivity Correlation/partial correlation of patterns of functional activity Structural connectivity Estimates of the structural integrity of DTI based estimates of neural connectivity

Predicting Cognition and Behaviour Conventionally brain network models have been studied through graph theory Networks are the collection of regions (nodes) and their connections (edges) Graph theory techniques explore global properties of the graph i.e. Clustering coefficients Path lengths node degree Modularity Hagmann, van den Heuvel, Patric, Martijn et al. "Mapping P., and Olaf the structural Sporns. "Network core of human hubs in cerebral the human cortex." brain." PLoS Trends Biol in 6.7 cognitive (2008): e159. sciences 17.12 (2013): 683-696.

Predicting Cognition and Behaviour Machine learning approaches are now becoming more popular Prediction of age/gender/developmental outcome/disease progression Using: Classification Regression Unsupervised Learning - CCA White-matter tract regions associated with age at scan (A) and postconceptional age at birth (B). Pandit, A. S., et al. "Whole-brain mapping of structural connectivity in infants reveals altered connection strength associated with growth and preterm birth." Cerebral cortex 24.9 (2014): 2324-2333.

Predicting Cognition and Behaviour Machine learning approaches are now becoming more popular Prediction of age/gender/developmental outcome/disease progression Using: Classification Regression Unsupervised Learning - CCA **** HCP data comes with 280 behavioural and demographic measures ******

Current limitations of population-based neuroimaging Population-based analysis are not yet sensitive enough to make accurate predictions about individuals Why? Imaging studies assume that at coarse scale all brains are the same i.e. fixed number of regions Regional organisation of an average human brain Appear in the same place in all brains Allows us to map data to a global average space for comparison HCP cortical segmentation v1.0

Current limitations of population-based neuroimaging Population-based analysis are not yet sensitive enough to make accurate predictions about individuals But Evidence that suggests that brains vary topologically Topological variability in the human brain Group 1 A B C e.g. Van Essen, David C. "A population-average, landmark-and surfacebased (PALS) atlas of human cerebral cortex." Neuroimage 28.3 (2005): 635-662. Glasser, Matthew F., et al. "A multi-modal parcellation of human cerebral cortex." Nature (2016). Amunts, K., A. Schleicher, and K. Zilles. "Cytoarchitecture of the cerebral cortex more than localization." Neuroimage 37.4 (2007): 1061-1065. Group 2 A C B

Topological Variance in the HCP feature-set Van Essen, David C. "A population-average, landmark-and surface-based (PALS) atlas of human cerebral cortex." Neuroimage 28.3 (2005): 635-662. Glasser, Matthew F., et al. "A multi-modal parcellation of human cerebral cortex." Nature (2016). FEF PEF 55b

Future Challenges To improve the sensitivity of future analysis we must consider: New approaches for spatial normalisation Improve multi-modal integration Account for topological variation and functional non-stationarity Enhanced predictive models: Account for correlations between behavioural variables Do not rely on global average models of brain organisation

Conclusions The HCP v 1.0 multi-modal parcellation: Cytoarchitecturally and functionally consistent Sensitive & Robust reference framework Future iterations of the method will Map labels to diseased or developing populations Capture greater individual variation Increase sensitivity to subtle differences in behaviour/cognition/genetics disease

Acknowledgements Prof. David Van Essen Matthew Glasser Tim Coalson Dr Carl Hacker Prof. Mark Jenkinson Prof. Steven Smith Prof. Saad Jbadi Dr Stamatios Sotiropoulos Prof. Daniel Rueckert Dr Bernhard Mainz Dr Ben Glocker Dr Martin Rajchl Ira Ktena Salim Arslan Dr Sarah Parisot Prof Jo Hajnal Prof David Edwards Prof Julia Schnabel

Structure does not always align microstructure Cyto-architectonics the subdivision of the brain based on cellular composition The relative placement of cytoarchitechtonic regions within a sulcus varies across subjects Amunts, Schleicher, Zilles 2007 V1 Broca s Alignment of cytoarchitectonic regions using morphological alignment leads to variable degrees of regional overlap Amunts, FIschl Zilles, et al. Fischl Cortical folding patterns and predicting cytoarchitecture. (2008)

MYELIN CURV MYELIN CURV MSM for multimodal alignment 3D feature sets: sulcal depth, curvature and myelin Do not agree on optimal alignment. Registration driven using multimodal metric: α MI A. UNIVARIATE B. NO WEIGHTING E VARIABLE WEIGHTING Cost function weighting used to up/downweight features locally C. UPWEIGHTED FOLDS D. UPWEIGHTED MYELIN This can lead to an improved joint-alignment of these features

MSM framework c 1 and c 2 represent cliques (groups of control point nodes) In the original MSM framework: c 1 = unary cost c 2 = pairwise cost

MSM with higher order clique reduction Higher order Clique reduction (proposed by Ishikawa CVPR 2009, 2014) Reduces higher order cliques to pairwise In the new MSM framework - Multimodal Surface Matching with Higher-Order Smoothness Penalties: for Alignment of Cortical Anatomies (in preparation) - c 1 = triplet data cost

MSM with higher order clique reduction - c 2 = triplet deformation penalty such as Glocker, Ben, et al. "Triangleflow: Optical flow with triangulation-based higher-order likelihoods." European Conference on Computer Vision. Springer Berlin Heidelberg, 2010.

The HCP Multi-modal Parcellation: improved statistical significance

The HCP multimodal parcellation of the Human Cerebral cortex However, not all subjects brains are topologically consistent 55 b FEF PEF A Multi-modal Parcellation of Human Cerebral Cortex Glasser et al. Nature (in press)

Training data = 110 D feature vectors Cortical thickness Cortical myelin Cortical curvature 20 task ICA + mean 77 rest ICA 5 hand engineered visuotopic features Feature set

Group-wise Discrete Registration A x x x k G 0 B x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x G i G i G 2 G n C S i I 1 I 2 I 2

Group-wise Discrete Registration Pairwise similarity TRIPLET regularisation QUARTET global cost Inclusion of Higher-Order terms is made possible through clique reduction techniques, Ishikawa 2009, 2014

Experiments and Results Warp distortion Pairwise feature correlations 1 = registration to template 2 = pairwise registration 3 = group-wise registration Tested on cortical folding alignment of 10 HCP subjects

Experiments and Results 1.3 GROUP AVERAGES ` -1.3 >0.3 HCP TEMPLATE SINGLE REFERENCE PAIRWISE GROUPWISE Discrete Optimisation for Group-wise Cortical Surface Atlasing E.C. Robinson et al. The Workshop on Biomedical Image Regisration (WBIR) 2016

HCP Gender Classification Result Cross-validated performance for best parameters: Random Forest = 87.6% Linear (SVM) classifier = 86.6% cv performance without feature selection =77% L R Feature Importance Mapped back to the image space

HCP Fluid Intelligence Predictions Cross-validated performance for best parameters: Random Forest = 0.347 ( r^2 value) Linear (SVM) classifier = 0.340 CV performance without selection =-0.38 L feature R Feature Importance Mapped back to the image space