DRAWING INFERENCES FROM GROUP DATA: Construc)ng 2 nd level models and working with ROIs

Transcription

1 DRAWING INFERENCES FROM GROUP DATA: Construc)ng 2 nd level models and working with ROIs

2 So you have group data What now? Whole- brain voxelwise analysis ConstrucJng appropriate 2 nd level models n Characterizing within- subject (repeated measures) effects n Characterizing between- subject (group) effects n Using covariates at the 2 nd level Region- of- interest (ROI) analysis Choosing your ROIs Avoiding voodoo n a.k.a. inferenjal circularity; non- independence errors

3 Working with repeated measures data You have data from a group of subjects and you want to look for acjvity changes across condijons The simplest approach is to create contrast images for each subject and then conduct a one- sample t- test across subjects Contrast images are simply weighted linear combinajons of condijon- specific beta images e.g., Cond A - Cond B: [1-1] Referred to as COPE images in FSL and.con images in SPM

4 Working with repeated measures data Contrasts with more than two condijons: Cond A and Cond B vs. Cond C: [1 1-2] n NOTE: If Cond A & B have vastly different # s of trials, it may be becer to run a new GLM that combines these condijons, or weight each condijon by the # of trials (Cond A Cond B) vs. (Cond C Cond D): [ ] n The interacjon term! Null hypothesis: across subject mean of contrast variable = 0

5 Working with repeated measures data Probing for parametric effects with linear contrast coefficients 3 condijons: [ ] 4 condijons: [ ] 5 condijons: [ ] 6 condijons: [ ] Can also probe for 2 nd order (quadrajc) and 3 rd order (cubic) effects Any set of contrast coefficients can work, so long as they sum to zero Write your contrast coefficients as posijve numbers and then subtract the mean from each number But note that custom contrasts might not be orthogonal to each other

6 An alternajve (slightly more involved) way to code a paired t- test FSL representajon of 2 nd level model 8 subjects 2 condijons (A & B)

7 What if one subject is missing data from a given condijon? RelaJvely common situajon to be faced with E.g., Analysis of Correct vs. Incorrect trials n subject may have no incorrect trials PotenJal solujons: Analysis can be run omiong that subject s data Or, analysis can be run as an unpaired t- test (essenjally considering the two condijons as two separate groups of subjects) n More conservajve, but may somejmes be useful, especially if muljple subjects are missing data

8 Between- subject models We have two groups of subjects (e.g., 9 pajents and 7 controls) with potenjally different cross- subject variance. Two- Sample T- Test (equal variance not assumed) Specify two group memberships so that FSL s FEAT esjmates each group's cross- subject variance separately. SPM s default is also to assume unequal variance We want to test whether or not acjvity between the two groups is equivalent for a given condijon

9 Two- sample unpaired t- test

10 How about two groups with two condi)ons? E.g., Comparing the magnitude of pre- vs. post- training acjvajon changes in pajents and controls Within- subject factor: pre- vs. post- training Across- subject factor: pajents vs. controls One easy- to- implement approach: First compute A- B difference image for each subject Then perform simple two- sample (unpaired) t- test to find group differences n i.e., group x condijon interacjon This basic approach can generalize to any linear combina=on of repeated measures factors

11 How about two groups with two condi)ons? An alternajve approach (slightly more involved):

12 Seong up a repeated measures ANOVA Toy example: 2 subjects; 1 factor with 4 levels Subject 1's 4 levels are Inputs 1,3,5,7 Subject 2's 4 levels are Inputs 2,4,6,8.

13 Single group with covariate You have a single group of subjects and you also have measured age. You would like to see if there is an age effect. What would the model look like? What contrasts would you specify for the age effect?

14 Single group with covariate Use age Use demeaned age Both models will give exactly the same result for C2, but C1 will be different. Slide from Jeane>e Mumford

15 Simulated data Take home: Mean centering is only necessary if you want your PE of column of 1s to be the overall mean Slide from Jeane>e Mumford

16 Two groups with a conjnuous covariate We have two groups and a confounding covariate (depression score). Our primary interest is in the difference of mean brain acjvajon between the two groups. We want to make sure this difference wasn t due to between group differences in depression. What should the model look like? Slide from Jeane>e Mumford

17 Two groups with a conjnuous covariate Okay to demean the confounding measure across all subjects But only necessary if you intend to look at each group vs. baseline (e.g., [1 0 0]) But do NOT demean the confounding measure within group This could remove any confounding effect the measure might have Slide from Jeane>e Mumford

18 Why you shouldn t demean within group What if this is what your data look like? Difference in means is clearly due to range of X sampled, not the group membership Slide from Jeane>e Mumford

19 TesJng the interacjon What if you want to test whether the relajonship between brain acjvity and depression score differs between your groups? Slide from Jeane>e Mumford

20 2 nd level analyses in SPM

21 2 nd level analyses in SPM

22 2 nd level analyses in SPM

23 2 nd level analyses in SPM

24 2 nd level analyses in SPM

25 2 nd level analyses in SPM

26 AlternaJves to voxelwise analysis ConvenJonal fmri stajsjcs compute one stajsjcal comparison per voxel. Advantage: can discover effects anywhere in brain. Disadvantage: low stajsjcal power due to muljple comparisons. Small Volume Correc=on: Only run tests on a small proporjon of voxels (by reducing the search space, allows for more lenient voxel- level stats) Region- of- interest: Pool data across a region for single stajsjcal test. Example: how many comparisons on this slice? Voxelwise: 1600 SVC: 57 ROI: 1 Slide from Chris Rorden SPM SVC ROI

27 Why use ROIs? Convenient way to alleviate the muljple comparisons problems that arise in whole- brain analyses Should s=ll adjust stats for # of ROIs tested! Allows for more hypothesis- driven analyses Exploring the enjre dataset can be unwieldy and somejmes leads to unfocused and highly- speculajve papers ROI results are easier to present and discuss Hemodynamic Jmecourse plots can be informajve ROIs don t require subjects to acjvate the exact same voxel n But an ROI- only analysis is vulnerable to Type II errors!

28 Region- of- interest analysis Choosing the right ROI(s) Anatomically- defined n Atlas- based (e.g., AAL atlas)

29 Region- of- interest analysis Choosing the right ROI(s) Anatomically- defined n Automated segmentajon- based (e.g., Freesurfer)

30 Region- of- interest analysis Choosing the right ROI(s) Anatomically- defined n Automated segmentajon- based (e.g., Freesurfer)

31 Region- of- interest analysis Choosing the right ROI(s) Anatomically- defined n Hand- traced using anatomical landmarks

32 Region- of- interest analysis Choosing the right ROI(s) Anatomically- defined n Coordinate- based (e.g., based on a previous study)

33 Region- of- interest analysis Choosing the right ROI(s) Anatomically- defined n Coordinate- based (e.g., based on a previous study)

34 Region- of- interest analysis Choosing the right ROI(s) Anatomically- defined n Coordinate- based (e.g., based on a previous study)

35 Region- of- interest analysis Choosing the right ROI(s) Anatomically- defined n Coordinate- based (e.g., based on a previous study) Power et al. (2013) Curr Opin Neurobio

36 Region- of- interest analysis Choosing the right ROI(s) Func=onally- defined n Based on a parjcular acjvajon effect in your data n ROI- defining contrast needs to be orthogonal to stajsjcal tests conducted on extracted data n Independent localizer tasks can be useful here n AVOID DOUBLE- DIPPING AT ALL COSTS!

37 Examples of stajsjcal circularity To invesjgate whether empathic responses are modulated as a funcjon of the perceived fairness of others we iden=fied, for men and women, peak voxels of ac=va=on in bilateral FI observed when pain was applied to self and to fair players from the analysis described above (see Fig. 2a, b). Figure 2c, d illustrates the average acjvajon (parameter esjmates) for painful non- painful sjmulajon in these voxels when subjects observed either fair or unfair players in receipt of pain. This analysis revealed that less empathic ac=vity was elicited by the knowledge that an unfair player was in pain.

38 Examples of stajsjcal circularity

39 Examples of stajsjcal circularity ROIs defined by linear contrast across memory levels: [ ] What s wrong with this results figure?

40 Another (all too common) problem Cap)on: Highlighted in gray (D) is the anterior lev inferior frontal gyrus, in which the stajsjcal pacern is disjnguished from C because IR < AR = F Main Text: Thus, disjnct acjvajon pacerns were seen along the antero- posterior extent of the inferior frontal gyrus, with alifg showing the pacern IR < AR = F and plifg showing IR< AR < F. Need to explicitly test for the REGION X CONDITION interac=on!

41 Another (all too common) problem Nieuwenhuis et al. (2011) Nature Neuroscience

42 ROI definijon is affected by noise overfitted ROI true region independent ROI ROI-average activation overestimated effect Slide from Niko Kriegeskorte

43 ROI definijon is affected by noise Baker, Hutchison, & Kanwisher (2007) Slide from Ed Vul High selectivity from pure noise.

44 Voodoo correlajons Ed Vul

45 Voodoo correlajons

46 Voodoo Puzzlingly high correlajons

47 Voodoo correlajons

48 Voodoo correlajons Neural correlates of human virtue judgment r = 0.82 r = -0.83

49 Voodoo correlajons This figure is the only real data

50 Voodoo correlajons To sum up, then, we are led to conclude that a disturbingly large, and quite prominent, segment of fmri research on emojon, personality, and social cognijon is using seriously defecjve research methods and producing a profusion of numbers that should not be believed. Although we have focused here on studies relajng to emojon, personality, and social cognijon, we suspect that the quesjonable analysis methods discussed here are also widespread in other fields that use fmri to study individual differences, such as cognijve neuroscience, clinical neuroscience, and neurogenejcs.

51 To avoid selecjon bias, we can......perform a nonselec=ve analysis OR e.g. whole- brain mapping (no ROI analysis)...make sure that selecjon and results stajsjcs are independent under the null hypothesis, because they are either: inherently independent or computed on independent data e.g. independent contrasts e.g., separate study; func=onal localizer; cross- valida=on

52 For more informajon Puzzlingly High Correla=ons in fmri Studies of Emo=on, Personality, and Social Cogni=on. Vul, E., Harris C., Winkielman, P., & Pashler, H. (2009) PerspecJves on Psychological Science, 4, [Formerly Jtled: Voodoo CorrelaJons in Social Neuroscience]. Circular analysis in systems neuroscience the dangers of double dipping. Kriegeskorte N, Simmons WK, Bellgowan PSF, Baker CI. (2009) Nature Neuroscience 12(5): Everything you never wanted to know about circular analysis, but were afraid to ask. Kriegeskorte N, Lindquist MA, Nichols TE, Poldrack RA, Vul E. (2010) J Cereb Blood Flow Metab. 30(9): Voodoo and circularity errors. Vul, E. and Pashler, H. (2012). NeuroImage, 62,