Personal and Laboratory Events. Marie St-Laurent. A thesis submitted in conformity with the requirements. for the degree of Doctor of Philosophy

Transcription

1 Medial Temporal Lobe Function and the Perceptual Richness of Memory for Complex Personal and Laboratory Events. by Marie St-Laurent A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Psychology University of Toronto Copyright by Marie St-Laurent (2012)

2 Medial Temporal Lobe Function and the Perceptual Richness of Memory for Complex Personal and Laboratory Events Marie St-Laurent Doctor of Philosophy Graduate Department of Psychology University of Toronto, 2012 Abstract Reliving the past requires the integration of multi-modal sensory details into a coherent mental impression of the initial event. In most people, memory for life episodes, or Autobiographical Memory (AM), is rich in sensory-perceptual elements that provide the vivid impression of travelling back in time. Abundant evidence indicates that the hippocampus plays a central role in AM recollection, but much research is still needed to determine which AM attributes engage the hippocampus at retrieval. My work assessed the relationship between hippocampal function and the perceptual richness of memory episodes. I designed a paradigm that captured the complexity of AM, and that manipulated perceptual richness while controlling for other AM confounds, such as recency, rehearsal, personal relevance, and story content. Participants studied and recalled perceptually enriched and impoverished laboratory events (film clips and written narratives, respectively) matched for the complexity of their storyline. An AM condition was also included for comparison. I tested healthy individuals and participants with unilateral medial temporal lobe epilepsy (mtle), a clinical population with well documented hippocampal ii

3 damage, on this paradigm. Perceptual richness was greatly reduced in people with mtle, an effect that was most salient in the perceptually enriched conditions (AM and film clips). In a functional MRI version of this paradigm conducted on healthy individuals, I identified neural regions sensitive to the perceptual richness of AM and laboratory events, which included the anterior portion of the right hippocampus and other regions known to play a role in imagery and visual processing. In patients with right-lateralized mtle, activation in these brain regions was markedly reduced in all memory conditions, which was consistent with the reduced perceptual richness I observed behaviourally. I reveal a clear relationship between hippocampal function and the perceptual richness of episodic memory, suggesting that the hippocampus plays a central role among brain regions that support the integration of multimodal details into enriched memory experiences. My findings also advance our knowledge of how pathology and the nature of memory representation affect the neural correlates of episodic memory. iii

4 Acknowledgments I would like to thank my mentors, Morris and Mary Pat, for their support and guidance through all these years. It s been quite the journey, and I cannot count the ways in which I learned from the both of you, both as people and as scientists. Thanks for believing in me and in this project. I would also like to thank John Paul Koning, my partner and my best and favorite pilot subject, for his love, his patience and his brilliant cooking, but also for providing the golden voice for the narrative voice-over! Many thanks also to Susanne Ferber for her positive energy, her support, her smiling nod during committee meetings, her editing skills and her eagle-eye for typos! Thanks to colleagues and friends at the Toronto Western Hospital, without whom I could not have completed this work. Rachel for her brilliant scoring and much needed transcription help, Irene and Conny for being awesome office mates and for sharing their imaging experience, Melanie for her neuropsych expertise and her flawless recruitment job, Keith and Eugene for putting up with my nerves during scanning, Adrian for his help with T2 relaxometry, and the Epilepsy team for their help with recruitment. Many thanks also to my lab mates in the Moscovitch lab, and to staff at the U of T Psychology Department. Xianwei for her video editing wizardry, Yunjo, Kris, Marnie, Jordan, Signy, Ralluca, Jess, Zhongxu, Susan, Yohav, and Lillian for all their constructive feedback and moral support, Mike Goggan for resurrecting my laptops, and Ann Lang for keeping me on track. Thanks to all the friends who helped making Toronto home: Jing, Andreea and Hana, Tom, Sarah, Lex, Marion, Laurent, and Robin. Thanks to my parents and brothers for trying to understand what I do, and to my extended Koning family for cheering me on during the final stages of writing. Finally, I would like to thank to all my participants for trusting me with their brains and their memories, and for giving their time to this project. This work could not have been possible without financial support from the National Sciences and Engineering Research Council, the Savoy Epilepsy Foundation, the School of Graduate Studies at the University of Toronto, the James S. McDonnell Foundation, and the Canadian Institutes of Health Research. iv

5 Table of Contents Abstract... ii Acknowledgments... iv Table of Contents... v List of Tables... ix List of Figures... x List of Appendices... xii Chapter 1 General Introduction 1.1 Episodic Memory and Determinants of Medial Temporal Engagement Spatio-Temporal Specificity Context-Specific Details Recollection Perceptual Richness, Memory and the MTL Imagery in Theoretical Memory Models Perceptual Memory Richness and Hippocampal Engagement at Retrieval Assessing Perceptual Richness Medial Temporal Lobe Epilepsy Etiology, Characterization and Surgical Management of mtle Impact of mtle on Memory Extra-temporal brain damage in mtle Objectives of the Present Research Chapter 2 The Impact of Temporal Lobe Epilepsy on the Perceptual Richness of Complex Episodic Memories v

6 2.1 Introduction Methods Participants Paradigm Encoding Retrieval Scoring and data analysis Scoring reliability: intraclass correlations Results Self-Ratings Detail Scoring Story and Perceptual details Errors LIWC Effect of Surgical Status Discussion Summary of Findings Self-Ratings Theoretical Implications Laterality and Surgical Status Encoding versus Retrieval Conclusion Chapter 3 The Neural Correlates of the Perceptual Richness of Memory for Complex Personal and Laboratory Episodes 3.1 Introduction Methods Participants vi

7 3.2.2 Procedure Image Acquisition Preprocessing Statistical Analysis Parametric Analysis Results Behavioural Results Functional MRI Results Common memory regions Contrasts among memory conditions Differences in hippocampal activation between memory conditions Parametric ROI analysis Discussion Summary of Findings Neural Correlates of Memory Retrieval Neural Correlates of Perceptual Richness Parametric Analysis Lateralization Differences between AM and Laboratory tasks Encoding versus Retrieval Conclusion Chapter 4 The Impact of Temporal Lobe Epilepsy on the Neural Correlates of Perceptual Richness for Complex Episodic Memories 4.1 Introduction Methods Participants Procedure vii

8 4.2.3 Statistical Analysis Results Behavioural Results Neural Correlates of Autobiographical Memory Neural Correlates of Memory for Film Clips Neural Correlates of Memory for Narratives Differences in Hippocampal Activation Between the RTLE and the Control Groups Discussion Summary of Findings Behavioural Results Comparison Between the Behavioural and the Brain Imaging Results Laterality Conclusion Chapter 5 General Discussion 5.1 Summary of Findings Retrieving Naturalistic and Laboratory Events Lesion Specificity: Hippocampus and the Medial Temporal Lobe Impact of mtle on Encoding versus Retrieval Laterality of Hippocampal Function Age of the Memory Role of the Hippocampus in Memory Nature of the Memory Representation The Role of Imagery in Episodic Memory Limitations Future Directions: mtle s Impact on Functional Connectivity Conclusion viii

9 List of Tables Table 2.1 Mean Demographic and Neuropsychological Characteristics of the Control and mtle groups. 31 Table 2.2 Inter-Rater Reliability: Intraclass Correlation Coefficients per Detail Subcategory 37 Table 3.1 Brain Regions Showing Differences in Activation Between the Counting and the Three Memory Conditions 72 Table 3.2 Brain Regions Showing Differences in Activation Between the Clip and the Narrative Memory Conditions.. 74 Table 3.3 Brain Regions Showing Differences in Activation Between the Autobiographical and the Narrative Memory Conditions..77 Table 3.4 Brain Regions Showing Differences in Activation Between the Autobiographical and the Clip Memory Conditions...78 Table 3.5 Coordinates of Hippocampal Activation Differentiating Between the Experimental Conditions.80 Table 4.1 Mean Demographic and Neuropsychological Characteristics of the Control and the RTLE group..97 Table 4.2 Brain Regions Activated in the Autobiographical Memory Condition, per Group..105 Table 4.3 Brain Regions Showing Greater Activity for Controls than for Patients with RTLE in the Autobiographical Memory Condition.106 Table 4.4 Brain Regions Activated in the Film Clip Memory Condition, per Group 109 Table 4.5 Brain Regions Showing Greater Activity for Controls than for Patients with RTLE in the Film Clip Memory Condition.110 Table 4.6 Brain Regions Activated in the Narrative Memory Condition, per Group. 113 Table 4.7 Brain Regions Showing Greater Activity for Controls than for Patients with RTLE in the Narrative Memory Condition 114 Table 4.8 Coordinates of Hippocampal Activation Differentiating Between Groups for Each Experimental Condition 116 ix

10 List of Figures Figure 1.1 Conway s model of the organization of autobiographical memory..12 Figure 2.1 Recall of three laboratory events by a control participant, scored according to my procedure. 36 Figure 2.2 Mean self-ratings of Story Memory Content and Vividness for all trials, plotted per memory condition for individuals with TLE and for healthy controls 40 Figure 2.3 Mean number of trials for which a memory was retrieved (Story Content rating > 1) plotted per condition for individuals with TLE and for healthy controls...41 Figure 2.4 Mean tallied number of correct Story Details and Perceptual Details per memory condition shown for successful trials only (Story Content rating > 1), and for all trials...42 Figure 2.5 Mean number of incorrect Story and Perceptual Details for the Narrative and Clip conditions. Errors are plotted as a total number, and as a percentage of total (correct and incorrect) details from its category.44 Figure 2.6 Mean number of words per successful trial (Story Content rating > 1) from LIWC2007 s Perceptual Processes and Visual Processes categories 46 Figure 3.1 Mean number of trials retrieved successfully, and mean Story Content and Vividness ratings, per condition.69 Figure 3.2 Mean number of words per trial from LIWC2007 s Verbs and Perceptual Processes categories, plotted per memory condition 70 Figure 3.3 Contrast between all three memory conditions (AM, Narrative and Clip) and the control (Counting) condition 73 Figure 3.4 Contrast between the Clip and the Narrative memory condition (top), the AM and the Narrative condition (middle), and the AM and the Clip condition (bottom) 75 Figure 3.5 Medial temporal voxels showing significantly different levels of activation between the experimental conditions 79 Figure 3.6 Hippocampal voxels whose activity was modulated parametrically by the number of Perceptual Processes words and the number of Verbs in each memory condition.81 Figure 4.1 Mean number of trials retrieved successfully, and mean Story Content and Vividness ratings, per group and experimental condition..101 x

11 Figure 4.2 Mean number of words per successful trial (Story Content rating > 1) from LIWC2007 s Verbs and Perceptual Processes categories..102 Figure 4.3 Contrast between the AM and the Counting condition in controls (top) and in the RTLE group (middle), and regions showing greater levels of activity for this contrast in the control than in the RTLE group (bottom) Figure 4.4 Contrast between the Film Clip and the Counting condition in controls (top) and in people with RTLE (middle), and regions showing greater levels of activity for this contrast in the control than in the RTLE group (bottom) Figure 4.5 Contrast between the Narrative and the Counting condition in controls (top) and in people with RTLE (middle), and regions showing greater levels of activity for this contrast in the control than in the RTLE group (bottom) 112 Figure 4.6 Medial temporal voxels showing significantly greater activation in the controls than in patients with RTLE, per experimental condition..115 Figure 11.1 Mean tallied number of correct Story Details and Perceptual Details per memory condition shown for the first recording and for both recordings Figure 12.1 Mean number of verbs per successful trial (Story Content rating > 1)..220 Figure 12.2 Mean number of words per successful trial (Story Content rating > 1) from LIWC2007 s Affect category..221 Figure 12.3 Mean number of words from LIWC2007 s 1 st Person Singular category. Words are plotted per successful trial (Story Content rating > 1), and as a percentage of all the words in its text sample 222 xi

12 List of Appendices Appendix A Instructions Behavioural Task Appendix B Autobiographical Event Cues Appendix C List of Laboratory Events Narratives and Clips..192 Appendix D Scoring Manual for Complex Episodic Memories Appendix E Examples of Memory Recall per Group and Condition Appendix F Detail Count for the First versus Both Recordings..218 Appendix G Supplementary LIWC Results Behavioural Study Appendix H Instructions - fmri Task.223 xii

13 Chapter 1 General Introduction La mémoire, c est l imagination à l envers Daniel Pennac, La Fée Carabine (1987) Episodic memory has been shown to be supported by the medial temporal lobe (MTL), and much ongoing research is investigating which of its characteristics engage the MTL. Perceptual richness is among the defining features of episodic memory, and the purpose of the current work was to investigate its relationship with medial temporal function. More specifically, I tested the complementary hypotheses that 1) perceptual richness is one of the determinants of hippocampal engagement during episodic memory retrieval, and that 2) damage to the MTL reduces the perceptual richness of episodic memory. In the next chapters, I present results from behavioural and functional magnetic resonance imaging (fmri) work conducted in healthy controls, and in a clinical population with unilateral medial temporal lobe epilepsy or excisions (together referred to as mtle), that support both these hypotheses. The current chapter contains a summary of the literature linking different episodic memory features to MTL function, a discussion of perceptual richness as an important dimension of episodic memory and as a potential determinant of hippocampal engagement, a description of mtle and its impact on long term memory, and an overview of the objectives of the body of work presented in this thesis. 1.1 Episodic Memory and Determinants of Medial Temporal Engagement The severe amnesia that followed patient HM s bilateral temporal lobe resection drew attention to the core role played by the MTL, and more specifically by the hippocampus, in human declarative memory (Scoville & Milner, 1957). In the past 50+ years, a great deal of effort has been expended on refining our understanding of the types of memory processes and representations that engage the hippocampus during retrieval. Early reports indicated that HM 1

14 CHAPTER 1 GENERAL INTRODUCTION 2 and other medial temporal amnesics demonstrated severe anterograde amnesia (i.e., the inability to acquire new memories), severe retrograde amnesia for recent events, and preserved memory for remote time periods such as childhood years (Corkin, 2002; Scoville & Milner, 1957). Consolidation theory was formulated to address the presence of this retrograde temporal gradient that appeared to follow Ribot s law of regression (Ribot, 1887). According to consolidation theory, memory formation depends on the hippocampus. Over the timescale of years, memory gradually becomes consolidated, that is, integrated into a cortical representation which fully supports the memory trace independently of the MTL (McClelland, McNaughton, & O'Reilly, 1995; Squire, et al., 1992; Squire, et al., 2010; Squire & Wixted, 2011). However, inconsistencies in the literature regarding the timeline of the consolidation process, which appeared to range from hours to decades based on the effects of MTL lesions in humans and animals, led Nadel and Moscovitch to propose the Multiple Trace Theory (MTT) to provide a better account of the evidence (Moscovitch & Nadel, 1998; Nadel & Moscovitch, 1997). MTT draws on Tulving s (1972) conceptual distinction between memory that is retrieved with (episodic) and without (semantic) the experience of an autobiographical index, that is, information about the memory s context of acquisition. While semantic memory represents knowledge about oneself and the world that is accessed independently from the context in which the information was acquired (e.g., facts, ideas, concepts, rules, schemata, etc.), episodic memory corresponds to specific events from a person s life for which contextual information is part of the memory. Unlike consolidation theory, which claims that all declarative memory s hippocampal dependence is time-limited, MTT stipulates that episodic memory retrieval, recent and remote, always involves the hippocampus, while semantic memory can be retrieved independently from the MTL. Thus, the nature of memory content, rather than its recency, is what determines hippocampal involvement at retrieval. Some of the defining features of episodic memory which Tulving believed distinguished it from semantic memory include eventspecific learning, the retrieval of contextual details ( what, where and when details), and the subjective experience of reliving the past, also known as recollection (Tulving, 1972, 1984, 1985; Tulving & Markowitsch, 1998; Wheeler, Stuss, & Tulving, 1997).

15 CHAPTER 1 GENERAL INTRODUCTION 3 Since its formulation, much evidence has accrued in support of MTT. Clinical work has identified patients with MTL damage showing a deficit in both recent and remote episodic memory, with relatively preserved remote semantic memory (Gilboa, et al., 2006; Rosenbaum, McKinnon, Levine, & Moscovitch, 2004; Steinvorth, Levine, & Corkin, 2005; Vargha-Khadem, et al., 1997; Viskontas, McAndrews, & Moscovitch, 2000). This work indicates that the effect of hippocampal damage does not depend on the timeline of a memory, but on its nature episodic versus semantic. Also, neuroimaging work has shown that the hippocampus is engaged during the retrieval of both remote and recent episodic memories (Addis, Moscovitch, Crawley, & McAndrews, 2004; Cabeza & St Jacques, 2007; Gilboa, Winocur, Grady, Hevenor, & Moscovitch, 2004; Moscovitch, et al., 2005; Nadel & Hardt, 2011; Steinvorth, Corkin, & Halgren, 2006; Svoboda, McKinnon, & Levine, 2006; Viard, et al., 2007). While individuals with medial temporal amnesia with intact memory for remote personal events have also been reported (Squire & Wixted, 2011), inconsistencies in the literature may be task-dependent. Consistent with MTT, tasks that are sensitive to detailed, reexperiential memory content tend to reveal deficits regardless of the age of the memory, and it has been suggested that memories that are retrieved by the neocortex without involving the hippocampus lose their details and become more schematic or gist-like (Moscovitch, et al., 2005; Winocur & Moscovitch, 2011; Winocur, Moscovitch, & Bontempi, 2010). In recent years, research has gone beyond the episodic versus semantic dichotomy in order to pin-point which of episodic memory s properties are most important for determining hippocampal engagement at retrieval. Proponents of a representational view have called for a greater emphasis on actual memory representations and content, such as context-specific details, when attempting to characterize hippocampal function. They suggest, for example, that the retrieval of information about what, when and where a memory took place are better indicators of hippocampal engagement than memory recency (Hoscheidt, Nadel, Payne, & Ryan, 2010; Nadel & Hardt, 2011). Some scientists have also emphasized subjective dimensions such as the experience of recollection (Winocur & Moscovitch, 2011; Yonelinas, Aly, Wang, & Koen, 2010), while others have emphasized the importance of processes such as the rapid acquisition of complex and flexible representations (e.g., single-trial learning; Henke, 2010), and

16 CHAPTER 1 GENERAL INTRODUCTION 4 the reconstruction of complex and coherent spatial scenes (Hassabis & Maguire, 2007, 2009). Below, I review evidence regarding three different episodic memory characteristics that are in line with the original formulation of MTT (Nadel & Moscovitch, 1997): spatio-temporal specificity, the retrieval of context-specific details, and the experience of recollection Spatio-Temporal Specificity Episodic memory consists of the retrieval of memory content ( what ) in conjunction with information about its context ( where and when an event took place; Tulving, 1972, 1984, 2002). Temporal specificity ( when ) refers to the specification of a memory episode along one s life s timeline, while spatial specificity ( where ) refers to its specification in space. In MTT s original formulation, spatio-temporal specificity was proposed as a determinant of hippocampal engagement at retrieval (Moscovitch & Nadel, 1998; Nadel & Moscovitch, 1997). It was suggested that the hippocampal complex binds neocortical neurons into a coherent memory trace at encoding, and that it provides a spatial context for the episode. New hippocampal traces are formed each time the memory is reactivated, so that multiple traces can link the neocortical neurons that support episodic memory features. Greater numbers of co-existing traces contribute to the formation of associations between the neocortical portion of the trace, and semantic memory stored in the neocortex. Memory can eventually be retrieved independently from the hippocampus, but it will lack the spatio-temporal contextual information that depends on the involvement of the hippocampal complex. In the animal literature, single-trial learning, and knowledge of when and where an event took place, have been used successfully to model human episodic memory (episodic-like memory; e.g., Wiltgen et al., 2010; also see Henke, 2010, and Nadel & Hardt, 2011, for reviews). However, recent evidence suggests that the mentalizing of events with poor or unspecified spatio-temporal context, such as memory for repeated events (Addis, et al., 2004; St-Laurent, Moscovitch, Levine, & McAndrews, 2009), fairy tales (Rosenbaum, Gilboa, Levine, Winocur, & Moscovitch, 2009) or the imagining of never-experienced spatial scenes and scenarios (Addis & Schacter, 2008; Hassabis, Kumaran, Vann, & Maguire, 2007; Hassabis & Maguire, 2009), can depend on the hippocampus when such mental representations are sufficiently detailed and

17 CHAPTER 1 GENERAL INTRODUCTION 5 vivid. This evidence suggests that episodic memory features other than spatio-temporal specificity may be better predictors of hippocampal engagement at retrieval Context-Specific Details Context-specific details make up the content of a memory episode. They reflect when and where an event took place, what happened, as well as what was felt and perceived. Many contextual details can be conjured up when retrieving memory for specific personal episodes, but also when mentalizing events with poor spatio-temporal specificity, such as memory for repeated events (generic memories; St-Laurent et al., 2009), fairy tales (Rosenbaum, et al., 2009), or imagined new or future events, as those mental constructs can be quite detailed as well (e.g., Addis, Wong & Schacter, 2008; Hassabis et al., 2007). In fact, memory from patients with damage to the MTL has been described as skeletal or schematic: the memory s gist is preserved, but they are sparse in details (Moscovitch, et al., 2005; Rosenbaum, et al., 2009; St- Laurent, Moscovitch, Tau, & McAndrews, 2011; Winocur & Moscovitch, 2011) and have poor coherence (Hassabis, et al., 2007). At encoding, the hippocampus is known to bind memory details into complex associative memories (Eichenbaum, Otto, & Cohen, 1992; Konkel, Warren, Duff, Tranel, & Cohen, 2008; McCormick, Moscovitch, Protzner, Huber, & McAndrews, 2010; Staresina & Davachi, 2008, 2009), and at retrieval it supports the recombination of memory details into complex mental constructs (Hassabis & Maguire, 2009; Rosenbaum, et al., 2009; Ryan, Lin, Ketcham, & Nadel, 2010). However, more investigation is needed in order to determine whether the hippocampus main function is to retrieve individual memory details, or to integrate them into a coherent experience ( binding ), or both. While self-reported levels of naturalistic personal memory details have been shown to correlate with hippocampal activation at retrieval (Addis, et al., 2004; St Jacques, Rubin, & Cabeza, In Press), attempts to link hippocampal activation to increasing numbers of details or features manipulated experimentally have been inconclusive so far (Bird, Capponi, King, Doeller, & Burgess, 2010; Summerfield, Hassabis, & Maguire, 2010), suggesting that sheer number of details or complexity does not capture the entire story. Also, more work is needed in order to determine whether all detail types are equally supported by the hippocampus (Eichenbaum, et al., 1992; Konkel, et al., 2008), or whether some are more

18 CHAPTER 1 GENERAL INTRODUCTION 6 privileged than others (St-Laurent, et al., 2009; St-Laurent, Moscovitch, et al., 2011). For example, some have argued that hippocampal function is mainly spatial (Hoscheidt, et al., 2010; O'Keefe & Dostrovsky, 1971; O'Keefe & Nadel, 1978), and that the structure s role in memory retrieval is one of construction, the building of coherent scenes from spatial information (Hassabis & Maguire, 2009) that provide a framework into which other contextual details are integrated (Moscovitch & Nadel, 1998; Nadel & Moscovitch, 1997). However, others have demonstrated hippocampal involvement during the mentalizing of constructs with minimal spatial information (Rabin, Gilboa, Stuss, Mar, & Rosenbaum, 2010; Schacter & Addis, 2009). Thus, while the retrieval of contextual details is an important dimension of hippocampal function, more work is needed to clarify how factors such as the number of details, the nature of memory content (e.g., spatial or perceptual), and the intricacy and coherence of the associations among memory elements, modulate the relationship between memory details and hippocampal function Recollection Another memory dimension relevant to the characterization of hippocampal function is the experience of recollection, the subjective sense of reliving the past in one s mind s eye (Moscovitch, et al., 2005; Nadel & Moscovitch, 1997). Recollection is a core defining feature of episodic memory (Tulving, 1985, 2002; Wheeler, et al., 1997). A great deal of evidence indicates that memory recognition based on recollection, as opposed to familiarity, involves the hippocampus (Aggleton & Brown, 1999, 2006; Cohn, Moscovitch, Lahat, & McAndrews, 2009; Daselaar, Fleck, Dobbins, Madden, & Cabeza, 2006; Eichenbaum, Yonelinas, & Ranganath, 2007; Fortin, Agster, & Eichenbaum, 2002; Moscovitch & McAndrews, 2002; Ranganath, 2010; Vann, et al., 2009; Yonelinas, et al., 2010). Recollection is a complex phenomenon with several key characteristics, including 1) the experience of autonoetic consciousness, the self-awareness that one is travelling mentally in time (Tulving, 1984, 2002; Wheeler, et al., 1997), 2) a sense of pastness (to distinguish recollection from the imagining of the self in a never-experienced situation; Greenberg & Rubin, 2003; Wheeler et al., 1997), 3) perceptual imagery / the retrieval of episodic details that reflect sensory experiences (Brewer, 1995; Conway, 1995, 2009; Rubin, Schrauf, & Greenberg, 2003), and 4) the re-living of thoughts and feelings originally experienced

19 CHAPTER 1 GENERAL INTRODUCTION 7 at encoding (Conway & Loveday, 2010; Larsen, 1998; Moscovitch, et al., 2005). Both imagery and the retrieval of thoughts and feelings contribute to the vividness of the memory (Larsen, 1998; Moscovitch, et al., 2005), which correlates with one s subjective impression of reliving the past (Rubin, et al., 2003). Because they reflect specific aspects of the memory s content, imagery and thoughts/feelings can be considered specific categories of contextual memory details (see Levine, Svoboda, Hay, Winocur & Moscovitch, 2002) which, as discussed above, seem to engage the hippocampus. One of the key components of recollection, autonoetic consciousness, was defined by Tulving (1985) as the self-awareness associated with the mentalizing of a reality other than the immediate present (e.g., either past, future or imagined; Wheeler et al., 1997). Moscovitch (1995, 2008) has suggested that, along with episodic memory content, consciousness is part of the memory trace that is supported by the hippocampus. Buckner and Carroll (2007) also theorized that brain regions that compose the default mode network (Raichle, et al., 2001), which include the hippocampus, support self-projection, a concept akin to autonoetic consciousness in the sense that the self is mentally projected into the past, the future, or alternative realities (e.g., during spatial navigation, tasks requiring theory of mind, etc). However, Wheeler et al (1997) have argued that autonoetic consciousness is mainly supported by the prefrontal cortex, and the MTL may instead serve as an index of memory content stored in cortical regions that store elements forming the projection (Buckner, 2010; Greenberg, Eacott, Brechin, & Rubin, 2005; Teyler & Rudy, 2007). In fact, the evidence is inconsistent as to whether autonoetic consciousness is a defining feature of hippocampal representations. Addis and colleagues (2004) showed that an episodic memory s personal relevance, which could indirectly reflect self-projection, engages the hippocampus during retrieval. Also, tasks of autobiographical memory that measured the subjective sense of reliving personal events have shown impaired performance in individuals with damage to the MTL (Noulhiane, et al., 2008; Park, St-Laurent, McAndrews, & Moscovitch, 2011), although both studies acknowledge a relationship between reliving and the retrieval of memory details. In addition, some brain imaging studies have observed greater hippocampal activation, and greater connectivity with regions such as the medial prefrontal cortex, when people imagine situations relevant to

20 CHAPTER 1 GENERAL INTRODUCTION 8 themselves (memories, future events), in comparison to situations involving someone else (e.g., theory of mind tasks; Andrews-Hanna, Reidler, Sepulcre, Poulin, & Buckner, 2010; Cabeza, et al., 2004; Spreng & Grady, 2010; St Jacques, Conway, Lowder, & Cabeza, 2011; but see Szpunar, Watson, & McDermott, 2007). However, recent work has revealed that differences in hippocampal activation could be accounted for by perceptual content and vividness rather than self-relevance (Andrews-Hanna, Reidler, Sepulcre, Poulin, & Buckner, 2010; Rabin, et al., 2010). These results suggest that the retrieval of details may mediate the relationship between MTL engagement and the subjective impression of imagining or reliving events. In support for this claim, people with hippocampal amnesia have been shown to struggle to imagine detailed spatial scenes, and to narrate wellknown fairy tales and bible stories (Hassabis, et al., 2007; Rosenbaum, et al., 2009), two tasks that have limited self-relevance and require little self-projection. Finally, theory of mind, the capacity to experience another person s state of mind as one s own, involves self-projection, but is intact in amnesic patients with bilateral hippocampal damage (Rosenbaum, Stuss, Levine, & Tulving, 2007). Given this evidence, it is plausible that the hippocampus is not sensitive to self-relevance, self-projection or self-awareness per se, but is primarily sensitive to the content of the mental representation (Nadel & Hardt, 2011). In other words, the relationship between hippocampal activation and recollection, rather than being based on an elusive sense of selfawareness or self-projection in time, might be mediated by the retrieval of experiential memory details that provide the building blocks of a vivid recollective experience (Slotnick, 2010; but see Moscovitch 1995, 2008). Accordingly, the experience of perceptual imagery, another key component of recollection, might be a better predictor of hippocampal engagement at retrieval than autonoetic consciousness. Perceptual imagery represents the combination of impressions from different sensory modalities experienced in one s mind s eye. It is a core characteristic of recollective memory (Brewer, 1986, 1995), and its presence at retrieval contributes to one s subjective sense of reliving the past (Conway, 2009; Rubin, et al., 2003). The perceptual richness of episodic memory can be quantified by tallying up sensory-based memory details, which we refer to as perceptual details (Levine, et al., 2002; St-Laurent, et al., 2009). Perceptual details

21 CHAPTER 1 GENERAL INTRODUCTION 9 are a subcategory of context-specific details, and their retrieval supports recollection by contributing to a subjective impression of vividness (Larsen, 1998). In a sense, perceptual richness reflects two dimensions of episodic memory that seem to engage the MTL: contextspecific details, and the experience of recollection. For this reason, perceptual richness seems particularly well positioned to be an important determinant of hippocampal engagement at retrieval. Thus, this thesis assessed the relationship between the perceptual richness of episodic memory and medial temporal function. The next section focuses on perceptual imagery content in the context of episodic memory. I first discuss memory theories that consider perceptual richness to be an important dimension of the retrieval experience. I then provide empirical evidence supporting the hypothesis that perceptual richness is an important determinant of hippocampal engagement at retrieval. Finally, I discuss how perceptual richness was quantified and operationalized in the context of this thesis. 1.2 Perceptual Richness, Memory and the MTL Imagery in Theoretical Memory Models The perceptual richness of remembered events is based on the detailed combination of impressions from different sensory modalities retrieved from past experiences. While philosophers have long shown an interest in the experience of mental imagery during memory retrieval (e.g., Hume, Locke, Bergson, Russel, as reviewed in Brewer, 1995), imagery has gone in and out of fashion as a legitimate topic of rigorous scientific investigation in the field of modern psychology (see Brewer, 1995). In the early days of experimental psychology, Sir Francis Galton (1880; Brewer, 1995) documented individual differences in mental imagery for memory of morning breakfast scenes, and Wundt s Leipzig school of introspective psychology specialized in the scientific assessment of memory s imagery content and its properties (e.g., intensity, perspective, etc.; see Larsen, 1998 for a review). In his book Remembering, Bartlett (1932) theorized about the role of mental imagery in memory, a topic which, according to him, hardly appear[ed] to be a popular view in psychology (p. 215). For decades, the behaviorist and

22 CHAPTER 1 GENERAL INTRODUCTION 10 cognitive eras brought distrust for introspection, a general lack of interest in imagery, and a shift of focus toward outward, objective measures of memory performance. It is only within the last four decades that contemporary experimental psychology experienced a revival of interest in mental life and a renewed belief that the subjective experience that accompanies memory retrieval can be studied scientifically. Pioneers of this change included Tulving (1972, 1984; Wheeler, et al., 1997), who criticized the lack of interest in studying the mental experience of remembering, and whose approach to the study of episodic memory emphasized its experiential aspects. Tulving (1985) introduced a new technique to assess subjective mental life in a controlled experimental setting, the Remember/Know paradigm. In a typical memory recognition experiment using this paradigm, participants are required to specify whether they recognize previously studied stimuli based on a sense of familiarity with the items ( know ), or whether they recollect studying the items ( remember ). Following Tulving s lead, Brewer declared that a complete account of the human mind must include an account of the subject's phenomenal experience" (Brewer, 1986, p.29). He introduced a classification system for autobiographical memory, that is, memory that relates to the self, based on two orthogonal features: 1) temporal specificity, which distinguishes between memory for single episodes and memory for repeated or generic episodes and personal facts, and 2) the presence or absence of imagery. Brewer made an important distinction between memory for single personal episodes that is accompanied by recollection (personal memory, or recollective memory in his later work, e.g., Brewer 1995), and memory for single episodes that are retrieved without imagery (autobiographical facts); the latter are known (in Tulving s sense) without being re-experienced. In a revised, more nuanced version of this classification system, Brewer (1995) specified that recollective memories for single personal events are comprised of both imaginal (imagery-based) and non-imaginal (e.g., thoughts and emotions) information. Martin Conway s hierarchical model of autobiographical knowledge (Conway, 1995, 2009; Conway & Loveday, 2010; Conway & Pleydell-Pearce, 2000) also recognizes imagery as an important memory dimension. This model is very explicit about which form of personal knowledge is associated with imagery ( experience-near ), and which form is represented

23 CHAPTER 1 GENERAL INTRODUCTION 11 conceptually. Conway s model (Conway, 1995; Conway & Pleydell-Pearce, 2000) is composed of several nested levels of self-related knowledge that differ in temporal specificity (see Figure 1.1). Life-period knowledge (e.g., when the kids were young, when I worked for Microsoft ) is the least specific level, followed by general events knowledge. General events knowledge includes memory for single (e.g., my 30 th birthday party), repeated (e.g., Wednesday lab meetings), and extended events (e.g., my last trip to Paris), as well as memory for collections of events linked by themes and goals (e.g., how I met your mother; how I got into grad school at U of T ). Both life-period and general events knowledge are conceptual rather than imagerybased. The next level in the model is episodic memory (complex and simple), which represents specific events (e.g., my last coffee break; calling my brother on his birthday), and it is composed of episodic elements (EEs; also known as event-specific knowledge in earlier versions of this model, e.g., Conway, 1995; Conway and Pleydell-Pearce, 2000). EEs are the most temporally precise category, and they represent specific sensory information, images, emotions and highly specific facts that are the building blocks of episodic memories. To form episodic memories, several EEs are bundled together within a conceptual frame of contextualizing knowledge, which provides meaning to the memory (e.g., meeting my future wife ). Thus, episodic memory is composed of both conceptual and experiential knowledge. Because EEs are experience-near representations, they are needed in order to experience recollection (Conway, 2009; Conway & Loveday, 2010). Conway suggests that frames are supported by a fronto-temporal brain network, while EEs are supported by a temporo-occipital-parietal network (Conway, 2009; Conway & Loveday, 2010). He claims that damage to either the temporal lobe or to posterior cortical regions should lead to a dramatic loss of EE, as observed in people with amnesia (Conway & Loveday, 2010; Greenberg, et al., 2005; Rubin & Greenberg, 1998). Conway s position that the retrieval of EEs depends on the integrity of the temporal lobe is consistent with the main hypothesis of this thesis that the perceptual richness of episodic memory, which implies the retrieval of EEs, determines hippocampal engagement during retrieval. In the next section, I discuss empirical evidence that support this hypothesis.

24 CHAPTER 1 GENERAL INTRODUCTION 12 Figure 1.1 Conway s model of the organization of autobiographical memory Perceptual Memory Richness and Hippocampal Engagement at Retrieval Some studies indicate that the retrieval of imagery-based episodic details depends on the integrity and involvement of the hippocampus. For example, the perceptual features of memory for personal life events (also known as autobiographical memories) have been shown to be especially vulnerable to damage of the MTL (St-Laurent, et al., 2009). Brain imaging studies of autobiographical memory retrieval have also shown that hippocampal activation is correlated with ratings of vividness (Gilboa, et al., 2004; Rabin, et al., 2010), use of imagery (Andrews-Hanna, et al., 2010) and sense of reliving (St Jacques, Kragel & Rubin, In Press; but see Daselaar et al., 2008). In addition, a greater sense of reliving, and greater levels of hippocampal activation, are achieved when memory is cued with photographs of the original event, rather than with a memory title (e.g., going to the beach last weekend ), because snapshots facilitate access the memory s perceptual representation (St Jacques, Conway, Lowder, & Cabeza, 2011, April). By the same token, Vargha-Kadhem s patient Jon, who suffers from perinatal damage restricted to the hippocampus and has thus grown up with extensive bilateral hippocampal

25 CHAPTER 1 GENERAL INTRODUCTION 13 lesions, describes himself as the complete opposite of a visual person. His self-described experience of imagining scenes is as follow: "I find it difficult to visualize things in my mind's eye. When I do try, I can do it. It doesn't come automatically, though. I know it probably does with most people. It's not something I used to be able to do, but I've worked on it a lot over the years" (Maguire, Vargha-Khadem, & Hassabis, 2010). Greenberg and Rubin have argued that the storage of visual memory elements takes place in posterior cortical regions, while the hippocampus integrates such representational elements into a coherent memory construct at retrieval. Consistent with their neuropsychological model of autobiographical memory representation, they report that damage to either the MTL or to posterior visual areas can lead to a severe memory deficit (Greenberg, et al., 2005; Greenberg & Rubin, 2003; Rubin & Greenberg, 1998). Interestingly, while MTL damage leads to severe anterograde amnesia, damage to posterior cortical regions is linked to a memory deficit that is mostly retrograde, as one would expect if the hippocampus serves as an index of knowledge stored in the neocortex (Teyler & DiScenna, 1986; Teyler & Rudy, 2007). In fact, patterns of hippocampal connectivity provide an anatomical template for the hippocampus to play a central role in the integration of perceptual details into rich memory constructs. Input from the parahippocampal gyrus, which supports spatial and scene representation, reaches the hippocampus through the medial entorhinal cortex (perforant pathway), and converges into the cornu ammonis (CA) subfields with input from the apex of the ventral visual stream which enters the hippocampus through the perirhinal and lateral entorhinal cortex (Derdikman & Moser, 2010; Eichenbaum & Lipton, 2008; Litman, Awipi, & Davachi, 2009; Ranganath, 2010; Suzuki, 2010). A convergence of indirect input from olfactory and polysensory cortical regions, as well as from temporal cortical regions (Amaral & Lavenex, 2007; Insausti, Amaral, & Cowan, 1987) known to play a prominent role in semantic memory (Lambon Ralph, Cipolotti, Manes, & Patterson, 2010; Patterson, Nestor, & Rogers, 2007), are also funneled through the entorhinal cortex. Together with direct projections from the amygdala (Amaral & Lavenex, 2007), which plays a role in emotion (Buchanan, 2007), these inputs contribute to add richness and complexity to memory for past events.

26 CHAPTER 1 GENERAL INTRODUCTION 14 Evidence that the hippocampus supports scene construction, the assemblage and retrieval of spatial details into complex scenes (Hassabis, et al., 2007; Hassabis & Maguire, 2007, 2009), also provides support for a link between hippocampal function and the perceptual richness of memory. While not all perceptual memory details are spatial (e.g., Huijbers, Pennartz, Rubin, & Daselaar, 2011), and while the hippocampus supports memory for nonspatial information (e.g., Manns, Howard, & Eichenbaum, 2007; Wood, Dudchenko, & Eichenbaum, 1999), some forms of spatial representation are clearly perceptual. For example, scene construction involves imagery, and the binding of disparate multimodal elements such as sounds and smells in addition to visual inputs, people, objects, entities and their actions" (Hassabis & Maguire, 2009). Also, brain imaging work shows that the hippocampus is more activated when either episodic or semantic memory contains spatial information (Bird, et al., 2010; Hoscheidt, et al., 2010; Ryan, et al., 2010), some of which may be represented perceptually. In general, the line between perceptual and spatial representations is blurry. In the body of work presented here, all memory elements (spatial or not) that were linked to a sensory modality were categorized as perceptual memory content, without any further distinction. The cumulative evidence discussed in this section is consistent with the hypothesis that the experience of multi-modal imagery during episodic memory retrieval is supported by the hippocampus. To test this hypothesis, I designed a task that allowed me to measure episodic memory s perceptual richness and to identify its neural correlates. Then, I compared performance on this task between individuals with damage to the MTL, and healthy controls. The next section addresses how the perceptual richness of episodic memory was manipulated and quantified Assessing Perceptual Richness While the literature suggests that memory with high perceptual content necessitates the hippocampus at retrieval, much of the evidence available is correlational, as perceptual richness is not usually manipulated in tasks that assess personal life events. Also, in naturalistic memories for autobiographical episodes, perceptual richness often co-varies with other

27 CHAPTER 1 GENERAL INTRODUCTION 15 memory attributes such as personal relevance, rehearsal or emotionality (Daselaar, et al., 2008; Levine, et al., 2002; Rubin, et al., 2003). Designing a task during which participants acquired episodic memories in the laboratory allowed me to manipulate perceptual richness, while controlling for other confounding memory characteristics such as emotionality and personal relevance. I also wanted to insure that the laboratory events captured key characteristics of personal memory episodes, such as their complexity. The literature indicates that standard laboratory tasks of episodic memory retrieval, and tasks of autobiographical memory, can activate very different sets of neural regions (Gilboa, 2004; McDermott, Szpunar, & Christ, 2009; Wheeler, et al., 1997), but that these differences can be reduced dramatically when laboratory and naturalistic memories are cued and tested in similar ways (Burianova & Grady, 2007; St-Laurent, Abdi, Burianova, & Grady, 2011; Stokes, Mazuz, Daselaar, Moscovitch, & Cabeza, 2011, April). With my task, participants studied short laboratory stories that captured the complexity and causal structure of autobiographical episodes (Radvansky, Copeland, & Zwaan, 2005). Stories were presented under one of two different formats: either as perceptually enriched audio-visual film clips with visual features in the background and noise or music, or as perceptually impoverished written narratives that described interactions among story characters. In order to compare performance between the laboratory task and a more naturalistic task, a condition during which participants retrieved autobiographical memories of their own selection was also included. At retrieval, laboratory and autobiographical memories were cued with a title followed by free recall. This task was used first in a behavioural experiment, and was then adapted to functional magnetic resonance imaging (fmri) in order to assess the neural correlates of perceptual richness. Behaviourally, the perceptual richness of my participants memory was quantified in two different ways, objectively and subjectively. The objective measure was a tally of Perceptual memory details, memory elements that were associated with a sensory modality. I considered Perceptual details a direct measure of perceptual memory content. Perceptual details fit under what Larsen (1998) calls Perceptual Qualities, according to his classification of the phenomenal qualities of remembering. Perceptual Qualities correspond to external sensory information that is part of the memory content; vision is the dominant modality through which perceptual

28 CHAPTER 1 GENERAL INTRODUCTION 16 memory qualities are experienced. Examples of perceptual qualities include impressions of perspective, colour and movement, visuo-spatial, auditory, olfactory, gustatory, tactile, kinaesthetic, somatic or visceral impressions, as well as impressions of order and duration, day or night, and weather (Larsen, 1998, p.172). The second measure of perceptual richness was a subjective rating of vividness performed on a Likert scale by all participants. This measure reflected their overall impression of the perceptual richness of their memory. Larsen considers vividness to be a Surface Quality, because it reflects a subjective impression of the remembering experience, rather than being a direct indicator of memory content. He considers vividness an indirect and subjective measure of a memory s Perceptual Qualities, namely, its perceptual memory content. Other examples of Surface Qualities include impressions of clarity, coherence, completeness, and stability of image (Larsen, 1998). As mentioned, both healthy individuals and individuals with damage to the MTL were tested on this paradigm. My goal was to assess the impact of MTL damage on the perceptual richness of episodic memory, measured behaviourally, and on its neural correlates, measured with fmri. A group of individuals who either suffered from seizures from unilateral hippocampal origin, or had received a temporal lobe resection in order to eliminate the neural substrate giving rise to hippocampal seizures, participated in these experiments. In the next section, I discuss their condition and its impact on the integrity of brain tissue and on cognition. 1.3 Medial Temporal Lobe Epilepsy Etiology, Characterization and Surgical Management of mtle Patients with medial temporal lobe epilepsy (mtle) experience seizures generated spontaneously by a network of cells located in the hippocampus. In cases of unilateral mtle, seizures are consistently generated within the same hemisphere. MTLE is the most prevalent form of drug resistant epilepsy. In most cases, it is associated with the presence of mesial temporal sclerosis (MTS), but it can also be linked to lesional pathologies such as cortical dysplasia, tumours or vascular malformations, or it can occur in the absence of physical damage

29 CHAPTER 1 GENERAL INTRODUCTION 17 observable with MRI technology (Walker, Chan, & Tom, 2007). Cases of mtle with MTS are typically associated with the occurrence of an initial insult, such as childhood febrile seizures or neonatal hypoxia. The risk of developing seizures after a period of latency is worse when combined with an additional insult or second hit, such as a genetic predisposition, hippocampal maldevelopment or the occurrence of head trauma (see Mathern et al., 1996; Walker et al., 2007, for reviews). MTLE is associated with MTS in roughly half the cases (Bruton, 1988; Walker et al., 1997), although the causal link between MTS and mtle is unclear. In the case of a childhood insult, MTS can sometimes precede the occurrence of complex-partial seizures, although seizure frequency and a long history of seizures can also contribute to neuronal loss (Bernasconi, Natsume, & Bernasconi, 2005; Kalviainen, et al., 1998; Mathern, et al., 1996; Walker, et al., 2007). MTS is associated with hippocampal volume loss due to reduced cell count, mostly among principal pyramidal cells in CA1 and the hilar region, and in the presubiculum. Dentate granule cells and cells in CA3 and in the nearby cortex are more resilient, but severe MTS will affect all hippocampal subfields (Luby, Spencer, Kim, delanerolle, & McCarthy, 1995; Mathern, et al., 1996; Walker, et al., 2007). MTS is also associated with the proliferation of astrocytic glial cells in areas of cell loss. Additional changes include sprouting of mossy fibers into the CA fields, a potential mechanism for increased excitability within hippocampal tissue (see Mathern et al., 1996; Walker et al, 1997, for a review). Both atrophy and sclerosis can be measured with MRI technology using techniques such as voxel-based morphometry (Keller & Roberts, 2008; Li, Zhang, & Shang, 2012), volumetrics (Luby, et al., 1995; Pardoe, Pell, Abbott, & Jackson, 2009), or T2-relaxometry (Jackson, Connelly, Duncan, Grunewald, & Gadian, 1993; Pell, Briellmann, Pardoe, Abbott, & Jackson, 2008; Wendel, et al., 2001). In cases of drug-resistant mtle, a temporal lobe resection can be performed to remove the epileptogenic focus. A traditional temporal lobe resection removes the anterior hippocampus, amygdala, temporal pole, parahippocampal cortex (perirhinal, entorhinal, and some parahippocampal gyrus), and some lateral temporal cortex (Walker et al., 1997). More selective forms of resection that are limited to MTL structures, such as the amygdalo-

30 CHAPTER 1 GENERAL INTRODUCTION 18 hippocampectomy, are also performed as an alternative to more extensive resections (Schramm, 2008). A temporal lobe resection has been shown to lead to seizure-freedom in more than 60% of cases (Engel, et al., 2003; Wiebe, 2004; Wiebe, Blume, Girvin, & Eliasziw, 2001), but this number can be as high as 75-80% when MTS is present, and EEG and neuropsychological evidence are concordant (Arruda, et al., 1996; Bonilha, Martz, Glazier, & Edwards, 2012). In general, MTS and hippocampal atrophy are good predictors of successful surgical outcome, and resections are most efficient at reducing seizures when they include the hippocampus proper (Luby, et al., 1995; Walker, et al., 2007) Impact of mtle on Memory In individuals with mtle, the most typical cognitive complaint, and the deficit most consistently observed with neuropsychological assessment, is a deficit in episodic memory. This deficit is material-specific, and it is affected by whether seizure onset is localized in the left (typically language-dominant) or the right (non-language dominant) hemisphere (Jones- Gotman, et al., 2010; McAndrews & Cohn, In Press; Milner, 1970). Evidence for material-specific lateralization is consistent with a larger literature showing that MTL lesions to the lefthemisphere lead to an impairment in memory for verbal material (Djordjevic, et al., 2010; Frisk & Milner, 1990; Helmstaedter, Grunwald, Lehnertz, Gleissner, & Elger, 1997; Rausch & Babb, 1993; Sass, et al., 1995), while right-hemisphere lesions affect memory for non-verbal/spatial material (Bohbot, et al., 1998; Jones-Gotman & Milner, 1978; Morris, Pickering, Abrahams, & Feigenbaum, 1996; Smith & Milner, 1981, 1989; Spiers, Maguire, & Burgess, 2001). With respect to laterality, indistinguishable levels of impairment are typically observed between individuals with left and right mtle on tasks that require re-experiencing autobiographical episodes (Herfurth, Kasper, Schwarz, Stefan, & Pauli, 2010; Lah, Grayson, Lee, & Miller, 2004; Lah, Lee, Grayson, & Miller, 2006; Noulhiane, et al., 2007, 2008; St-Laurent, et al., 2009; St- Laurent, Moscovitch, et al., 2011; Viskontas, et al., 2000; but see Voltzenlogel, et al., 2006 for a greater deficit in patients with left-lateralized mtle, and see McAndrews, In Press, for a review). Such findings suggest that both hemispheres contribute importantly to memory for personal life events. The multi-modal and complex nature of memory for personal life events may explain why bilateral deficits are observed, although testing procedures with greater

31 CHAPTER 1 GENERAL INTRODUCTION 19 sensitivity to material-specificity could uncover subtle patterns of hemispheric specialization (e.g., see Buchanan, Tranel, & Adolphs, 2006, for evidence of an emotional bias in AM fluency in individuals with right-lateralized mtle). Some evidence also suggests that pre and post-surgery mtle patients may not differ significantly from one another on autobiographical memory tasks (Herfurth, et al., 2010; St- Laurent, et al., 2009; St-Laurent, Moscovitch, et al., 2011; Viskontas, et al., 2000), even those based heavily on narration. Although the left temporal pole and lateral temporal cortex play a role in the expressive use of language (Burnstine, et al., 1990; Friederici, 2011; Lambon Ralph, et al., 2010; Seidenberg, Geary, & Hermann, 2005), it should be kept in mind that the AM tasks are not assessing verbal expression per se, but the content of episodic memory representations, which are multi-modal. Indeed, people with TLE prove not to be impaired at generating detailed scripts for familiar activities, such as washing the dishes or eating at a restaurant (St-Laurent, et al., 2009), and their performance on AM tasks is unrelated to their performance on tasks of verbal fluency (Addis, Moscovitch, & McAndrews, 2007), indicating that patients poor performance on AM tasks is due to a memory rather than a language deficit. Previous behavioural evidence from our group has revealed a dramatic loss of perceptual content in autobiographical memory for both unique and repeated personal episodes in individuals with mtle (St-Laurent, et al., 2009). With the current project, I followed up on these results by testing individuals with mtle on a paradigm that allowed me to manipulate perceptual richness experimentally. During this task, participants recalled complex laboratory episodes that were either perceptually enriched or impoverished. As a first step, individuals with unilateral mtle (left or right) were tested behaviourally pre or post-surgery, and their performance was compared with healthy individuals. Based on previous studies, I expected similar performance within the mtle group regardless of surgical status or laterality. Then, I tested pre-surgery candidates with mtle and healthy controls on a version of the behavioural task adapted for fmri, in order to assess how focal MTL damage affects taskrelated activation among neural regions sensitive to perceptual richness. All participants with mtle who underwent fmri had right-lateralized pathology due to recruitment constraints.

32 CHAPTER 1 GENERAL INTRODUCTION Extra-temporal brain damage in mtle It is important to note that other forms of brain damage can sometimes be observed in mtle beside MTS. Evidence from structural brain imaging has shown extra-hippocampal atrophy in people with mtle, within (e.g., Moran, Lemieux, Kitchen, Fish, & Shorvon, 2001) and outside (e.g., Bernhardt, et al., 2009; Keller & Roberts, 2008) the temporal lobe, although the greatest and most consistent atrophy was, by far, found within the epileptogenic hippocampus proper (Keller & Roberts, 2008; Mathern, et al., 1996; Moran, et al., 2001). Recently, evidence from MR diffusion imaging has also revealed wide-spread white matter anomalies in individuals with mtle, as indicated by reduced fractional anisotropy, especially on the ipsilateral side (Focke, et al., 2008; Gross, 2011; Gross, Concha, & Beaulieu, 2006). In a recent review, Bell and colleagues (2011) discussed how extra-temporal brain damage can lead to a more subtle pattern of generalized cognitive dysfunction than the pattern of memory impairment typically linked to the presence of MTS (e.g., Baxendale, Thompson, & Van Paesschen, 1998; Baxendale, van Paesschen, et al., 1998; Mueller, et al., 2011; Wendel, et al., 2001). The authors also discussed factors that predict the presence of extra-temporal damage, such as lowered IQ, poor performance on neuropsychological tests beside memory tests (e.g., language, executive function), a switch of language dominance, and the occurrence of seizures following a temporal lobe resection. With these issues in mind, I tried to minimize the presence of extra-temporal damage in the current research participants through careful screening. Data were collected from individuals with unilateral mtle from hippocampal origin, or from individuals who had received either a classic temporal lobectomy or a more selective resection (amygdalohippocampectomy), in order to eliminate seizures from unilateral hippocampal origin. Patients with extra-temporal damage visible on high resolution and T2-weighted MRI scans, as identified by a neuroradiologist, were excluded. Also, all participants with mtle had IQ scores that fell within the normal range, that is, no lower than 85. Although a few individuals included in the current sample had memory in the normal range, the majority of these mtle participants neuropsychological profile reflected a disproportionate memory deficit in relation to other cognitive domains, providing confirmation that poor performance on the current task would be

33 CHAPTER 1 GENERAL INTRODUCTION 21 linked to disrupted MTL function, rather than to wide-spread, unspecific damage. Partial transfer of language dominance to the right hemisphere can indicate rewiring or neural plasticity in mtle (Bell, et al., 2011). Only two of the current mtle participants had language lateralized to the right rather than the left hemisphere, both of whom were tested behaviourally, limiting the presence of altered cerebral organization in my sample. Finally, it has been demonstrated that patients who show more restricted damage in their pre-operative MRI have a better probability of being seizure-free post-operatively (Keller & Roberts, 2008), and all of the current post-surgery participants were entirely seizure-free post-operatively at the time of testing. While mtle can affect brain function and tissue integrity outside the MTL, the selective patient screening implemented for this thesis insured that the obtained results could provide insightful knowledge about the impact of damage to the MTL on complex episodic memory. For this thesis, individuals with mtle were tested on tasks of episodic memory in order to determine whether MTL engagement is essential to the retrieval of perceptual memory content. The detailed objectives of the different studies that are presented in this thesis are outlined below. 1.4 Objectives of the Present Research The main goal of the present research was to assess the relationship between hippocampal function, and the perceptual richness of complex episodic memory. I tested the hypothesis that the hippocampus is a hub within a network of brain regions that support the retrieval and integration of perceptual episodic memory details, and that damage to this hub leads to behavioural and neural changes that reflect a loss of perceptual richness. I designed a paradigm that allowed me to dissociate perceptual richness from other components of episodic memory, by comparing memory for perceptually enriched and for perceptually impoverished complex laboratory events (stories) matched for story content. Each laboratory story was randomly shown as a perceptually enriched audio-visual film clip, or as a perceptually impoverished voiced-over narrative that described the action taking place in the clip. An autobiographical memory condition was also included for comparison. I compared performance on these tasks between healthy controls, and individuals who either had medial temporal lobe

34 CHAPTER 1 GENERAL INTRODUCTION 22 epilepsy, or who had received a unilateral medial temporal lobe excision to eliminate seizures from hippocampal origin. Together, both patient groups are referred to as mtle. I also tested a different cohort of controls and of pre-surgery patients with right-lateralized mtle on a version of this paradigm adapted for functional magnetic resonance imaging (fmri). These experiments were designed to test the following hypotheses: 1) The hippocampus is necessary for the retrieval of perceptually rich memory information. It follows that damage to the hippocampus will reduce the perceptual richness of complex episodic memory. Hence, participants with mtle will demonstrate a paucity of perceptual details in their recall for all three conditions in comparison to healthy controls on my behavioural task. 2) The hippocampus is a hub in a network of brain regions that support the retrieval of perceptually rich memory episodes. It follows that: (i) The neural correlates of perceptually enriched episodic memory retrieval will include the hippocampus, as well as regions known to be involved in perceptual processes and in vivid recollection, such as the precuneus, the parahippocampal gyrus and the retrosplenial cortex. In healthy controls, the retrieval of perceptually enriched autobiographical memories and film clips should reveal greater activation in the hippocampus, and in other regions sensitive to perceptual memory content, than the retrieval of perceptually impoverished narratives. Also, hippocampal activation should be modulated parametrically as a function of the perceptual richness of the memory, measured objectively with free recall. (ii) Damage to the hippocampus in individuals with mtle will reduce activation among brain regions sensitive to perceptual richness, including but not limited to the hippocampus. This decrease will be most pronounced during the retrieval of memories in the AM and Film Clip conditions. Data supporting these hypotheses are presented in the next chapters. In Chapter 2, I present evidence of a paucity of perceptual memory details for both personal and laboratory events in individuals with mtle. In Chapter 3, I discuss results from an fmri study that identified brain regions whose activity is modulated by perceptual memory richness, which include the

35 CHAPTER 1 GENERAL INTRODUCTION 23 hippocampus proper. Finally, in Chapter 4, I present data indicating that right-lateralized mtle (R-mTLE) leads to a reduction in activation among regions sensitive to perceptual memory content during episodic memory retrieval.

36 Chapter 2 The Impact of Temporal Lobe Epilepsy on the Perceptual Richness of Complex Episodic Memories 2.1 Introduction The purpose of this behavioural study was to test the causal link between the integrity of the medial temporal lobe (MTL) and the perceptual richness of complex episodic memories. I recruited participants who suffered from seizures of unilateral hippocampal origin (left or right), as well as participants who had received a unilateral resection of temporal lobe structures, including at least the anterior half of the hippocampus, in order to control seizures from hippocampal origin. Throughout this thesis, I refer to both these conditions as medial temporal lobe epilepsy or mtle. MTLE is known to compromise the integrity of the MTL and to disrupt autobiographical memory (AM; Addis, Moscovitch, & McAndrews, 2007; Noulhiane, et al., 2008; St-Laurent, Moscovitch, Tau, & McAndrews, 2011; Viskontas, McAndrews, & Moscovitch, 2000). Previous findings from our group showed that people with mtle demonstrate a disproportionate paucity of perceptual AM details, suggesting that the perceptual richness of episodic memory is especially sensitive to MTL insult (St-Laurent, Moscovitch, Levine, & McAndrews, 2009). In the current study, I compared the performance of individuals with mtle to the performance of healthy controls on a new paradigm that manipulated the perceptual richness of memory experimentally. My goals were 1) to replicate previous findings indicating that the perceptual richness of AM is disrupted in people with mtle, and 2) to expand these findings to memories of perceptually rich or impoverished laboratory events. Showing that memories of autobiographical and laboratory events are similarly affected by damage to the MTL would lend credence to the idea that common psychological processes and neural mechanisms are implicated in both. In naturalistic personal memories, establishing a clear relationship between perceptual richness and MTL engagement can be difficult, because perceptual content is often correlated with other kinds of memory features, such as emotionality or personal relevance (Daselaar, et 24

37 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 25 al., 2008; Levine, Svoboda, Hay, Winocur, & Moscovitch, 2002; Rubin, Schrauf, & Greenberg, 2003). In order to achieve greater experimental control, I designed a task during which participants acquired episodic memories in the laboratory, which allowed me to manipulate perceptual richness independently from overall story content ( What happened? ), and to control for other memory characteristics of no interest to the main hypothesis, such as emotionality. Participants studied short laboratory events with an unfolding story line presented in one of two formats: as perceptually enriched audio-visual film clips played on a computer screen, or as perceptually impoverished written narratives explaining in words the story content, namely, what happened in the event. Written narratives were shown on screen one sentence at a time, and were accompanied by a voice-over. This paradigm also included an AM condition during which participants retrieved personal life episodes pre-selected with the experimenter. The purpose of this condition was to compare the magnitude of the deficit on the AM condition to performance on the two laboratory conditions in people with mtle. Short laboratory stories were chosen over more traditional laboratory stimuli such as single words or pictures because they captured some of the complexity of AM not usually embodied in standard laboratory episodic memory tasks (Ben-Yakov & Dudai, 2011; Furman, Dorfman, Hasson, Davachi, & Dudai, 2007; Hasson, Furman, Clark, Dudai, & Davachi, 2008; Rubin, 2005; Wheeler, Stuss, & Tulving, 1997). Evidence from functional brain imaging reveals that most laboratory tasks of episodic memory share minimal neural correlates with AM retrieval (Gilboa, 2004; McDermott, Szpunar, & Christ, 2009). Some of these differences appear to stem from systematic methodological differences, as AM tasks typically require recalling complex events, while laboratory tasks of episodic memory typically test the recognition of simple stimuli. In fact, discrepancies between the two types of memory can be mitigated when both conditions are cued and tested in similar ways. For example, significant overlap in patterns of brain activation has been observed between AM and laboratory tasks when both forms of memory are tested using multiple-choice questions (Burianova & Grady, 2007; St-Laurent, Abdi, Burianova, & Grady, 2011; Stokes, Mazuz, Daselaar, Moscovitch, & Cabeza, 2011, April) using a recognition paradigm with photograph cues (Cabeza, et al., 2004), or when the complexity and time required to retrieve laboratory events are increased (Stokes, et al., 2011, April).

38 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 26 During my task, both AMs and laboratory events were tested using a cued recall paradigm during which memories were retrieved over several seconds. The laboratory events were designed to resemble autobiographical episodes: they had a narrative structure that unraveled over time, with one or more characters interacting in a given situation, performing sequences of actions which could be re-told and rehearsed (Radvansky, Copeland, & Zwaan, 2005). Unlike AM though, the laboratory events had no personal relevance to the participants, which provided the advantage of controlling for important confounds such as centrality, emotionality, and rehearsal, among others. Retention intervals were on a different time scale for AMs than for the laboratory episodes, which were between 20 and 100 minutes old, rather than going back weeks, months or years. However, retention intervals were equated between the two laboratory memory conditions, ruling out that this factor mediated differences in performance between the Narrative and the Clip condition. AMs were selected with the participant at the very beginning of the experiment, before encoding the clips and narratives. At retrieval, participants were cued with a memory title that belonged to an AM, clip or narrative, and they were given 16 seconds to retrieve the memory in their mind. Then, they underwent a two-step probing procedure during which they described, in their own words, the content of their retrieval experience. First, they described the story elements they had time to retrieve ( what happened : who did what, what was the situation, etc.). Then, they listed the perceptual features they re-experienced in their mind (what they saw in their mind s eye ). During the second probing, participants were instructed to describe memory details experienced through all senses, including visual, auditory, olfactory, gustatory, tactile and proprioceptive details. Transcripts of the participants free recall were then coded for their content. Story and perceptual elements were identified and tallied up by a scorer according to a scoring procedure adapted from Levine and colleagues (Levine, et al., 2002). Then, the number of details from each category, story and perceptual, was averaged per memory condition and was compared between the groups. Additionally, transcripts were fed to the Linguistic Inquiry and Word Count computer software, or LIWC, to perform an automated count for words falling under 80 different categories defined by an integrated dictionary (LIWC2007; Pennebaker, Chung, Ireland, Gonzales, & Booth, 2007). Examples of LIWC

39 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 27 categories include positive emotions, visual processes, etc. Word counts from categories that reflected perceptual memory content were included in the current analysis. It is important to note that participants were instructed to recall whichever sensory elements they experienced in their mind at retrieval, regardless of whether such details were perceived originally at encoding. This is especially important for the Narrative condition, for which, unlike in the Clip and AM conditions, participants were told rather than shown what happened. Thus, if participants visualized elements from a narrative at retrieval which they were not shown at encoding (e.g., a person s shape or face, his or her voice, elements from the scenery, etc.), they were instructed to report these perceptual details just as for those they were shown in the Clip condition. Growing evidence indicates that damage to the MTL disrupts non-recall tasks that require imagining new scenes and visualizing future or alternative events (Addis, Wong, & Schacter, 2008; Hassabis, Kumaran, Vann, & Maguire, 2007; Rosenbaum, Gilboa, Levine, Winocur, & Moscovitch, 2009). Thus, I considered it likely that participants with mtle would be impaired at generating these contextual images when retrieving narratives in comparison to controls, and my aim was to quantify the perceptual qualities of the participant s retrieval experience, whether it was based on true retrieval or on imagination. As mentioned before, I tested individuals with unilateral mtle located in either the right or the left hemisphere. Past evidence of an AM deficit in mtle has failed consistently to reveal differences in performance based on laterality, suggesting that both temporal lobes contribute importantly to AM retrieval (Herfurth, Kasper, Schwarz, Stefan, & Pauli, 2010; Lah, Grayson, Lee, & Miller, 2004; Lah, Lee, Grayson, & Miller, 2006; Noulhiane, et al., 2007, 2008; St-Laurent, et al., 2009; St-Laurent, Moscovitch, et al., 2011; Viskontas, et al., 2000; but see Voltzenlogel, et al., 2006 for a greater deficit in people with left-lateralized mtle, and see McAndrews, in press, for a review). However, for the current task, memory performance for laboratory events presented verbally was contrasted with performance for non-verbally presented events. Given the left hemisphere s central role in memory for stories presented verbally (Djordjevic, et al., 2010; Frisk & Milner, 1990), I anticipated the possibility of observing an interaction between verbal and non-verbal modes of presentation, and the hemisphere affected by mtle. For this reason, I compared performance between individuals with mtle localized in the language

40 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 28 dominant and the non-dominant hemisphere. Also, past studies have failed to report differences in AM performance based on surgical status, namely, between pre-surgery candidates with seizures from hippocampal origin, versus seizure-free individuals who received a temporal lobe resection (Herfurth, et al., 2010; St-Laurent, et al., 2009; St-Laurent, Moscovitch, et al., 2011; Viskontas, et al., 2000). As such, I did not expect to observe differences in performance between pre and post-surgery participants with mtle with the current paradigm, although it remained possible that the greater excision of dominant temporal neocortex would have a greater influence on narrative retrieval. In Chapter 1, I formulated the hypotheses that the hippocampus is necessary for representing perceptually rich information in memory, and that it is a hub in a network of brain regions that support the retrieval of perceptually rich memory episodes. Based on these hypotheses, the literature and my experimental manipulations, I made the following predictions for the current study: - In controls, I expected more perceptual details to be retrieved in the perceptually enriched Film Clip than in the perceptually impoverished Narrative condition, which would validate my experimental manipulation of perceptual richness. - I expected participants with mtle to retrieve fewer perceptual details than controls across all three memory conditions. This reduction would indicate that the paucity of perceptual AM details observed in people with mtle can also be observed with episodes encoded in the laboratory. - I expected the difference in perceptual details between the Film Clip and the Narrative condition to be greater in the controls than in individuals with mtle, indicating that mtle reduces one s capacity to remember the additional perceptual features presented in the enriched condition. - As the story content of the Film Clip and the Narrative conditions were matched, no differences in the number of reported story details were expected between these conditions in either individuals with mtle or controls. However, the number of story details could differ between the mtle and the control group, based on evidence of the supportive role played by the MTL in memory for episodic details in general.

41 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 29 - There may be an interaction between the number of story details in the two laboratory conditions and the laterality of mtle, due to the verbal and pictorial nature of the used stimuli, although I had no specific expectations about the magnitude of such an effect. Also, based on the literature, I did not expect laterality or surgical status to influence performance in the AM condition. 2.2 Methods Participants Twenty-four participants with mtle (12 with right- and 12 with left-lateralized pathology) and 14 neurologically intact controls were tested on this paradigm. Participants with mtle were recruited through the Epilepsy Clinic of the Toronto Western Hospital following a protocol approved by the University Health Network s Research Ethics Board. Controls were recruited through advertisement in the community, among staff from Toronto Western Hospital, and among friends and colleagues. All pre-surgery participants with mtle were candidates for a unilateral temporal lobe resection, either classic or selective, and had seizures originating in the hippocampus. One presurgery participant had right-lateralized seizures, and four participants had left-lateralized seizures. Two pre-surgery participants were diagnosed with mesial temporal sclerosis (MTS) by a radiologist according to clinical criteria of atrophy on T1-weighted MRI scans and high intensity indicative of gliosis on T2-weighted MRI scans. One pre-surgery participant had an arteriovenous malformation in the medial temporal lobe, and the other two had normallooking brains based on MRI scans. Among the post-surgery participants with mtle, 14 participants (six left, eight right) had received a classic unilateral resection of the anterior temporal lobe that included the temporal pole, the amygdala, the anterior portion of the hippocampus, the rhinal and lateral temporal cortex, and portions of the parahippocampal cortex. The remaining five post-surgery participants (two left, three right) had received a selective amygdalo-hippocampectomy that included the amygdala, the anterior portion of the hippocampus, the rhinal cortex and portions of the parahippocampal cortex. One participant

42 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 30 with a left selective resection had an arachnoid cyst in the right, contralateral temporal lobe that displaced the tissue, which was described as otherwise healthy by a radiologist. In all the other participants, no structural brain damage was observed outside the epileptogenic / resected temporal lobe. All post-surgery participants were seizure free at the time of testing. For the purpose of this study, participants with mtle were classified according to whether their pathology was ipsilateral or contralateral to their language dominant hemisphere. The encoding of narratives requires more language comprehension than the encoding of film clips, and this process may be more taxing for individuals with mtle whose seizure focus or surgical resection is located in the hemisphere that supports language, which is typically the left. In order to quantify potential language-driven effects, two post-surgery participants with left-lateralized mtle whose language was atypically localized in the right hemisphere were merged with pre- and post-surgery participants with right mtle whose language was left-lateralized. This group is referred to as non-language-dominant. Participants with left mtle whose language was left-lateralized (n=10) are referred to as languagedominant. All participants were fluent English speakers, among whom only two controls were not native English speakers. Table 2.1 presents the mtle participants performance on neuropsychological tests, as well as additional demographic information about the mtle participants and the control group. The Matrix Reasoning Subscale of the Wechsler Abbreviated Scale of Intelligence was administered to controls to estimate non-verbal intelligence and compare their scores to those of patients. Two pairs of tests, the Warrington Words and Warrington Faces, and the Rey Auditory Verbal and Rey Visual Design Learning Tests, indicate relatively poorer memory for verbal and non-verbal material in patients with language-dominant and non-languagedominant mtle, respectively. Both language-dominant and non-language-dominant mtle participants performed similarly on a task of verbal fluency, but participants with languagedominant mtle were poorer at naming.

43 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 31 Table 2.1 Mean Demographic and Neuropsychological Characteristics of the Control and mtle groups Controls N-Domin mtle Domin mtle Norms (n = 14) (n = 14) (n = 10) Gender (M / F) 2M / 12F 4M / 10F 3M / 7F n/a Surgical Status (Pre / Post - surgery) n/a 13 Post / 1 Pre 6 Post / 4 Pre n/a Age in Years 40.1 (9.9) 40.6 (7.9) 47.7 (7.8) n/a Years of Education 15.9 (2.3) 13.9 (2.1) 14.4 (2.3) n/a WASI Full Scale IQ n/a (7.1) (14.4) 100 (15) b Performance IQ n/a (8.0) (15.3) 100 (15) b Verbal IQ n/a (9.6) (14.9) 100 (15) b WASI Matrix Reasoning Subtest (Standard Score) 11.5 (3.0) 11.3 (3.4) 10.7 (3.1) 10 (3) b RAVLT Total Recall Score n/a 49.6 (7.0) 44.5 (8.8) 51.1 (8.6) b RVDLT Total Recall Score n/a 32.9 (11.2) 37.2 (11.6) 44.4 (12.4) a Warrington Words n/a 47.5 (3.2) 41.6 (4.0) 45.5 (3.2) c Warrington Faces n/a 39.3 (5.5) 41.3 (3.5) 44.8 (3.3) c Verbal Phonemic Fluency (FAS) n/a 38.0 (7.0) 38.7 (9.1) 34.2 (12.5) b Boston Naming Test n/a 53.6 (7.6) 47.8 (10.7) 55.5 (3.9) a Note. Standard deviation is between parentheses. Norms were obtained from years old from a Spreen and Strauss (1991), b Strauss, Sherman and Spreen (2006), and c Warrington (1984). Domin = language-dominant; N-Domin = non-language-dominant; F = female; M = male; IQ = Intellectual Quotient; n/a = not applicable; RAVLT = Rey Auditory Verbal Learning Test, RVDLT = Rey Visual Design Learning Test; WASI = Wechsler Abbreviated Scale of Intelligence Paradigm A transcript of the verbatim instructions given to the participants is provided in Appendix A Encoding. Eleven memories from the participants personal life were selected from a six page list of suggestions (e.g., New Year s Eve celebration, Learning about someone s condition, Walking through a new city for the first time ; See Appendix B). One memory was reserved for practice. Participants were instructed to select personal events that were older than a year, and that took place within the course of a few minutes to a few hours. A title was given to each AM, and its date and duration were recorded. In most cases, memories were selected on testing day, right before encoding the laboratory events. In a few cases, however, personal memories were collected a few days in advance (e.g., over the phone).

44 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 32 Those participants were read the titles chosen for their AMs right before encoding, to make sure the cues were still effective. During the encoding phase which lasted 11 minutes plus 2 minutes of instructions, participants were shown 20 short laboratory events (LEs) on a computer screen (Lenovo T500) using E-Prime 2.0 (Psychology Software Tools Inc. Release candidate version ). Sound was delivered through headsets. For each participant, events were randomly assigned to one of two conditions, so that 10 LEs were presented as verbal narratives, and 10 LEs were presented as film clips. The film clips were 23s in duration, and contained minimal or no English dialogue, so that the story was carried by the actions of the actors on screen. Out of 20 clips, only 3 contained minimal English dialogue (e.g., goodbye while waving), 10 contained no dialogue, and 7 contained dialogue in a foreign language. The clips were presented at the center of a 15 screen, within a window that occupied 45% and 42% of the screen s width and height, respectively. The narratives were verbal descriptions of what took place in the film clip. Appendix C includes all the narratives and a screen shot from the corresponding film clips. Five written sentences were presented in the middle of a white screen (Courier New, Black, Font 18), one sentence at a time, for 6 seconds each. A male voice-over with an English-Canadian accent, recorded with Window XP s Sound Recorder and a microphone, was played simultaneously, so that each sentence was read to the participant. A title was given to each LE, which was displayed on the screen for 2s immediately before and after the LE was presented. Participants were instructed to pay attention to the titles, and to try their best to remember what took place in the story ( who did what, what was the situation? ), as they would need to recall it later, in their own words. LEs were presented in blocks of 3 or 4 from the same condition, Narrative or Clip, in the following order: 3 Narratives, 3 Clips, 3 Narratives, 3 Clips, 4 Narratives, 4 Clips. Events were assigned randomly to either the Clip or the Narrative condition, and their order of presentation, namely the block to which they were assigned, and their order within that block, was also randomized. Small blocks were preferred over a complete randomization of trials in order to

45 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 33 reduce the distraction engendered by switching from reading to watching a clip. Blocks were presented in a continuous manner, without breaks Retrieval. Immediately after the encoding phase, additional instructions were provided to the participants (see Appendix B), followed by a practice session, all lasting minutes. Retrieval trials were presented using E-Prime 2.0 (Psychology Software Tools Inc. Release candidate version ). For the practice session, two additional LEs were shown, one Narrative and one Clip, followed immediately by one AM, one Narrative and one Clip retrieval practice trial. One of the participant s 11 personal memories was reserved for practice. During the actual retrieval, the condition, Laboratory Memory or Autobiographical Memory, was indicated on screen for 1s, followed by either three or four trials from that condition. Trials for LEs encoded as Narratives and Clips were intermixed randomly within the Laboratory Memory condition. For each trial, a title was presented to the participant for 16s. During those few seconds, participants were instructed to either relive their personal memory in as much detail as possible in their mind s eye in the AM condition, or to silently recount what took place in the event, from beginning to end, in the LE conditions. The use of a retrieval time limit allowed to constrain the amount of AM details retrieved by participants, and to equate retrieval content between the current study, and a version of this task adapted for functional MRI presented in the next chapters. Based on piloting, 16s was determined to be sufficient for participants to recall most LEs. After the 16s lapsed, participants answered four questions about what they were able to recall within the time allotted. First, they rated their memory s Story Content on a 1-5 Likert scale, for which 1 = no LE / AM details, and 5 = all the LE details / my most detailed AM (in the context of this task). Then, participants rated the Vividness of their memory on a 1-5 scale, for which 1 = no visual / perceptual details, and 5 = my most vivid LE / AM (in the context of this task). Vividness was defined to participants as the totality of sensory details (visual, auditory, olfactory, gustatory, tactile, proprioceptive, etc.) they experienced while recalling the memory. Participants answered by typing on a keypad attached to the computer with a USB port. Then, while recorded with a microphone, participants narrated all the details they had time to recall about what happened during their personal event, or what happened in the LE

46 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 34 (who did what, what was the situation, etc.). Finally, they listed all the perceptual elements they had time to experience in their mind during the 16s allotted, such as images of characters or people, scene elements, scents, tastes, physical sensations, sounds, etc. Participants were given a maximum of 100s to narrate the event and the perceptual details, respectively. Participants underwent three blocks of retrieval trials, interspaced by two preprogrammed breaks of minimum 30s. Two blocks contained 3AMs and 6LEs ((3 AM, 3 LE, 3 LE), and (3 LE, 3 LE, 3 AM), respectively), and one block contained 4AM and 8 LE trials (4 LE, 4 AM, 4 LE). Each AM or LE was attributed to a block randomly, and the order of first, second and third block was randomized for each participant. On average, participants took about 45 minutes to complete the retrieval portion of the study Scoring and data analysis. Recordings from the retrieval trials were transcribed by the author of this thesis and a lab assistant (RJ), and were coded according to a system adapted from Levine and colleagues (2002). Appendix D contains a detailed description of the scoring system employed here, and explains how it differs from Levine and colleagues. Each retrieval trial produced two recordings: first, the recall of story content, and second, the recall of perceptual content. Transcripts from both recordings were broken down into meaningful units of information, or details. Only information that was specific to what took place within the LE or AM was scored. Less specific information, such as general opinions or facts, which fall into what Levine et al. (2002) refer to as External details, was not scored. In both recordings, memory-specific details were categorized into one, or both, of two detail categories: Story Details, and Perceptual Details. Story Details corresponded to information about what happened, such as the type of event described (e.g., a lab meeting), the people who were present, their actions and conversations, and other happenings. Thoughts or emotions experienced by the person at the time a personal event took place, what Levine et al. (2002) term Thought / Emotion details, were considered mental events, and were counted as Story details (e.g., I was shocked when she told me ). Opinions about an event that were not experienced at the time the event took place were not counted (e.g., this is a very sad story). Perceptual details corresponded to information experienced through the senses, that was either visual (light intensity, elements from a scene, pieces of clothing, objects in the

47 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 35 room), auditory (laughter, street sounds), olfactory or gustatory (smell of rain, coffee taste), tactile or proprioceptive (feeling tired or sick, being cold, dizzy, feeling your skin burning). While most details were categorized exclusively as Story details or as Perceptual details, some details were counted in both detail categories (e.g., a boy was leaning back against a wall (Story, Perceptual) by his bicycle (Perceptual) ). This dual categorization differed from Levine s et al. (2002) scoring system, according to which detail categories are exclusive. I did not score information about time (e.g., it happened two years ago ; that was in January ), unless it provided perceptual information (e.g., it was a hot summer day ), or information about the type of event described (e.g., Christmas dinner). Details that were inaccurate in the LEs were tallied up separately as Story Errors rather than being counted as Story or Perceptual details. For the AM condition, participants were given the benefit of the doubt unless 1) a participant corrected him/herself, or 2) two memory elements contradicted each other, in which case the correct detail was determined arbitrarily. For the Narrative and Clip conditions, Story details that differed from what was presented at encoding (e.g., made up or distorted Story details) were counted as Errors. In the Clip condition, perceptual elements that were not part of the original film clip (e.g., recalling a hat when the person did not wear one) were considered Perceptual Errors. For the Narrative condition, imagined percepts that were not shown at encoding were accepted as Perceptual details (e.g., I could imagine the boy s face with tears running down his cheeks ), unless they were unrelated to or in conflict with the studied narrative (e.g., visualizing a young boy running down the street when the narrative s main character was a girl; imagining additional characters not mentioned in the story). An extract from a control participant s recall scored according to the employed technique is shown below in Figure 2.1. Additional examples of scored recall per condition are included in Appendix E.

48 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 36 1 st Recording (Story Content) What happenend? For this story no paper in the washroom. There s an older man (Pe ) in the washroom, he washed his hands (Sto ), and he s looking for paper to dry his hands (Sto ), there s no paper (Sto ). He checked in the bathroom (Sto ), there s no toilet paper either (Sto ). He tried to use the dryer (Sto ), the dryer won t even work (Sto ). 2 nd Recording (Perceptual Content) What did you perceive in your mind s eye? I envisioned for this story no paper in the washroom, frustration on the guys face (Pe ) because there s no paper for him to dry his hands after washing them. 1 st Recording (Story Content) What happenend? For this story I envision a cyclist (Sto ), he s waiting outside an apartment building (PeErr ) for a lady (Sto ) as she walks out of the building (Sto ). 2 nd Recording (Perceptual Content) What did you perceive in your mind s eye? I envision a guy who s actually the cyclist (Pe ), and he walks up to the older lady (StoErr ) to tell her that he s been waiting for her (StoErr ) and he looks very excited to see her (Pe) and she looks pretty surprised (StoErr ). 1 st Recording (Story Content) What happenend? There s this lady standing outside (Sto ) in an open area (Pe ) and she was playing with some toy, a toy dog (Sto ) and all of a sudden a guy, an older man (Pe), picks up a long (Pe) shot gun (Sto ) and he walks towards the window (Sto, Pe) and he aims it, he opens the window (Sto ) and aims it at the maybe it was the lady or toy dog she was using (Sto ) and he shot the toy dog dead (Sto ) and the lady looked up at the window (Sto ) and she looked very surprised (Pe ). 2 nd Recording (Perceptual Content) What did you perceive in your mind s eye? Visual details for this story, a picture of the gun, and a picture of the gunmen at the window aiming the gun at the woman s toy dog (Pe). Figure 2.1 Recall for three laboratory events scored according to the procedure employed for this thesis. Note: No additional points were given for repetitions. Pe = Perceptual detail; PeErr = Perceptual Error detail; Sto = Story detail; StoErr = Story Error detail. During piloting, it was observed that several Perceptual details that were provided during the first (story content) recording were not repeated during the second (perceptual content) recording. Also, a few participants provided additional Story details during the second (perceptual content) recording. Hence, I decided to tally up Story and Perceptual details across the two recordings, so that a single score was available per trial for each detail category. These two scores, Story and Perceptual details, were then averaged per condition, per participant. However, the number of Story and Perceptual details retrieved during the first recording is also shown per condition in Appendix F. One of the purposes of the current analysis was to gain insight into participants retrieval experience in order to interpret patterns of brain activation measured with a version of this task adapted to functional magnetic resonance imaging (fmri; these data are presented in Chapters 3 & 4). For the fmri analysis, I excluded trials that were

49 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 37 given a Story Content rating of 1 ( no story details ) by the participants, in order not to dilute the neural signature of the memory conditions with trials during which nothing was retrieved. In the current analysis, I averaged the number of details retrieved over successful trials only (Story Content rating > 1), as well as over all trials (Story Content rating = 1 to 5). There was little difference between the two sets of results (see Figure 2.4 A and B), and so I am reporting analyses performed on successful trials only, in order to be consistent with the fmri analysis. The mean numbers of successful trials per condition are provided in the results section. Trials with missing recordings due to recording errors were also excluded Scoring reliability: intraclass correlations. Intraclass correlations (McGraw & Wong, 1996) were calculated using SPSS 15.0 (Statistical Package for the Social Sciences, IBM) for the tallied number of Story details and of Perceptual details, respectively. The author and a blind external scorer both scored the same 120 memory transcripts (40 AM, 40 Narratives and 40 Clips) obtained from 4 different participants. Intraclass correlation coefficients (Two-Way mixed model, absolute agreement, single measures) are reported in Table 2.2 and were deemed acceptable. Table 2.2 Inter-Rater Reliability: Intraclass Correlation Coefficients per Detail Subcategory Global Score Score per Memory Condition AM Narrative Film Clip n = 120 n = 40 n = 40 n = 40 Story Details Perceptual Details Note: Intraclass correlations were calculated following the guidelines of McGraw & Wong (1996). The global score was calculated based on trials from all three memory conditions. AM = Autobiographical Memory LIWC. Memory transcripts were fed to the LIWC2007 computer software (Linguistic Inquiry and Word Count) for an automated word count analysis. Transcripts of a participant s first (Story Content) and second (Perceptual Content) recordings from all successful trials (Story Content rating > 1, excluding trials with missing recordings) were saved into a single Microsoft Word document for each memory condition. Each document was fed to

50 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 38 LIWC2007, which calculated the number of words falling into each of the 80 different categories defined by its 2007 English dictionary, which included pronouns, verbs, punctuation and categories defined based on semantics (e.g., humans ; Pennebaker, et al., 2007). These results were imported into Microsoft Excel, and the total number of words per category was divided by the number of successfully retrieved memories entered in the analysis for each condition. Each participant obtained one score per memory condition for each of the word categories defined by the software. Here, I am reporting results from two word categories that are relevant to my hypotheses: the Perceptual Processes word category, and its largest subcategory, Visual Processes. These two categories included words that referred to the process of perceiving ( observing, seen, heard, feeling, listen, touch ) and seeing ( view, saw, seen ), respectively. The goal of this LIWC analysis was to measure perceptual memory content in a non-user dependent manner that could validate results from the manual scoring. While I expected the LIWC measures to provide results similar to the tally of Perceptual details, the two techniques quantified related but different elements of the memory transcripts. The LIWC measures counted words reflecting a perceptual experience in the narrator (e.g., I heard a loud scream coming from the living room ), while manually tallied details were based on the content of what was perceived (e.g., I heard a loud scream coming from the living room ). Results from LIWC categories that reflect additional memory features, such as affect and personal relevance, are presented in Appendix G. 2.3 Results Throughout this chapter, I conducted separate analyses on the AM condition, while I compared the Narrative and the Film Clip conditions to each other. The rationale was that the two laboratory conditions were well matched in terms of overall story content, while AM contained more information, and so it would be difficult to interpret a direct comparison between AM and the laboratory tasks. Nevertheless, observing similar patterns of group differences in both the laboratory tasks and the AM task in the same cohort of participants

51 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 39 would indicate that the AM deficit observed in individuals with mtle extends to new memories acquired in the laboratory under greater levels of experimental control Self-Ratings Following the 16s allowed for retrieval, participants were requested to rate the story content ( how much do you remember about what happened? ) and the vividness ( how much sensory information did you visualize at retrieval? ) of their memory, on 1-5 Likert scales. Mean ratings were calculated per condition for each of the groups (see Figure 2.2). A MANOVA conducted on the 6 ratings (Story Content and Vividness, for each of the three conditions), with language dominance entered as a fixed factor, revealed no overall group difference between language dominant and non-dominant participants with mtle (Wilk s λ =.68, p =.30). Separate t-tests performed on each rating also failed to reveal any significant differences between the two mtle groups (p >.05). Based on these results, the two mtle groups were merged for subsequent rating analyses. Mean ratings were compared between the controls and participants with mtle for the AM condition. A 2 x 2 ANOVA with repeated measure over the rating scale (Story Content and Vividness) did not reveal significant main effects of group (F(1, 36) = 0.27, p =.61) or scale (F(1,36)=2.88, p =.10), nor a significant group x scale interaction effect (F(1, 36) = 2.40, p =.13). The results suggest a ceiling effect in the AM condition. Although participants were instructed to compare AMs to AMs, and stories to stories rather than to compare memories across conditions, it is possible that alternating between AM and LE trials at retrieval elicited a direct comparison between the conditions, inflating AM ratings due to their greater content.

52 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 40 Figure 2.2 Mean self-ratings of Story Content and Vividness (on 1-5 Likert scales) for all (successful and unsuccessful) trials plotted per memory condition for people with language dominant (Domin) and non-dominant (N-Domin) mtle and for controls. Error bars represent standard errors of the mean (SEM). Note: AM = Autobiographical Memory; Narra = Narrative Condition; Clip = Film Clip Condition. Next, mean ratings for the Narrative and Clip conditions were compared between participants with mtle and controls. A 3-way repeated measures ANOVA with scale (Story Content and Vividness) and condition (Clip and Narrative) as within-subject factors revealed a significant main effect of group, with controls producing higher ratings than participants with mtle (F(1, 36) =11.88, p <.005). The test also revealed a significant main effect of condition, with the Clip ratings being significantly higher than the Narrative ratings (F(1,36)=83.46, p <.001), and a significant effect of scale, with the Vividness ratings being significantly lower than the Story Content ratings (F(1,36) =6.66, p <.05). None of the interaction effects were significant (p >.05). These results indicate that overall, participants with mtle rated their memories for Narratives and Clips significantly lower than controls did, and that both groups rated their memory for Narratives as lower in story content and vividness than their memory for Clips.

53 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 41 Based on the self-ratings, trials for which participants failed to retrieve an AM or a LE (Story Content rating = 1, no story / AM details ) were identified. I calculated the number of trials for which a memory was retrieved successfully for each condition (Story Content rating = 2 to 5), which I compared between controls and participants with mtle (see Figure 2.3). A 2 x 2 repeated measures ANOVA with condition (Narrative and Clip) as within-subject factor revealed that controls retrieved significantly more memories than participants with mtle (F(1, 36) = 9.23, p <.005). A significant main effect of condition (F(1,36) = 15.75, p <.001) revealed that fewer memories were retrieved successfully in the Narrative condition in comparison to the Clip condition. These results could be due to narratives being less memorable, less distinguishable, or less easily accessed with cuing than the film clips. A trend for a group x condition interaction effect (F(1,36) = 3.54, p =.068) indicated that this condition effect (Clip > Narrative) was mostly driven by the mtle group. Figure 2.3 Mean number of trials for which a memory was successfully retrieved (Story Content rating > 1; maximum 10 trials per condition) plotted per condition for the language dominant (Domin) and non-dominant (N-Domin) mtle groups and the control group, respectively. Error bars represent SEM. Note: AM = Autobiographical Memory; Narra = Narrative Condition; Clip = Film Clip Condition.

54 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES Detail Scoring Story and Perceptual details. Immediately after the self-ratings, participants were recorded while narrating the story content and perceptual features of the memory they retrieved within the 16s retrieval phase. From the transcripts of these recordings, I tallied up Story and Perceptual details. In Figure 2.4, the mean number of these details is plotted per group, per condition. The means were calculated for successful trials only (Story Content ratings > 1; Figure 2.4A), as well as for all trials (successful and unsuccessful; Story Content ratings 1-5; Figure 2.4B). A score of 0 details was given when no memory was retrieved. When unsuccessful trials are included, a slight decrease in numbers of details in the Narrative condition, for which fewer trials were retrieved successfully, was observed (see Figure 2.3). However, as can be seen in Figure 2.4, including or excluding unsuccessful trials does not change the overall pattern of results I observed. Excluding unsuccessful trials based on the absence of Story details, rather than based on self-ratings, lead to results indistinguishable from those presented in Figure 2.4A. Interestingly, self-ratings suggest a lack of group difference for AM, and a subtle deficit in the two LE conditions in the mtle participants, but the current scoring-based analysis reveals an important loss of details in participants with mtle in all the memory conditions. Figure 2.4 Mean tallied number of Story Details and Perceptual Details per memory condition for A. successful trials only (Story Content rating > 1), and B. for all trials. Trials with recording

55 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 43 errors, and incorrect details, were excluded from these counts. Details are plotted for language dominant (Domin) and non-dominant (N-Domin) mtle participants, and for controls. Error bars represent SEM. Note: AM = Autobiographical Memory; Narra = Narrative Condition; Clip = Film Clip Condition. Given the similarity of results obtained with and without the inclusion of unsuccessful trials, I conducted statistical analyses on successful trials only (Story Content ratings = 2 to 5). This decision was based on the desire to be consistent with the fmri study conducted using a modified version of the current paradigm (see Chapters 3 & 4), for which trials with Story Content ratings equal to 1 were also excluded. By excluding unsuccessful trials in both analyses, I felt confident making inferences about the patterns of brain activation observed in the different conditions based on insight from the current detail analysis. A MANOVA comparing language dominant and non-dominant mtle participants on all six details measures (Story and Perceptual details for each memory condition) failed to reveal a significant group difference (Wilk s λ =.81, p <.68). Also, none of the individual t- tests conducted on these measures was significant (p >.05), so all individuals with mtle were merged into a single group. Next, I compared the number of Story and Perceptual details for the AM condition between this single mtle group and the controls. A two-way repeated measures ANOVA with details (Story and Perceptual) as within-subject factors revealed significant main effects of group (F(1, 36) = 11.57, p <.005) and type of details (F(1, 36) = 45.13, p <.001), with the mtle group recalling fewer details than the control group, and with participants recalling fewer Perceptual details than Story details. A significant group x detail interaction effect (F(1,36) = 7.38, p =.01) was also observed, indicating that the mtle group had a larger memory deficit for Perceptual details than for Story details. I then compared the number of Story and Perceptual details recalled by the control and the mtle group in the Narrative and Clip conditions. A three-way ANOVA with repeated measure over detail type and memory condition revealed significant main effects of group (F(1,36) = 42.53, p <.001), detail type (F(1,36) = 5.22, p <.05), and condition (F(1,36) = 44.75, p <.001). I also observed significant group x details (F(1,36) = 13.48, p <.005), group x condition

56 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 44 (F(1,36) = 17.29, p <.001), details x condition (F(1, 36) = 39.15, p <.001) and group x details x condition (F(1,36) = 21.71, p <.001) interaction effects. These results indicated a general decrease in all detail categories in the mtle group. More interestingly, the large increase in Perceptual details in the Clip in comparison to the Narrative condition observed in the control group was significantly reduced in the mtle group. In other words, the gain in Perceptual details that controls experienced with the recall of perceptually enriched clips was reduced in participants with mtle, whose recall of Perceptual details for the enriched clips was as poor as when they recalled the perceptually impoverished narratives. Neither the mtle (t(23) = 1.19, p =.25) nor the control group (t(13) = 1.43, p =.18) showed differences in the number of Story details recalled between the Narrative and the Clip condition, indicating that story content was equated between these conditions in both groups Errors. Participants were given the benefit of doubt as to the veracity of AM details because their accuracy could not be verified. However, in the Narrative and Clip conditions, Story and Perceptual details that did not correspond to what was presented at encoding were counted as errors. Figure 2.5 shows errors plotted (A.) as a mean number per condition, and (B.) as a percentage of the total number of (correct and incorrect) details from its category (either Story or Perceptual). Figure 2.5 Mean number of incorrect Story and Perceptual Details per memory for the Narrative and Clip conditions (excluding trials with recording errors). Errors are plotted A. as a tallied number, and B. as a percentage of total (correct and incorrect) details from its category

57 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 45 (Story or Perceptual details). Details are plotted for participants with language dominant (Domin) and non-dominant (N-Domin) mtle, and for controls. Error bars represent SEM. Note: Narra = Narrative Condition; Clip = Film Clip Condition. As can be seen in the figure, only one or two errors were made per trial, on average, for each condition, and the total number of errors made by individuals with mtle was within the same range as that of controls. When looking at percentages of errors, higher values can be noted for the mtle group than for the controls, due to the mtle groups lower levels of correct details. Due to the low number of errors, and to a lack of hypothesis regarding errors committed by the two groups, I am not reporting statistical analyses on error data LIWC Memory transcripts were fed to the LIWC software, which counted the number of words falling under categories identified based on its 2007 English dictionary (Pennebaker, et al., 2007). Unlike the manual scoring, LIWC could not discriminate between what was correctly and incorrectly recalled, but given the small number of errors made per memory (see Figure 2.5), this was not an issue. Here, I am presenting results from two word categories relevant to my hypotheses: Perceptual Processes and its largest subcategory, Visual Processes. I conducted a MANOVA comparing results from participants with language dominant and non-dominant mtle on all six measures discussed here, that is, mean word counts per memory in the Perceptual Processes and the Visual Processes categories, for each memory condition. The omnibus test failed to reflect a significant group difference (Wilk s λ =.89, p =.90). Also, none of the individual t- tests conducted on these measures was significant (p >.05), so all individuals with mtle were merged into a single group for subsequent analyses. Figure 2.6 illustrates the mean number of words per memory from the Perceptual Processes and Visual Processes categories.

58 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 46 Figure 2.6 Mean number of words per successful (Story Content rating > 1, complete recording) trial from LIWC2007 s Perceptual Processes and Visual Processes categories. Numbers of words are plotted per memory condition for each group. Error bars represent SEM. Note: AM = Autobiographical Memory; Domin = language dominant; Narra = Narrative Condition; N-Domin = non language-dominant, Clip = Film Clip Condition. I compared the number of words falling under the Perceptual Processes category between the mtle and the control group for the Narrative and the Clip condition. A two-way ANOVA with repeated measure over condition (Narrative and Clip) revealed significant main effects of group (F(1,36) = 15.60, p <.001) and condition (F(1,36) = 19.94, p <.001), indicating that controls produced significantly more Perceptual Processes words than individuals with mtle, and that more words were produced during the Clip than the Narrative condition. The analysis also revealed a significant group x condition interaction effect (F(1,36) =4.86, p <.05), indicating that the difference in Perceptual Processes words between the Narrative and Clip conditions was greater in controls than in participants with mtle. This pattern is very similar to what was observed for tallied scores of Perceptual details: memory for enriched clips seemed as perceptually impoverished as memory for narratives in the mtle group. In fact, Perceptual details correlated highly with the number of Perceptual Processes words recalled across

59 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 47 participants (r =.76,.75 and.76 for the AM, Narratives and Clips conditions, respectively; p <.001). A similar analysis conducted on the Visual Processes word category revealed significant main effects of group (F(1, 36) = 11.86, p <.005) and condition (F(1, 36) = 27.63, p <.001), and a trend for a group x condition interaction effect (F(1, 36) = 3.99, p =.054), mimicking the pattern observed for Perceptual Processes words. I also tested for group differences on these two word categories for the AM condition. T-tests revealed that controls produced significantly more Perceptual Processes (t(36) = 2.45, p <.05) and Visual Processes words (t(36) = 2.43, p <.05) than participants with mtle. Again, these patterns are consistent with the group differences observed with the analysis of Perceptual details. They also reveal that the automated count of Perceptual Processes words could serve as a reasonable substitute for the manual scoring of Perceptual details Effect of Surgical Status The proportion of pre- and post-surgery participants with mtle was unbalanced between the language dominant and the language non-dominant mtle groups, and so surgical status was not orthogonal to laterality in this mtle sample. Still, I conducted a preliminary analysis to assess the effect of surgical status on the different measures reported in this chapter, that is, ratings, scored details and LIWC outputs. A series of non-parametric Kruskal- Wallis tests comparing pre-surgery participants (n = 5), participants with a selective hippocampo-amygdalectomy (n = 5), and participants with a classic resection of the anterior temporal lobe (n = 14) did not reveal significant differences among these groups. The only significant group difference was observed for ratings of Vividness in the AM condition (χ 2 (2, N = 24) = 6.63, p =.04). In this condition, pre-surgery participants produced significantly lower Vividness ratings than the two post-surgery groups. Overall, this preliminary analysis provides no indication that a post-surgery status influenced mtle participants performance negatively on the current task.

60 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES Discussion Summary of Findings I designed a new paradigm that differentiated successfully between two complex episodic memory dimensions: perceptual richness, which corresponded to what people perceived in their mind s eye or through their mind s senses, and story content, which corresponded to what happened over the course of an event. In controls, the amount of retrieved story content was similar between the perceptually enriched Film Clip condition, and the perceptually impoverished written Narrative condition, while the amount of retrieved perceptual content differed significantly between the two conditions. A considerable amount of Story and Perceptual details were retrieved in the AM condition, which is consistent with evidence that memory for event-specific personal episodes is characterized by a rich narrative structure (Conway, 2009; Neisser, et al., 1996; Radvansky, et al., 2005), and by experience-near sensory content (Brewer, 1986, 1995; Conway, 2009; Conway & Loveday, 2010; Moscovitch, et al., 2005). My main interest was how the two laboratory conditions might line up against AM retrieval with respect to the impact of medial temporal damage. Two main patterns of results emerged from comparing performance between participants with mtle and healthy controls on this paradigm. First, I observed a general loss of story content in the mtle group in all three memory conditions: AM, Clip and Narrative. This impairment occurred regardless of the nature of the stimuli: mtle participants recalled the same number of Story details in the Narrative and in the Film Clip condition, indicating that their loss of story content was unrelated to the perceptual richness of the stimuli. Consistent with this loss of story content, evidence indicates that damage to the MTL impairs the detailed recollection of episodic memory, which renders memory more schematic or gist-like (Moscovitch, et al., 2005; Nadel & Moscovitch, 1997; Rosenbaum, et al., 2009; St-Laurent, Moscovitch, et al., 2011; Winocur & Moscovitch, 2011). Although I hypothesized that perceptual richness is an important determinant of MTL engagement at retrieval, this result indicates that non-perceptual information can also be affected by damage to the MTL, especially if it is highly context-specific, like specific thoughts, rapid sequences of actions, etc. Of note, the similar number of Story details retrieved by participants with mtle in the Narrative

61 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 49 and the Clip conditions indicates that the two conditions were matched for story content, just as for the controls. Secondly, and more interestingly, a loss of Perceptual details was observed in the mtle group that was most pronounced in the perceptually enriched memory conditions, AMs and Clips. While controls retrieved more Perceptual details in the Clip than in the Narrative condition, participants with mtle demonstrated the same paucity of Perceptual details in the Film Clip than in the Narrative condition, suggesting that their memory for the clips was just as perceptually impoverished as their memory for the narratives. In other words, the mtle group failed to retain or retrieve much of the multi-sensory perceptual details presented with the Clip condition at encoding. This pattern of results was also observed with an automated count of words reflecting perceptual and visual processes. In the AM condition, a significant interaction effect revealed that Perceptual details were more readily lost than Story details in the mtle group, which replicates results from a previous study comparing mtle participants to healthy controls on a different AM task (St-Laurent, et al., 2009). The fact that the AM deficit that has been reported in individuals with mtle can also be observed with well controlled laboratory memories is consistent with the observation that lesions to the MTL interferes with memory for recent as well as distant episodes. In addition, the current results contribute to the memory literature by highlighting the importance of perceptual information to the characterization of the memory deficit caused by MTL dysfunction. It must be specified that while Perceptual details were retrieved in the Film Clip condition, they were imagined or elicited in the Narrative condition, for which no sensory details were actually shown at encoding. The current results indicate that the retrieval of narratives elicited some sensory imagery in the mtle and control groups. In controls, such imagery, which was reflected in the number of Perceptual details enumerated in the Narrative condition, was no match for the high numbers of Perceptual details recalled in the Clip condition, as clips contained visuo-spatial scenery details, motion, colour, voices, background noise and/or music. In participants with mtle, however, the perceptual experience associated with the retrieval of clips was no richer than that which was elicited when recalling bare-bones written stories. Also, recalling narratives elicited significantly fewer Perceptual details in the

62 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 50 mtle than in the control group, which is consistent with a literature linking MTL function to one s capacity to imagine new visuo-spatial scenes, future events and other detailed mental constructs (Addis, Wong, & Schacter, 2007; Hassabis, et al., 2007; J. S. Rabin, Gilboa, Stuss, Mar, & Rosenbaum, 2010; Rosenbaum, et al., 2009) Self-Ratings The self-rating results, which reflected subjective impressions of memory content, tell a somewhat different story from the scoring of memory details. Both the mtle and the control groups rated Story Content as higher for clips than for narratives, even though mean numbers of Story details did not differ between these two conditions within groups. The literature indicates that experiencing imagery at retrieval is linked to one s confidence in the accuracy of one s memory (Brewer, 1995; Rubin, et al., 2003). In controls, the high imagery content of memory for film clips may have contributed to inflate participant s subjective sense of retrieval success in this condition, in comparison to the perceptually impoverished Narrative condition. However, this explanation cannot explain the discrepancy between the self-ratings and the detail count in the mtle group, as they retrieved a similar number of Perceptual details in the Clip and the Narrative condition. Additionally, self-ratings did not differ between groups in the AM condition, and only differed slightly in the two LE conditions, while the scoring-based analysis reveals a significant loss of details in the mtle group in all three conditions. In fact, high ratings in the AM condition suggest a ceiling effect. Although participants were instructed to use the full scale to rate their AMs, alternating between AM and laboratory trials may have contributed to inflate their AM ratings, because laboratory memories had fewer details than AMs. Also, the absence of a group difference in the AM condition may reflect anchoring differences between the mtle and the control group, as participants rate their AMs in relation to their best personal memories rather than to an absolute standard of goodness. In the LE conditions, memory was based strictly on the retrieval of specific episodic details, which individuals with mtle were aware of lacking, because their ratings were lower than those of controls. In the AM condition, however, even though they retrieved fewer event-specific experiential details, participants with mtle probably retrieved normal amounts of semantic and general knowledge information (Addis, Moscovitch,

63 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 51 et al., 2007; Levine, et al., 2002; St-Laurent, et al., 2009; Viskontas, et al., 2000; Voltzenlogel, et al., 2006), and this availability of information may have influenced their ratings. Overall, detail coding provided a clearer and more objective portrayal of the mtle participants memory deficit than the self-ratings, which proved to be a less sensitive measure of memory performance on my task. The comparison highlights the limitations on the use of participants own ratings to evaluative their memory, while also providing a sense of what may underlie a participant s subjective ratings Theoretical Implications I report that perceptual richness is especially prone to disruption when MTL integrity is compromised. These findings confirm results from previous studies linking MTL function to the imagery / perceptual content of episodic memory and other mental constructs (e.g., imagined new scenes; Greenberg & Rubin, 2003; Hassabis & Maguire, 2009; St-Laurent, et al., 2009; St Jacques, Conway, Lowder, & Cabeza, 2011, April; St Jacques, Rubin, & Cabeza, in press). My study s strength lies in the additional control provided by the use of laboratory stimuli, which, unlike naturalistic memories, are matched for age, personal relevance, emotionality and degree of rehearsal. Also, I manipulated perceptual richness while controlling for narrative-driven story content, two memory dimensions often correlated in AM (Brewer, 1995). Controlling for these different factors allows me to conclude that although retention or retrieval of both event content and perceptual details is impaired in mtle patients, perceptual richness is especially vulnerable. The latter finding suggests that perceptual richness is an important determinant of hippocampal engagement during episodic memory retrieval, whether it is of memories acquired in the laboratory or outside of it. The hippocampus forms indirect reciprocal connections with the apex of the ventral visual stream, and with cortical regions processing multimodal spatial information, enabling it to integrate such features from memory into a rich and coherent mental representation at retrieval (Coward; Derdikman & Moser; Eichenbaum & Lipton, 2008). Eleanor Maguire s group has shown that patients with amnesia due to bilateral hippocampal damage struggle to imagine rich visuo-spatial scenes they have never experienced before, leading them to claim that the hippocampus plays an important role in constructing scenes from multimodal memory details

64 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 52 (Hassabis, et al., 2007; Hassabis & Maguire, 2009). While my analysis included both spatial and non-spatial features into the Perceptual details category, it revealed a paucity of sensory episodic memory details in patients with damage to the MTL, which is consistent with Maguire and her colleagues views on hippocampal function. The experience of vivid imagery during memory retrieval contributes to one s sense of reliving the past (Brewer, 1995; Park, St- Laurent, McAndrews, & Moscovitch; Rubin, et al., 2003), a phenomenon known as recollection. Recollection is a hallmark of episodic memory (Tulving, 1985, 2002; Wheeler, et al., 1997), and it is well established that the experience of recollection is a determinant of hippocampal engagement during episodic memory retrieval, whether the episode is recent or remote (Aggleton & Brown, 2006; Cohn, Moscovitch, Lahat, & McAndrews, 2009; Eichenbaum, Yonelinas, & Ranganath, 2007; Moscovitch & Nadel, 1998; Moscovitch, et al., 2005; Nadel & Moscovitch, 1997; Yonelinas, Aly, Wang, & Koen, 2010). The current results indicate that perceptual episodic memory details, which are severely disrupted following damage to the MTL, could be a mediating factor in the relationship between hippocampal function and recollection (Piolino, Desgranges, & Eustache, 2009). Since the retrieval of perceptual details is supported by the hippocampus, and since these details provide a vivid sense of re-experiencing the past, one s inability to retrieve perceptual details due to hippocampal damage should lead to a specific deficit in recollection. The analysis of AM details reveals that the controls memories were more perceptual than those of individuals with mtle. Brewer s (1986, 1995) classification of personal memory takes into consideration different modes of representation, and it draws a distinction between memory that is imagery based ( recollective memory ), and memory that is propositional or conceptual ( autobiographical facts ). According to him, memories of specific events can sometimes be rehearsed and semanticized, and be accessed without experiencing imagery (Brewer, 1986, 1995). This theoretical distinction shares similarities with what Endel Tulving refers to as knowing and remembering, respectively (Tulving, 1985, 2002; Wheeler, et al., 1997). In the mtle group, the paucity of Perceptual AM details could indicate that a larger proportion of what they recalled about personal events is known rather than relived (see also Noulhiane, et al., 2007, 2008). With the current perceptual richness manipulation, I intended

65 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 53 for film clips to be remembered in an imagery-based manner, and for narratives to be remembered more conceptually. Both imagery-based and conceptual memory representations can contribute to a high Story details score, and so this paradigm could not quantify what proportion of the participants memory representation was conceptual. However, only imagerybased memory could contribute to high numbers of Perceptual details, which was quantified. The detail count indicates that the Clip and the Narrative conditions are not process-pure, since narratives elicited imagery in both the mtle and control groups. However, the Clip condition was significantly richer in imagery than the Narrative condition in controls. The fact that this was not the case in the mtle group suggests that the gist of their memory for clips was more readily available than its details (Brainerd & Reyna, 2001; Winocur & Moscovitch, 2011; Winocur, Moscovitch, & Bontempi, 2010), either due to an incapacity to encode, retain or retrieve context-specific perceptual details. Although mtle participants memories appeared more gist-like and less recollective than those of controls, the mtle group s memory performance did not benefit from studying stories in propositional form, as afforded by the Narrative condition. In other words, participants with mtle did not recall gist-like narratives better than they recalled perceptually rich Film Clips. One likely explanation for this finding is that controls benefitted from recollection when retrieving narratives, 1) by re-experiencing the narrative presentation, possibly in its exact wording, and 2) by relying on imagery to visualize the story, both at encoding and at retrieval. While I did not assess whether participants recall of narratives was literal, the Perceptual details count does indicate that controls engaged in more visualization than participants with mtle during narrative retrieval. In other words, my data suggest that memory representation was more abstract and schematic in participants with mtle than in controls in both the Clip and the Narrative condition. The current findings also provide support for Conway s theory of episodic memory (Conway, 2009; Conway & Loveday, 2010). According to his view, episodic memory is composed of experience-near episodic elements (EEs), which are often under the form of visual images. EEs are bundled together by a conceptual frame that provides the memory with its context and meaning, or gist. Conway suggests that the frame is supported by a fronto-temporal brain

66 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 54 network, while EEs are supported by a temporo-occipital-parietal network (Conway, 2009; Conway & Loveday, 2010). Damage to either the temporal lobe or to posterior cortical regions can lead to a dramatic loss of EE that reduces memory specificity: the frame is empty, and memory s gist is retrieved without re-experiential details (Conway & Loveday, 2010; Greenberg, Eacott, Brechin, & Rubin, 2005; Rubin & Greenberg, 1998). The Perceptual details of the memory stimuli, as defined in this chapter, are an example of EEs; as expected by Conway s model, the results show that these details are vulnerable to MTL damage. In the AM condition, it was also observed that story elements are more resilient to MTL damage than perceptual elements, and it is possible that some of these story details correspond to conceptual frames (see also St-Laurent, et al., 2009; St-Laurent, Moscovitch, et al., 2011). Conway s model also explains how EEs can be accessed through hierarchically organized conceptual knowledge that is linked semantically to their frame, or they can be cued directly, as when showing a detailed picture of an event. Recent evidence from an amnesic patient with extensive MTL damage indicates that some memories that cannot be retrieved with conceptual cues (e.g., do you remember going to the market two days ago? ) can be retrieved through direct cueing, for example using pictures (see also Browne, et al., 2011; Loveday & Conway, 2011). This finding is consistent with claims that the hippocampus plays an indexing role in the retrieval of EEs, rather than one of storage (Greenberg & Rubin, 2003; Teyler & DiScenna, 1986; Teyler & Rudy, 2007). In other words, perceptual details are stored in the cortex, but the hippocampus is essential to retrieve and integrate them into a multi-modal memory experience. Assuming that perceptual details were encoded successfully in the current mtle participants, it is plausible that details that could not be recalled may be accessible through more direct forms of cueing, as when performing recognition rather than recall Laterality and Surgical Status Performance on the current task was indistinguishable between participants with left and right mtle, indicating that both hemispheres contribute to the retrieval of complex episodic memories. This is consistent with other AM studies that did not observe significant differences in memory performance based on laterality of mtle (Herfurth, Kasper, Schwarz, Stefan, & Pauli, 2010; Lah, Grayson, Lee, & Miller, 2004; Lah, Lee, Grayson, & Miller, 2006;

67 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 55 Noulhiane, et al., 2007, 2008; St-Laurent, et al., 2009; St-Laurent, Moscovitch, et al., 2011; Viskontas, et al., 2000; but see Voltzenlogel, et al., 2006 for a greater deficit in patients with left-lateralized mtle, and see McAndrews, in press, for a review), and with functional brain imaging data indicating bilateral hippocampal activation during AM retrieval (see Svoboda, McKinnon, & Levine, 2006 for a review). For the current task, I anticipated that pathology localized in the language-dominant hemisphere may affect the language-based Narrative condition disproportionally, but no laterality effects were observed. Typically, damage to the left MTL disrupts memory for verbal material (Djordjevic, et al., 2010; Frisk & Milner, 1990; Helmstaedter, Grunwald, Lehnertz, Gleissner, & Elger, 1997; Rausch & Babb, 1993; Sass, et al., 1995), while damage to the right MTL interferes with memory for visual and spatial material (Bohbot, et al., 1998; Jones-Gotman & Milner, 1978; Morris, Abrahams, Baddeley, & Polkey, 1995; Morris, Pickering, Abrahams, & Feigenbaum, 1996; Smith & Milner, 1989; Spiers, Maguire, & Burgess, 2001). However, tasks that assess memory for associative material tend to be less lateralized based on material-specificity than tasks relying on familiarity with single test items (Cohn, McAndrews, & Moscovitch, 2009; Cohn, Moscovitch, et al., 2009; McCormick, Moscovitch, Protzner, Huber, & McAndrews, 2010; also see McAndrews & Cohn, In Press for a review). Thus, the complexity of the current laboratory events may account for the lack of laterality effect in the results. That being said, the current scoring system I used awarded points for Story details regardless of how they were worded, and it is possible that language-dominant mtle participants memories were less literal than those of non-language-dominant participants, as suggested based on findings from Djordjevic, et al. (2010), and Frisk and Milner (1990). A more stringent scoring system that is sensitive to wording, or a task of autobiographical fluency that is sensitive to language processes (Barnett, Newman, Richardson, Thompson, & Upton, 2000; Barr, Goldberg, Wasserstein, & Novelly, 1990; Lah, et al., 2004), may have revealed subtle modality-specific differences between the mtle groups. Nevertheless, the current data suggest that both hemispheres contributed importantly to performance on the current task, and that the episodic memory deficit observed in participants with mtle was not mediated by impaired

68 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 56 language processes, which is consistent with work from Lah and colleagues (2006; see also Race, Keane, & Verfaellie, 2011). My capacity to assess the influence of surgical status on performance was limited because the dominant and non-dominant mtle groups were not balanced, and surgical status and laterality were not orthogonal in the current data. I still conducted a preliminary analysis, which did not provide evidence for a difference in performance between pre- and post-surgery participants with mtle in any of the three memory conditions. In the literature, performance on verbal memory tasks has been shown to decline post-surgery in individuals with leftlateralized mtle, although the effect of a right temporal lobectomy on memory for non-verbal material is less clear-cut (Baxendale, Thompson, Harkness, & Duncan, 2006; Binder, et al., 2008; Harvey, et al., 2008; Rabin, et al., 2004; Richardson, et al., 2004). It is possible that participants with language-dominant mtle are more impaired on tasks of episodic memory than participants with non-language dominant mtle (Spiers, Burgess, et al., 2001), but that the smaller proportion of post-surgery cases in the former group masked this difference in my results. However, the pre-surgery language-dominant participants performed marginally worse than the post-surgery language-dominant participants on most of my measures, a trend opposite to this hypothesis (data not shown). Thus, I observed no evidence that surgical status affected performance in either the laboratory or the AM conditions. In the AM literature, others have also reported similar memory performance between pre- and post-surgery mtle groups (Addis, 2005; Herfurth, et al., 2010; St-Laurent, et al., 2009; Viskontas, et al., 2000; Voltzenlogel, Despres, Vignal, Kehrli, & Manning, 2007). Interestingly, a longitudinal fmri study assessing AM pre and post-surgically in a cohort of individuals with mtle reveals very little change in their whole-brain pattern of activation during AM retrieval following the surgery (Giannoylis, Lin, & McAndrews, 2011, May; McAndrews, In Press). This growing evidence suggests that AM may be so sensitive to disruption of normal MTL function that seizure-related damage is sufficient to disrupt activity within a network of brain regions that support AM, and that further removal of MTL tissue has little additional impact on AM performance. It also indicates that damage restricted to the hippocampus is sufficient to induce the kind of AM disruption observed in the current patient group (see also Gilboa, et al., 2006;

69 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 57 Rosenbaum, et al., 2008). Current theory of MTL function predicts that memory for single items can be supported by the cortex, but that memory that requires the formation of associations between items and context always involves the hippocampus proper (McAndrews & Cohn, In Press; Ranganath, 2010). It seems that the current laboratory tasks were so complex that, like AM but unlike episodic memory tasks that rely on simpler stimuli, damage limited to the hippocampus was sufficient to disrupt task performance, regardless of whether more extensive cortical lesions were also present Encoding versus Retrieval The results show that perceptual richness depends on the integrity of the MTL, but the current task could not be used to determine with certainty whether mtle interfered with memory encoding, with retrieval, or with both processes. The crucial role played by the MTL in memory acquisition is well documented (Eichenbaum, Otto, & Cohen, 1992; Scoville & Milner, 1957; Squire, 1992), and one must be mindful that participants with mtle encoded laboratory events with a defective MTL. With the exception of remote events such as childhood memories retrieved by individuals with late onset mtle, AM encoding was also performed with an epileptogenic MTL in the current mtle group. Importantly, the literature indicates that memory for personal events that precede the onset of seizures is just as impaired as memory for postonset events, and that age of onset is a poor predictor of AM performance in general (Bergin, Thompson, Baxendale, Fish, & Shorvon, 2000; Lah, et al., 2004; Noulhiane, et al., 2007; Viskontas, et al., 2000; Voltzenlogel, et al., 2006). Although perceptual memory content was not measured in these studies, qualitative losses of AM details were reported whether or not patients suffered from seizures at the time of encoding. Evidence of retrograde AM deficits observed in individuals with adult-onset damage to the MTL due to trauma or infections (Cipolotti, et al., 2001; Gilboa, et al., 2006; Kapur, 1999; Rosenbaum, et al., 2005; Rosenbaum, et al., 2008) clearly indicates that MTL damage interferes with the retrieval of episodic memory features. Together, these findings suggest that the presence of MTL damage at retrieval is sufficient to induce the kind of memory deficit observed on my task. Thus, although it is likely that mtle interfered to some extent with the encoding phase of my task, the literature suggests that mtle also interfered with the retrieval of

70 CHAPTER 2 mtle AND PERCEPTUAL RICHNESS OF COMPLEX MEMORIES 58 perceptual memory details. In the next chapter, I discuss results from healthy individuals tested on a version of this newly developed task adapted for fmri in order to test the hypothesis that hippocampal activation reflects the presence of perceptually rich episodic memory content during retrieval. 2.5 Conclusion I tested patients with mtle and healthy controls on a new memory paradigm that manipulated perceptual richness while controlling for other memory characteristics. The results indicate a general paucity of story details in the mtle group, as well as a disproportionate loss of perceptual richness. Self-ratings were not as sensitive to the mtle participants memory deficit as the objective scoring of individual memory elements. Participants with mtle localized in either their language dominant or their non-dominant hemisphere exhibited the same profile of impairment on the current task, consistent with evidence that both hemispheres contribute significantly to complex episodic memory retrieval. Preliminary evidence also indicates that surgical status did not affect performance in the mtle group, leading me to believe that damage restricted to the hippocampus is sufficient to cause the pattern of impairment observed in the mtle group. These findings support the hypothesis that perceptual richness is an important determinant of hippocampal function, and are in line with theories of hippocampal function that emphasize its role in the retrieval and assemblage of multi-modal memory elements into vivid recollective experiences, regardless of whether the memories were acquired in the laboratory or in the real world.

71 Chapter 3 The Neural Correlates of the Perceptual Richness of Memory for Complex Personal and Laboratory Episodes 3.1 Introduction The goal of the current study was to identify brain regions sensitive to the perceptual richness of episodic memory, and to determine whether the hippocampus is among them. As outlined earlier, one of the main hypotheses of this thesis is that the hippocampus is a hub in a network of brain regions that support the retrieval of perceptually rich memory episodes, and that perceptual richness is a major determinant of hippocampal activation during episodic memory retrieval. Reports of correlations between hippocampal activity and ratings of AM vividness (Gilboa, Winocur, Grady, Hevenor, & Moscovitch, 2004; Rabin, Gilboa, Stuss, Mar, & Rosenbaum, 2010), imagery content (Andrews-Hanna, Reidler, Sepulcre, Poulin, & Buckner, 2010) and reliving (St Jacques, Kragel & Rubin, In Press; but see Daselaar et al., 2008) support this hypothesis. With the current study, however, my goal was to test the relationship between perceptual memory content and hippocampal engagement with a paradigm that manipulated perceptual richness experimentally, rather than to rely on correlational evidence. For this purpose, I adapted the memory task introduced in Chapter 2 to functional magnetic resonance imaging (fmri), and I tested a group of healthy adults on this paradigm. Laboratory events provided the flexibility to manipulate perceptual richness, eliminating the risk of perceptual richness correlating with other memory dimensions, such as emotionality or personal relevance, as it sometimes does for naturalistic memories such as AMs (Daselaar, et al., 2008; Rubin, Schrauf, & Greenberg, 2003). Recent meta-analyses have shown that most laboratory tasks of episodic memory retrieval activate brain regions that share minimal overlap with those typically engaged by AM (Gilboa, 2004; McDermott, Szpunar, & Christ, 2009). However, many laboratory tasks are based on the recognition of simple items from study lists, rather than on the recall of complex memories, and greater levels of overlap are observed between the neural correlates of laboratory and autobiographical memory episodes with 59

72 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 60 paradigms that cue and test the two conditions in similar ways (Burianova & Grady, 2007; Burianova, McIntosh, & Grady, 2010; Cabeza, et al., 2004; Stokes, Mazuz, Daselaar, Moscovitch, & Cabeza, 2011, April). As such, I used laboratory memory events of the same type as those employed in Chapter 2, which were designed to mimic the complexity of personal episodes, for the current fmri study. To ensure that my personal and laboratory memory tasks engaged a common set of neural correlates, I performed a conjunction analysis comparing retrieval-related activation in all three tasks (AM, Narrative and Clip) to a non-mnemonic baseline condition. In this baseline condition, participants counted backward by intervals of three over a time period equivalent to the time provided for retrieval in the memory conditions, to prevent mind-wandering (Stark & Squire, 2001). Substantial overlap between the neural signatures of the different memory conditions would strengthen the claim that laboratory tasks can provide insight into the neural substrates that support different aspects of AM. Although I expected some of the canonical regions engaged during AM retrieval to be identified as parts of the conjunction analysis (Cabeza & St Jacques, 2007; Maguire, 2001a; Svoboda, McKinnon, & Levine, 2006), I also expected features that differed between AM and the laboratory conditions, such as personal relevance and emotionality (see Appendix G), to lead to differences in the neural signatures of my conditions, especially within cortical midline structures activated by self-referential processes (Andrews-Hanna, et al., 2010; Cabeza, et al., 2004; Cavanna & Trimble, 2006; Rabin, et al., 2010; Spreng & Grady, 2010; St Jacques, Conway, Lowder, & Cabeza, 2011; Vann, Aggleton, & Maguire, 2009). As mentioned before, the main goal of the current study was to investigate the role of the hippocampus in the retrieval of perceptually rich information. In the previous chapter, behavioural results from a group of healthy adults showed that perceptual content was significantly greater for the Film Clip than for the Narrative memory condition, but that both conditions were successfully matched for storyline content, or what happened during the event. As expected based on the literature (Brewer, 1995; Conway, 2009; Levine, Svoboda, Hay, Winocur, & Moscovitch, 2002; Rubin, et al., 2003; St-Laurent, Moscovitch, Levine, & McAndrews, 2009), perceptual content was also high for the autobiographical memory (AM)

73 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 61 condition. Based on these results, I felt confident that the AM and Clip conditions were perceptually richer than the Narrative condition, and that contrasting the blood oxygenation level dependent (BOLD) response between these conditions at retrieval would allow me to identify brain regions sensitive to perceptual richness. Because they were matched along multiple dimensions including recency, emotionality and personal relevance (see Appendix G), rehearsal, and story content, I considered the direct comparison between the two laboratory memory conditions, Clips and Narratives, to be the most appropriate contrast to identify the neural correlates of perceptual richness. Beside the hippocampus proper, I expected regions involved in perception and in sensory imagery to be among the neural correlates of perceptual richness of memory retrieval. Such regions included the lateral occipital cortex and the precuneus, which are activated by imagery and imagery-laden memory retrieval (Buchsbaum, Fang, Lemire-Rodger, & Young, 2011, April; Cavanna & Trimble, 2006; Daselaar, Porat, Huijbers, & Pennartz, 2010; Fletcher, et al., 1995; Johnson & Rugg, 2007; Kosslyn, 2005; Lehmann, Pascual-Marqui, Strik, & Koenig, 2010; Schulz, Woermann, & Ebner, 2007), the parahippocampal gyrus, which plays a role in the processing and remembering of spatial scenes (Epstein & Kanwisher, 1998; Epstein, 2008; Litman, Awipi, & Davachi, 2009; Stevens, Kahn, Wig, & Schacter, In Press), and other cortical regions that form the ventral visual stream, which provide an input of visual information into the hippocampus (Barense, Henson, & Graham, 2011; Eichenbaum & Lipton, 2008; Graham, Barense, & Lee, 2010; Suzuki, 2010). In addition to performing direct contrasts between all memory conditions, I also performed a parametric analysis correlating hippocampal activation with the perceptual content of episodic memory. Immediately after scanning, participants described, in their own words, the memory details they had time to retrieve for each memory while they were scanned. Recordings of this free recall were fed to LIWC2007 (Linguistic Inquiry and Word Count; Pennebaker, Chung, Ireland, Gonzales, & Booth, 2007), and a word count analysis was performed for each trial from each of the three memory conditions. I used a single LIWC word category, Perceptual Processes, to correlate with levels of hippocampal activation within each memory condition. In Chapter 2, I showed that this word category correlated highly with the

74 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 62 tallied number of Perceptual details identified through manual scoring, indicating that it is a sensitive measure of perceptual richness. LIWC is also performed much more efficiently than work-intensive manual scoring. I expected hippocampal activity to be correlated with Perceptual Processes words in all three memory conditions. Based on the literature and on my experimental manipulation, I formulated the following hypotheses: - Although key differences should be observed between all task conditions, the two laboratory conditions should elicit patterns of brain activation that overlap substantially with AM s neural correlates. - The retrieval of the two perceptually enriched conditions, AM and Film Clips, should lead to greater hippocampal activation than the retrieval of perceptually impoverished Narratives. Posterior medial, occipital and ventrotemporal regions, which are known to play a role in imagery and visual processing, should also be activated to a greater extent during AM and Clip retrieval than during Narrative retrieval. - I should observe a positive correlation between hippocampal activation and the LIWC measure of perceptual memory content in the parametric analysis. - Regions sensitive to personal relevance, among other features, should distinguish between AM and the two laboratory conditions. 3.2 Methods Participants Healthy adult participants were recruited through advertisement in the community, among staff from Toronto Western Hospital, and through word of mouth (friends and acquaintances). They were tested in accordance with a protocol approved by the Research Ethics Board of the University Health Network. 14 participants (5 male) aged between 21 and 58 years old (mean = 37.9) and with between 12 and 22 years of education (mean = 16.9) were tested on the current paradigm. One participant was left handed and had language localized in the right hemisphere. Adding this participant s data to the analysis did not change the pattern

75 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 63 of results, and so I elected to include her results in the final analysis. Participants were either native (n = 8) or fluent (n = 7) English speakers with no history of head injury, neurological or psychological disorder Procedure The behavioural task discussed in Chapter 2 was adapted for fmri. Twenty-one personal memories were collected with the participant over the phone prior to the scanning day. Memory titles that had been generated during that telephone interview were read to the participant on the scanning day to promote in-scan retrieval success. Immediately before scanning, participants studied 40 laboratory events (LEs), 20 of which were shown as film clips and 20 of which were shown as written narratives. These stimuli were of the same nature as those described in Chapter 2 (see Appendix C for a full list of LEs presented in the behavioural and fmri studies). The 40 LEs were assigned to either the Clip or the Narrative condition pseudo-randomly. Among the 40 film clips available as stimuli, a few pairs of clips featured redundant characters played by the same actors (e.g., the boy with his red balloon). Care was taken to ensure that when an LE from a redundant pair was assigned to the Clip condition, the other LE from that same pair was systematically shown as a narrative, so that participants were never shown two clips featuring the same actors. Also, one film clip s sound track was corrupted, so its corresponding story was presented as a Narrative to all participants. LEs were presented in the following sequence: blocks of three Narratives were presented in alternation with blocks of three Clips a total of six times, and the remaining two Narratives and two Clips were presented last (3N, 3C, 3N, 3C, 3N, 3C, 3N, 3C, 3N, 3C, 3N, 3C, 2N, 2C). The presentation of all 40 LEs lasted approximately 22 minutes. To maximize good inscan retrieval, participants were shown the entire sequence of LEs twice, consecutively. The order of the blocks was maintained between presentations, but LEs were shuffled within blocks (e.g., an LE assigned to the third block of Narratives could be the first in its block during the first viewing, and the 2 nd or 3 rd in its block for the second viewing). After the second study presentation, instructions were provided (see Appendix H for the full set of instructions), and participants completed a practice session to prepare them for in-

76 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 64 scan retrieval. Two additional LEs were shown for practice purpose (one Film Clip and one Narrative), and participants performed one practice trial for each of the conditions (AM, Narrative, Film Clip and control). During retrieval trials, a memory title was shown for 16s as a retrieval cue. AM titles were selected in advance with the participant, while LE titles were studied along with the Clip or Narrative at encoding (see Chapter 2 s methods section). During the control condition, participants were instructed to count backward for 16s, by intervals of three, from a given number. This non-memory task was added to the paradigm to serve as a baseline condition during which mind-wandering was prevented (Stark & Squire, 2001). Retrieval took place during scanning. The condition was announced on screen for 2s ( Autobiographical Memory, Laboratory Event, or Counting ), followed by two consecutive trials from that condition. LEs encoded as Narratives and Clips were intermixed randomly in the LE condition. Each trial started with a 1s fixation cross, followed by 16s of either counting or memory retrieval. Counting trials were immediately followed by a 1s inter-stimulus interval (ITI). Memory trials were followed by a self-rating of Story Content (4s), by a self-rating of Vividness (4s), and then by a 1s ITI. Rating instructions were identical to those given for the behavioural task (see Chapter 2), except that participants provided their answers on 1-4 Likert scales (rather than 1-5) with a 4 button MR-compatible response keypad. For the Story Content rating, 1 equaled no laboratory event / AM details, and 4 corresponded to all the laboratory event details / my most detailed AM (in the context of this task). For the Vividness rating, 1 equaled no visual / perceptual details, and 4 corresponded to my most vivid laboratory event / AM (in the context of this task). Participants completed five functional runs of 436s each (7 min. 16s). Each run contained four AM, four Counting and eight LE (Narrative and Clip intermixed) trials. A final retrieval session took place outside the scanner. Participants were shown the same AM and LE titles they saw inside the MR scanner, and they were required to provide additional information about their in-scan retrieval experience. The condition was announced on screen ( Autobiographical Memory or Laboratory Event ), followed by two consecutive trials from that condition. Memory was cued with the same titles that were shown inside the scanner. Participants first repeated Story Content ratings (on a 1-4 Likert scale) and then they

77 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 65 were recorded for up to 90 seconds while describing, in their own words, all the story details they had time to retrieve during the in-scan trial. Participants were not probed for perceptual details for the sake of brevity. To mitigate fatigue, there were designated breaks of at least 30 seconds between each of five retrieval blocks. Each block contained 4 AM and 8 LE trials. The AMs and LEs were assigned to blocks randomly, and the blocks order was also randomized. On average, participants took around 45 minutes to complete the post-scan retrieval session. The encoding, practice and post-scan retrieval sessions were presented on a Lenovo T500 laptop using E-Prime 2.0 (Psychology Software Tools Inc. Release candidate version ) using a setting identical to the one described in Chapter 2. In-scan retrieval trials were presented using E-Prime 1.2 (Psychology Software Tools Inc.) using scanner-compatible goggles worn by the participant. Participants who needed their vision adjusted had prescriptionappropriate corrective lenses inserted inside the goggles Image Acquisition All images were acquired with a 3-Tesla Signa MR System (GE Medical Systems) at the Toronto Western Hospital. Anatomical images (T1-weighted sequence; TR = 7.876ms; TE = 3.06ms; 146 slices, 220mm FOV, 256x256 matrix, x x 1.0 mm voxels) were acquired first, followed by five runs of functional scans. Functional data were acquired with an EPI sequence in an interleaved order (TR = 2 s; TE = 30ms; 32 slices for 11 participants, and 34 slices for 3 participants, 240 mm FOV, matrix, resulting in voxel sizes of ). Slices were acquired in an oblique orientation perpendicular to the long axis of the hippocampus. 218 frames were acquired for each of the five runs. The first three frames were dropped for signal equilibrium. A multi-echo T2-weighted sequence was acquired at the end of the session for the purpose of performing T2 relaxometry, but those results are not reported here Preprocessing I performed slice-timing, re-alignment (within and between runs), normalization to the Montreal Neurological Institute EPI template (4.0 x 4.0 x 4.0 mm voxels), and smoothing

78 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 66 (FWHM 8 mm) using SPM5 (Statistical Parametric Mapping 5; Welcome Department of Imaging Neuroscience). I used the Artifact Detection Tools software (Whitfield-Gabrieli & Mozes, 2010) to identify frames in which too much head motion was recorded (> 1.5mm or.015 radium), and to regress out these outlier frames during the first level analysis. Each trial was analyzed as a block of 5 TRs (10s) fitted with a box car function. I used short blocks because they provide greater signal detection power than event-related designs, and because I did not have a specific hypothesis about the shape of the hemodynamic response during the extended retrieval trials (Birn, Cox, & Bandettini, 2002; Liu, Frank, Wong, & Buxton, 2001). Based on piloting, AM retrieval could last the entire 16s allocated for retrieval, but the retrieval of LEs was completed within 13s, on average. Thus, I only analyzed the first 10s of retrieval time to avoid diluting the LE trials with dead waiting time Statistical Analysis For the memory conditions (AM, Narrative and Clip), only successful trials were entered into the fmri data analysis. Trials with in-scan Story Content ratings = 1 ( no LE / AM details ) were discarded. All direct contrasts between the different memory conditions were performed at a corrected threshold of p <.05 (False Discovery Rate FDR), with a cluster threshold of a minimum of five voxels. For each significant cluster, the coordinates of the voxel with the largest t value (peak voxel) are reported. When SPM listed more than one peak voxel per cluster (> 4mm from each other), additional voxels were reported only when they fell in a different Brodmann area from other voxels with larger t values in their own cluster. Multiple voxels were listed for the same Brodmann area when they belonged to separate clusters. I performed a conjunction analysis to identify brain regions activated during all three memory conditions, using an inclusive mask built with xjview.m ( The mask was composed of all the voxels activated above a p <.0005 threshold (uncorrected; clusters > 5 voxels) for each of the following three contrasts: AM > Counting, Narrative > Counting, and Clip > Counting. The p <.0005 threshold was more conservative than a.05 threshold (corrected; FDR), and was selected because xjview does not perform the FDR correction on multiple contrasts. This xjview mask was used as an

79 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 67 explicit mask in a second-level SPM analysis comparing all three memory conditions (AM, Narrative and Clip) to the control Counting condition (p <.05 corrected, FDR; cluster > 5 voxels). I used the xjview mask to exclude voxels for which significance was accounted for by one or two memory conditions, but fell below threshold in some conditions (Nichols, Brett, Andersson, Wager, & Poline, 2005). Importantly, the most stringent statistical test was performed during the formation of the mask, and the additional SPM contrast (Memory > Counting) was performed to identify peak voxels within clusters delineated by the mask. I also looked specifically at hippocampal activation for all the contrasts described here. I used a bilateral hippocampal mask created in MARINA (Bertram Walter Bender Institute of Neuroimaging, University of Giessen) to identify and count voxels falling within the anatomical mask s constraints, at the same threshold used for the whole brain contrasts (p <.05 corrected, FDR; clusters > 5 voxels). The mask was used for descriptive purposes, and no small volume correction was applied Parametric Analysis The Perceptual Processes word category was used as a parametric modulator of hippocampal activation, as delineated with a bilateral MARINA mask. Based on its high correlation with Perceptual details scored manually, I considered it a good indicator of perceptual memory content (see Chapter 2). Participants recordings from the post-scan retrieval session were transcribed into text files which were fed to the LIWC2007 word analysis software (Linguistic Inquiry and Word Count; Pennebaker, Chung, Ireland, Gonzales, & Booth, 2007). I performed a separate analysis for each of the three memory conditions, AM, Narrative and Clip, in SPM5. At the subject level, I entered the Perceptual Processes word count that corresponded to each trial from the condition of interest as a parametric modulator. Both successful and unsuccessful trials were entered into this analysis. I also modeled the other experimental conditions and the frames identified as head motion outliers. At the group level, I used the hippocampal MARINA mask as an explicit mask, and I adopted a threshold adjusted for small volume correction (p <.05 uncorrected, cluster > 4 voxels). Data from 12 participants were included in the parametric analysis. One recording was not available, and one participant

80 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 68 had too few successful trials to be included in a parametric analysis because two of her runs were spoiled by excessive motion. A similar parametric analysis was also conducted using mean ratings of Story Content and Vividness as modulators of hippocampal activity. A separate analysis was conducted for each rating for each memory condition, using data from all 14 participants. 3.3 Results Behavioural Results Ratings of Story Content were used to discard unsuccessful trials (rating = 1) from the fmri analysis. The mean number of trials retrieved successfully, and the mean ratings of Story Content and Vividness are plotted per memory condition in Figure 3.1. Because only trials for which Story Content was greater than 1 were included in the fmri analysis, only these trials ratings (2-4) were entered in the mean rating calculations, so that mean ratings best characterized the perceived story content and vividness of trials that were included in the fmri analysis. Participants number of retrieved trials was at ceiling (close to 20) in all conditions, perhaps because the LEs were presented twice at encoding and because the AM titles were also re-presented prior to scanning. One-way ANOVAs with repeated measures over memory condition revealed significant main effects of condition for both the Story Content and the Vividness ratings (F(2, 26) = 9.99, p <.005, and F(2, 26) = 15.29, p <.001, respectively). Series of paired sample t-tests revealed that Story Content ratings were significantly higher for the Clip than for the Narrative or the AM condition (t(13) = 3.40 and 4.21, respectively; p 0.005), while they did not differ significantly between the Narrative and the AM condition (t(13) = 0.88, p =.40). Vividness ratings differed significantly between each memory condition (p.05), and were highest for the Clip condition and lowest for the Narrative condition. These ratings show a different profile from what was observed in Chapter 2, possibly due to the more limited range of the current scale. Here, the AM condition was not rated higher than the other two conditions, and differences in rating between the Clip and the Narrative condition were larger for Vividness than for Story Content (as revealed by a 2x2 ANOVA with repeated measures over

81 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 69 ratings and condition: F(1, 13) = 10.33, p <.01). That being said, ratings were shown to be poor indicators of memory content in the behavioural analysis, and in the current study, their main purpose was to identify and discard unsuccessful trials from the fmri analysis. Figure 3.1 Mean number of trials retrieved successfully, and mean Story Content and Vividness in-scan ratings, per condition. AM = Autobiographical Memory; Narra = Narrative Story Condition; Clip = Film Clip Story Condition. Error bars represent standard error of the mean (SEM). Note: * p <.05, ** p <.01. The mean word counts from LIWC s Perceptual Processes category, which I used to investigate the modulation of hippocampal activation in the parametric analysis, are plotted in Figure 3.2. As in the behavioural study, I expected the number of Perceptual Processes words to be greater in the Clip and AM conditions than in the Narrative condition. In the parametric fmri analysis, both successful and unsuccessful trials (Story Content ratings 1-4, excluding corrupted recordings) were included, and so all trials were also included in the calculation of the word count means, to ensure the best correspondence between the fmri and the behavioural results. Given the low numbers of unsuccessful trials, this decision had little impact on this analysis. A one-way ANOVA with repeated measures over memory condition revealed a significant main effect of condition (F(2, 24) = 12.33, p <.001). As predicted, series of paired t- tests revealed significant differences between the Narrative and both the Clip and the AM condition (p <.05), but the difference between the AM and the Clip condition only showed a trend (t(12) = 1.95, p =.08).

82 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 70 I also plotted the mean number of verbs tallied by LIWC for each condition to provide an estimate of overall memory content. Although story content was shown to be equivalent between the Clip and the Narrative condition in the behavioural study, verb counts were slightly but significantly higher for the Clip condition in the behavioural study (see Appendix G), possibly because they were inflated by the description of perceptual content. Thus, I expected to observe a subtle difference in verb counts between the Clip and the Narrative condition that reflected their similar story content. A one-way ANOVA with repeated measures over memory category revealed a significant main effect of condition (F(2, 24) = 22.43, p <.001), and series of paired sample t-tests revealed that each memory condition differed significantly from the others (p <.005). The number of verbs was greatest for the AM condition, which is consistent with the greater number of Story details observed in that condition in the behavioural analysis. Of interest, the difference in verb counts between the Clip and the Narrative conditions was less prominent than their difference in Perceptual Processes words, although that interaction effect was not significant (F(1, 12) = 2.270, p =.158). Figure 3.2 Mean number of words per trial from LIWC2007 s Perceptual Processes and Verbs categories, plotted per memory condition (successful and unsuccessful trials). AM = Autobiographical Memory; Narra = Narrative Story Condition; Clip = Film Clip Story Condition. Error bars represent SEM. Note: * p <.05, ** p <.005.

83 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS Functional MRI Results Common memory regions. In order to demonstrate that the two laboratory tasks recruited similar brain regions from those engaged by the retrieval of naturalistic personal memories, I first report results from the conjunction analysis. Figure 3.3 displays the brain regions activated significantly by all three memory conditions, in comparison to the Counting condition. An explicit mask was used to constrain the voxels identified by this contrast to those activated above threshold in each of the three memory tasks. Regions activated significantly by all three memory conditions are listed in Table 3.1, and included the posterior cingulate cortex, the angular portion of the inferior parietal lobule, the left temporal pole, medial temporal regions such as the bilateral hippocampus and parahippocampal cortex, the left lingual gyrus, the caudate nucleus, and the inferior, middle and superior frontal gyri. Regions activated by the opposite contrast, Counting > Memory conditions, are also listed in this table. They were identified without the use of a conjunction mask (p < 0.05 corrected, FDR), and included frontoparietal regions and the right insula.

84 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 72 Table 3.1 Brain Regions Showing Differences in Activation Between the Counting and the Three Memory Conditions MNI coordinates Region Hemi BA t value x y z Memory > Counting Posterior Cingulate Cortex / Precuneus L BA 31 / R BA 31 / Retrosplenial Cortex L BA 30 / Inferior Parietal Lobule (AG) R BA 22 / L BA Parahippocampal Gyrus R BA 35 / Hippocampus R n/a Hippocampus / Rhinal Cortex L BA Fusiform Gyrus L BA L BA R BA Temporal Pole L BA Middle Temporal Gyrus L BA R BA L BA Medial Frontal Gyrus L & R BA L BA R BA Middle Frontal Gyrus L BA Inferior Frontal Gyrus R BA L BA Gyrus Rectus L & R BA Lingual Gyrus L BA Caudate R n/a L n/a Counting > Memory Inferior Parietal Lobule (SG) R BA L BA Precentral Gyrus L BA Inferior Frontal Gyrus R BA Insula R BA Note. All activations are significant at p <.05, corrected (FDR; cluster threshold > 5 voxels). Additionally, for the Memory > Counting contrast, I used an explicit mask made of all the voxels activated above a p <.0005 threshold for the following three contrasts: AM > Counting, Narrative > Counting, and Clip > Counting. The coordinates (MNI space) and t value, and the corresponding BA of the voxel with the highest t value ("peak voxel") within a cluster are provided. AG = Angular Gyrus; AM = Autobiographical Memory, BA = Brodmann Area, Hemi = Hemisphere, L = Left, MNI = Montreal Neurological Institute, R = Right, SG = Supramarginal Gyrus.

85 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 73 Figure 3.3 Contrast between all three memory conditions (AM, Narrative and Clip) and the control Counting condition (p <.05 corrected, FDR; clusters > 5 voxels). Results are masked with an explicit mask composed of all the voxels activated above a p <.0005 threshold (uncorrected; clusters > 5 voxels), for each of the following three contrasts: AM > Counting, Narrative > Counting, and Clip > Counting. Voxels in warm colors showed greater activation for the memory than for the control condition. This analysis was unidirectional, but see Table 3.1 for a list of regions showing significantly greater activation for the Counting than for the memory conditions Contrasts among memory conditions. I performed direct pairwise comparisons amongst the three memory conditions to assess how their different characteristics lead to different patterns of brain activation. I first contrasted brain activation between the Clip and the Narrative conditions (see Figure 3.4, top portion.) in order to identify brain regions sensitive to perceptual richness, and to determine whether the hippocampus was among them. No region showed significantly greater activity for the Narrative condition at the corrected threshold (FDR). Regions whose activity was greater during the perceptually enriched Clip condition were mainly right-lateralized, and included the right hippocampus and the right lateral temporal cortex. Regions such as the fusiform gyrus, the parahippocampal gyrus and the inferior parietal lobule expressed this contrast bilaterally, although significance was more prominent in the right hemisphere. Midline regions such as the posterior cingulate cortex and the precuneus were also identified by this contrast (see Table 3.2).

86 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 74 Table 3.2 Brain Regions Showing Differences in Activation Between the Clip and the Narrative Memory Conditions MNI coordinates Region Hemi BA t value x y z Clip > Narrative Retrosplenial Cortex R BA 31 / L BA 31 / R BA 23 / Precuneus R BA Inferior Parietal Lobule (SG) R BA L BA Inferior Parietal Lobule (AG) R BA Parahippocampal Gyrus L BA Amygdala / Hippocampus R n/a Fusiform Gyrus R BA L BA Superior Temporal Gyrus R BA Middle Temporal Gyrus R BA Medial Frontal Gyrus L BA R BA 9 / Superior Frontal Gyrus R BA L BA R BA Inferior Frontal Gyrus R BA Postcentral Gyrus L BA 2 / Superior Occipital Gyrus R BA Note. All activations are significant at p <.05 (corrected, FDR) with a cluster threshold of > 5 voxels. The coordinates (MNI space) and t value, and the corresponding BA of the voxel with the highest t value ("peak voxel") within a cluster are provided. AG = Angular Gyrus; AM = Autobiographical Memory, BA = Brodmann Area, Hemi = Hemisphere, L = Left, MNI = Montreal Neurological Institute, R = Right, SG = Supramarginal Gyrus.

87 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 75 Figure 3.4 Contrast between the Clip and the Narrative memory condition (top), the AM and the Narrative condition (middle), and the AM and the Clip condition (bottom; p <.05 corrected, FDR; cluster threshold > 5 voxels). Top: Clip > Narrative in warm colors. No significant voxels were identified for the opposite contrast. Middle: AM > Narrative in warm colours; Narrative > AM in cold colours. Bottom: AM > Clip in warm colours; Clip > AM in cold colours.

88 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 76 I then performed a direct comparison between the Narrative and the AM condition (see Figure 3.4, middle). As revealed by the total verb count, AM had a greater level of overall memory details than memory for Narratives. It was also perceptually richer, as well as more remote, rehearsed, and personally relevant than the Narrative condition, and I expected these differences to distinguish the two conditions at the neural level. Regions whose activity was greater for the Narrative than for the AM condition included the middle and right inferior frontal gyrus, the precuneus, and the supramarginal portion of the inferior parietal lobule. Note that the peak precuneus voxel was posterior to the region identified by the Clip > Narrative contrast. Regions activated to a greater extent by the AM than by the Narrative condition included the medial prefrontal cortex, the posterior cingulate region, the bilateral hippocampus, fusiform and parahippocampal cortex, the superior occipital gyrus, the lateral temporal cortex and the left temporal pole (see Table 3.3). With the notable exception of widespread medial prefrontal activation, many of these regions were similar to those showing greater activation for the Film Clip than for the Narrative condition. Finally, I compared the AM and the Film Clip conditions to each other (see Figure 3.4, bottom). AM had a greater level of overall memory details than memory for Clips, and it was also more remote, more rehearsed and more personally significant than the Clip condition. However, both conditions were rich in perceptual detail. Regions activated to a greater extent in the AM than in the Film Clip condition included midline regions such as the medial prefrontal and posterior cingulate cortex, the left superior occipital and left middle temporal gyrus, and the left parahippocampal gyrus, but not the hippocampus proper. Regions activated by the Film Clip condition to a greater extent than by the AM condition included the middle frontal gyrus, the middle cingulate cortex, the supramarginal portion of the inferior parietal lobule, the right lateral temporal cortex, the precuneus, and the posterior portion of the left fusiform gyrus. Overall, I observed minimal differences in activation in the medial temporal lobe, and none in the hippocampus proper between these two conditions. The AM condition activated the medial prefrontal and the retrosplenial cortex to a greater extent than both LE conditions, while both

89 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 77 the Clip and the Narrative condition recruited lateral prefrontal regions and lateral parietal regions to a greater extent than the AM condition. Table 3.3 Brain Regions Showing Differences in Activation Between the Autobiographical and the Narrative Memory Conditions MNI coordinates Region Hemi BA t value x y z AM > Narrative Retrosplenial Cortex L BA 30 / R BA 30 / Posterior Cingulate Cortex L & R BA Parahippocampal Gyrus R BA 35 / L BA 35 / Amygdala / Rhinal Cortex L BA Fusiform Gyrus R BA Middle Temporal Gyrus L BA Inferior Temporal Gyrus R BA Temporal Pole L BA Medial Frontal Gyrus R BA L BA L & R BA 9 / Inferior Frontal Gyrus L BA Superior Occipital Gyrus / IPL (AG) R BA 19 / Superior Occipital Gyrus L BA Narrative > AM Precuneus L BA R BA Inferior Parietal Lobule (SG) R BA L BA Middle Frontal Gyrus R BA R BA 46 / L BA L BA Inferior Frontal Gyrus R BA Middle Cingulate Cortex L BA Note. All activations are significant at p <.05 (corrected, FDR) with a cluster threshold of > 5 voxels. The coordinates (MNI space) and t value, and the corresponding BA of the voxel with the highest t value ("peak voxel") within a cluster are provided. AG = Angular Gyrus; AM = Autobiographical Memory, BA = Brodmann Area, Hemi = Hemisphere, IPL = Inferior Parietal Lobule, L = Left, MNI = Montreal Neurological Institute, R = Right, SG = Supramarginal Gyrus.

90 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 78 Table 3.4 Brain Regions Showing Differences in Activation Between the Autobiographical and the Clip Memory Conditions MNI coordinates Region Hemi BA t value x y z AM > Clip Retrosplenial Cortex L BA 29 / R BA 29 / Parahippocampal Gyrus L BA Middle Temporal Gyrus L BA Medial Frontal Gyrus R BA L BA Superior Frontal Gyrus L BA Middle Frontal Gyrus L BA Gyrus Rectus L & R BA Superior Occipital Gyrus L BA Clip > AM Inferior Parietal Lobule (SG) R BA L BA Precuneus R BA Superior Temporal Gyrus R BA R BA Middle Temporal Gyrus R BA L BA R BA Inferior Temporal Gyrus R BA Fusiform Gyrus L BA Middle Frontal Gyrus R BA R BA L BA L BA Inferior / Middle Frontal Gyrus L BA 46 / Medial Frontal Gyrus R BA 6 / Middle Cingulate Gyrus L BA Insula R BA Thalamus L n/a Note. All activations are significant at p <.05 (corrected, FDR) with a cluster threshold of > 5 voxels. The coordinates (MNI space) and t value, and the corresponding BA of the voxel with the highest t value ("peak voxel") within a cluster are provided. AM = Autobiographical Memory, BA = Brodmann Area, Hemi = Hemisphere, L = Left, MNI = Montreal Neurological Institute, R = Right, SG = Supramarginal Gyrus Differences in hippocampal activation between memory conditions. In order to assess how hippocampal activation differed amongst conditions, I applied a MARINA hippocampal mask to whole-brain contrasts between the task conditions. The mask was used

91 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 79 strictly for the purpose of counting significant hippocampal voxels, and to identify a peak hippocampal voxel. I used the same statistical thresholds as for the contrasts reported above (see Figures 3.3 and 3.4; p <.05 corrected, FDR) without applying a small volume correction. The number of significant voxels identified in each contrast, and the coordinates of the peak hippocampal voxels identified with this analysis are listed in Table 3.5. Figure 3.5 Medial temporal voxels showing significantly different levels of activation between the experimental conditions. Thresholding is identical to what was used in Figures 3.3 and 3.4, respectively (no hippocampal mask was applied to the images). No significant voxel was identified in the hippocampus for the Narrative > Clip, Narrative > AM, Clip > AM, AM > Clip or Counting > Memory contrasts. AM = Autobiographical Memory. As shown in Figure 3.5, the three memory conditions activated both hippocampi to a greater extent than the counting condition. Both perceptually enriched conditions (AM and Film Clip) activated the right hippocampus to a greater extent than the perceptually impoverished Narrative condition, revealing that the right hippocampus is sensitive to episodic memory s perceptual content. The AM condition also elicited greater levels of left hippocampal activation than the Narrative condition.

92 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 80 Table 3.5 Coordinates of Hippocampal Activation Differentiating Between the Experimental Conditions MNI coordinates Contrast # voxels t value x y z Memo > Counting Right Hippocampus Left Hippocampus Clip > Narrative Right Hippocampus AM > Narrative Right Hippocampus Left Hippocampus Note. I used a bilateral hippocampal mask to identify significant voxels from each of the whole brain analyses (p <.05 corrected, FDR; clusters > 5 voxels) that fell within the hippocampus. The coordinates (MNI space) and t value of the hippocampal voxel with the largest t value are listed here. The total number of significant voxels identified in each hippocampus is also listed. No significant voxels were identified for the AM > Clip, Clip > AM, Narra > Clip, Narra > AM and Counting > Memo contrasts. AM = Autobiographical Memory, MNI = Montreal Neurological Institute Parametric ROI analysis. In order to assess whether hippocampal activation was modulated by perceptual episodic memory content, I performed a parametric analysis that was restricted to bilateral hippocampal voxels delineated by the MARINA mask. Hippocampal activity was correlated with the number of Perceptual Processes words counted by LIWC2007 within each condition (see Figure 3.6). In the Film Clip condition, I identified a cluster of 44 voxels in the right hippocampus whose activity correlated positively with Perceptual Processes words (p <.05, uncorrected small volume correction; cluster > 5 voxels; peak voxel: t = 3.17; x = 32, y = -20, z = -20, MNI Coordinates). No significant voxels were identified in the AM or the Narrative conditions. In the behavioural study, LIWC was shown to underestimate perceptual memory content in the AM condition in comparison to manual scoring. It is also likely that the sensitivity of LIWC was reduced by the absence of probing for perceptual memory content,

93 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 81 which may have contributed to the current negative result in the AM and the Narrative condition. Correlations were also conducted between ratings of Story Content and Vividness, and hippocampal activity within each memory condition, but none of these analyses reached significance. Figure 3.6 Hippocampal voxels (as delineated with a bilateral MARINA mask) whose activity was modulated parametrically by the number of Perceptual Processes words in each memory condition. The threshold was adjusted for small volume correction (p <.05, cluster size > 5 voxels). AM = Autobiographical Memory. 3.4 Discussion Summary of Findings I used a paradigm with which I manipulated perceptual richness while keeping other memory features constant in order to determine whether the hippocampus was among the brain regions sensitive to this dimension of memory retrieval. Healthy adult volunteers encoded and recalled laboratory events designed to capture the complexity of personal episodes, in addition to actual memories from their own life, and I showed that both laboratory and real-life events engaged similar sets of brain regions at retrieval. This finding gives credence to my claim that knowledge gained from laboratory events can apply to naturalistic memories. By contrasting brain activity during the retrieval of perceptually enriched Clips and perceptually

94 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 82 impoverished Narratives, I observed that the hippocampus was in fact sensitive to the perceptual richness of episodic memory. This effect was strongly lateralized to the right hemisphere, a finding that was not originally anticipated based on behavioural evidence that damage lateralized to either the left or the right MTL leads to an equal paucity of perceptual episodic memory details (see Chapter 2). While both hippocampi showed greater levels of activation in the AM than in the Narrative condition, no difference in activation was observed in the MTL between the Clip and the AM condition, which were both perceptually enriched. Additional evidence of lateralization of hippocampal function was provided by the parametric analysis. In memory for Film Clips, I identified a cluster of voxels in the right hippocampus whose activity correlated positively with perceptual memory content. No such correlation was observed in the AM and the Narrative conditions, possibly due to limitations with the LIWC measure of perceptual richness. Beside the hippocampus, other neural correlates of perceptual richness were also disproportionally right-lateralized, and included regions from the occipital and parietal cortex, the medial temporal cortex, and the ventral visual stream. These results demonstrate that perceptually enriched episodic memories recruit a collection of regions known to partake in recollection and visual processing, with the right hippocampus clearly among them. Of interest, a clear pattern of hemispheric hippocampal specialization has not emerged from the autobiographical memory retrieval literature, and bilateral activation is frequently reported (Cabeza & St Jacques, 2007; Svoboda, et al., 2006; but see Maguire, 2001 for evidence of lateralization to the left hemisphere). The current study contribute to this literature by providing clear evidence, based on well-controlled laboratory events, that the right MTL contributes disproportionally to the retrieval of rich, multimodal perceptual memory features Neural Correlates of Memory Retrieval The conjunction analysis identified a widely distributed set of regions activated across all memory conditions, which confirms that my task s laboratory events engaged several of the canonical brain regions that support AM retrieval. Others have reported that cueing and testing AMs and laboratory memories following similar procedures reduces discrepancies between the two conditions neural signatures (Burianova & Grady, 2007; Burianova, et al.,

95 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS ; Cabeza, et al., 2004; Stokes, et al., 2011, April). In the literature, most laboratory tasks of episodic memory are based on recognition of simple items. Such tasks can engage the MTL successfully (Spaniol, et al., 2009), especially if they are associative and elicit recollection (Ranganath, 2010), however, their overlap with other brain regions that form the AM retrieval network is limited (Gilboa, 2004; McDermott, et al., 2009; Stokes, et al., 2011, April). On the other hand, the current laboratory memory task required the extended recall of complex, sequential stories, an experience more similar to the performance of most AM retrieval tasks. My results have implications for future studies of episodic memory retrieval, which may benefit from adopting story recall procedures for greater ecological validity. Among the regions identified by the conjunction analysis were the left temporal pole and lateral temporal cortex, the angular portion of the inferior parietal lobule, the posterior cingulate cortex, the caudate nucleus, the left lingual gyrus, the inferior, middle and superior frontal gyri, and the bilateral medial temporal lobe, including the hippocampus proper. Some of these regions, such as the inferior and middle frontal gyrus (left > right), the left hippocampus, the left lingual gyrus and the right caudate nucleus, were identified by Burianova and colleagues (Burianova & Grady, 2007; Burianova, et al., 2010) as parts of a common declarative memory network engaged during the retrieval of autobiographical, semantic and laboratory episodic memory. Also, the left ventrolateral prefrontal cortex is frequently activated during AM retrieval, and is thought to play a role in the selection, verification and online maintenance of information during strategic retrieval (see Svoboda et al., 2006 for a review). The left middle temporal cortex and temporal pole are known to support semantic memory (Martin & Chao, 2001; Patterson, Nestor, & Rogers, 2007). Finally, the inferior parietal lobule is thought to be involved in the retrieval of personal and laboratory episodes (Burianova & Grady, 2007; Cabeza, Ciaramelli, Olson, & Moscovitch, 2008; Vilberg & Rugg, 2008). Activation among these regions could reflect the engagement of general declarative memory retrieval processes, although differences in their levels of activation between the memory conditions also reflected qualitative differences between them in terms of imagery content, emotionality, personal relevance, and attentional processes, among other dimensions.

96 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS Neural Correlates of Perceptual Richness As mentioned, the contrast between the Clip and the Narrative conditions lead to the identification of neural regions that were also sensitive to perceptual memory content, beside the hippocampus proper. Such regions included the fusiform and parahippocampal cortex (right > left), the retrosplenial cortex and anterior portion of the precuneus, the inferior parietal lobule (right > left), and the right lateral occipital region (Brodmann area 19). These regions were also activated to a greater extent in the AM than in the Narrative condition. A rich literature links these different regions to imagery and perceptual processes. For example, the ventral temporal cortex provides an influx of highly processed visual information to the hippocampus proper (Amaral & Lavenex, 2007; Ettlinger, 1990; Goodale & Milner, 1992). Also, the parahippocampal gyrus supports the processing of complex scenes (Epstein & Kanwisher, 1998; Epstein, Parker, & Feiler, 2007; Litman, et al., 2009; Stevens, et al., In Press), while more anterior regions such as the perirhinal cortex support the visual processing of single objects (Dickerson & Eichenbaum, 2010; Graham, et al., 2010), and portions of the fusiform gyrus have been shown to process faces (Kanwisher, McDermott, & Chun, 1997). The precuneus ( the mind s eye ; Fletcher, et al., 1995) plays a role in visual-spatial imagery and proprioceptive visualization (Cavanna & Trimble, 2006), while the retrosplenial cortex is involved in spatial navigation and path integration, and is engaged during self-projection (Buckner, Andrews- Hanna, & Schacter, 2008; Buckner & Carroll, 2007; Vann, et al., 2009). The posterior portion of the inferior parietal lobule (angular gyrus) plays an important role in memory retrieval, and has been suggested to be involved in recollection (Cabeza, et al., 2008; Cabeza, et al., 2011) and in the retrieval of detailed memories (Vilberg & Rugg, 2008). Finally, the lateral occipital cortex is involved in visual processing and is activated by imagery and vivid retrieval (Buchsbaum, et al., 2011, April; Daselaar, et al., 2010; Grill-Spector, Kourtzi, & Kanwisher, 2001; Huijbers, Pennartz, Rubin, & Daselaar, 2011; Lehmann, et al., 2010; Malach, et al., 1995). As such, it is fair to assume that the activation of these different regions reflected the retrieval of imagery-laden memories. Moreover, the fact that these perceptual regions were identified along with the right hippocampus by the contrast between Clip and Narrative retrieval gives credence to the claim that right hippocampal activation is part of the neural signature of perceptual richness.

97 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS Parametric Analysis In addition to contrasting conditions that differed in perceptual richness, I assessed whether hippocampal activation was modulated parametrically by a measure of perceptual memory content obtained during a post-scan retrieval interview. Because some imagery was experienced even in the Narrative condition (see Chapter 2), I expected Perceptual Processes words to correlate with hippocampal activation in the three memory conditions. In the Film Clip condition, right, but not left, hippocampal activation was greater during the retrieval of clips for which higher numbers of Perceptual Processes words were produced post-scan. This finding is consistent with the observation that the right hippocampus is sensitive to perceptual richness, and it supports the main hypothesis that perceptual richness is a determinant of hippocampal engagement at retrieval. In the Narrative condition, hippocampal activation did not correlate with perceptual content. Lower numbers of Perceptual Processes words in that condition, approximately half of what was observed in the Clip condition, may not have provided a sufficient range of scores to observe significant correlations. Contrary to my expectations, the number of Perceptual Processes words did not correlate with hippocampal activation in the AM condition either. In the AM retrieval literature, others have shown that hippocampal activation increases with subjective ratings of imagery levels (Andrews-Hanna, et al., 2010; Viard, et al., 2007), vividness (Gilboa, et al., 2004), reliving (St Jacques, Rubin, & Cabeza, In Press, but see Daselaar, et al., 2008), and level of details (Addis, Moscovitch, Crawley, & McAndrews, 2004), many of which are imagery-based (Brewer, 1995; Conway, 2009; Rubin, et al., 2003). It is possible, however, that the current LIWC measure of perceptual content lacked sensitivity. In the behavioural study, participants were probed to recall perceptual memory content ( what did you see in your mind s eye ) as well as story content ( what happened ). In the current study, however, I only probed participants for story content, in the interest of saving time and to limit participant burden, because the full session already lasted 4 hours. It is likely that perceptual memory content was underestimated by LIWC, and that the count of Perceptual Processes words would have been a more accurate depiction of memory experience if perceptual content had been probed. This limitation would

98 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 86 also affect the manual scoring of Perceptual details, which correlated highly with Perceptual Processes words in the behavioural study. In addition, results from the behavioural study indicated that, in comparison to the manual scoring of Perceptual details, the LIWC count of Perceptual Processes words underestimated perceptual memory content in the AM condition relative to the other two conditions. Anecdotally, participants seemed to use Perceptual Processes words more spontaneously to describe their recollection in the Clip condition (i. e., I see a boy with a huge pack back; I see two bullies pushing him down ) than in the AM condition, in which they tended to adopt more of a story-telling style (i. e., I was walking my dog outside, rather than I can see myself walking my dog ). This difference in wording may explain why the parametric analysis was significant in the Clip, but not the AM condition. Lastly, memories for Clips were more homogeneous and less noisy than AMs, which varied along dimension that were neither measured nor controlled for, such as recency, personal relevance, and emotionality. This higher level of control over memory content in the Clip condition may have enhanced the correlation between perceptual content and hippocampal activation, by reducing the influence of variables of no interest on hippocampal activation. No significant correlations were observed between hippocampal activity and ratings of either Story Content or Vividness in any of the three memory conditions. These negative results are at odds with the literature, but they may be explained by the limited range of my scales. Given the very low number of unsuccessful trials, most ratings ranged between two and four on the scales. This range is rather limited compared to, for example, the five-point scale used by Addis and colleagues (Addis, et al., 2004), the eight-point scale used by St Jacques et al. (In Press), or the 10 cm analogical scale used by Viard et al. (2007). Of interest, both St Jacques and Viard s groups showed that right hippocampal activity during AM retrieval was predicted by ratings of reliving and imagery quality, respectively. Given my observation that the right hippocampus is engaged differentially by memory tasks that differ in perceptual content, and given findings by other groups, it is quite likely that the current lack of positive correlations between right hippocampal activation and either Vividness ratings or LIWC counts of Perceptual Processes words was due to the limited sensitivity of these two measures. Information that is

99 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 87 available to mind may not always be captured by behavioural measures, but it will always be reflected in patterns of brain activity. In such cases, contrasts between conditions known to vary along certain dimensions, such as perceptual richness, will reveal neural correlates that may not be captured by parametric analyses Lateralization Both the contrasts between the memory conditions and the parametric analyses indicate that regions sensitive to perceptual richness, including the hippocampus, are rightlateralized. These results are somewhat surprising based on the autobiographical memory literature and on my own behavioural results, which show that both the left and the right MTL are important to memory retrieval. In the mtle group tested in my behavioural study, pathology or surgery to either hemisphere led to an identical reduction of story and perceptual details, and there were no trends for group differences. Early studies of AM retrieval revealed left-lateralized profiles of activation (Maguire, 2001a, 2001b; Spiers, Burgess, et al., 2001), and more recent studies have reported bilateral activation, especially when more time is provided to retrieve memories, or when the retrieval experience is more emotional (Cabeza & St Jacques, 2007; Svoboda, et al., 2006). While the AM literature indicates that both hemispheres contribute to retrieval, the current lateralized findings are consistent with an extensive neuropsychology literature pointing to hemispheric specialization in the temporal lobe. The left hippocampus is known to support memory for verbal and abstract conceptual material (Djordjevic, et al., 2010; Frisk & Milner, 1990; Helmstaedter, Grunwald, Lehnertz, Gleissner, & Elger, 1997; Rausch & Babb, 1993; Sass, et al., 1995). Also, the left middle temporal gyrus and temporal pole seem to play an important role in semantic memory (Lambon Ralph, Cipolotti, Manes, & Patterson, 2010; Patterson, et al., 2007). On the other hand, the right hemisphere supports visuo-spatial representations, memory for visual stimuli with complex configurations such as faces, and social processing (Bohbot, et al., 1998; Jones-Gotman & Milner, 1978; Morris, Pickering, Abrahams, & Feigenbaum, 1996; Moscovitch & McAndrews, 2002; Nestor, Plaut, & Behrmann, 2011; Olson, Plotzker, & Ezzyat, 2007; Smith & Milner, 1989; Spiers, Maguire, & Burgess, 2001). That being said, laboratory tasks that assess memory for associative material tend to be less

100 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 88 lateralized than tasks relying on familiarity with single test items (Cohn, McAndrews, & Moscovitch, 2009; Cohn, Moscovitch, et al., 2009; McCormick, Moscovitch, Protzner, Huber, & McAndrews, 2010; also see McAndrews & Cohn, In Press for a review). Because AM is a complex mixture of personal, emotional, social, semantic and multi-modal re-experiential information, it is likely to engage both hemispheres. However, the current results show that each hippocampus may contribute disproportionally to different memory characteristics, and that the right hemisphere supports the retrieval of perceptually rich memory content disproportionally in healthy brains. Based on evidence from patients with mtle in a virtual-reality task (Spiers, Burgess, et al., 2001), Maguire (2001b) suggested that the left hippocampus might be engaged by the retrieval of contextual details (who did what when), and that the right hippocampus might support spatial memory and navigation. While evidence from neuropsychological cases has suggested that pure imagery deficits with intact perception are most commonly observed following damage to the left hemisphere (Farah, 1995; Stangalino, Semenza, & Mondini, 1995), Kosslyn and colleagues have demonstrated that the left hemisphere supports representations based on semantics and categorical distinctions, and that imagery that requires specific, faithful perceptual representations (e.g., specific spatial coordinates) requires the contribution of the right hemisphere (Kosslyn, et al., 1989; Kosslyn, Maljkovic, Hamilton, Horwitz, & Thompson, 1995; Kosslyn, Thompson, Sukel, & Alpert, 2005). Recent evidence from Stevens and colleagues (Stevens, et al., In Press) also shows interesting hemispheric differences in the parahippocampal cortex, a brain region that provides a significant input into the hippocampus proper. They observed that the right parahippocampal gyrus only showed repetition suppression to the exact same image of a scene, while the left parahippocampal cortex showed repetition suppression to either the same or conceptually similar scenes. In other words, the right MTL seemed to support form-specific visual processing, while the left supported formgeneral visual processing. Additionally, Stevens et al. (In Press) showed greater functional connectivity between the parahippocampal gyrus and posterior perceptual regions in the right hemisphere (e.g., middle occipital gyrus), and greater connectivity between the parahippocampal gyrus and regions involved in abstract and conceptual processes, including

101 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 89 the inferior frontal gyrus, in the left hemisphere. All these findings are consistent with my observation that perceptual richness for episodic memory relies disproportionally on the right MTL. Interestingly, evidence from electroencephalography (EEG) during AM retrieval suggests a shift from left to right hemispheric engagement within the time course of AM remembering (Conway, Pleydell-Pearce, & Whitecross, 2001; Conway, Pleydell-Pearce, Whitecross, & Sharpe, 2003). In other words, activity seems to shift from an early left anterior component (e.g., inferior frontal gyrus and temporal pole performing search and retrieval processes) to a late right-lateralized posterior component (e.g., temporo-occipital cortex supporting sensoryperceptual episodic knowledge). The authors could not measure subcortical activation with EEG, but it is possible that this switch from the left to the right hemisphere also takes place in the hippocampus during retrieval, especially if memories are cued conceptually, for example with a title. The left hippocampus may play a more dominant role during AM retrieval, searching and establishing the story. It may also serve as an essential node that supports the functional recruitment of the right hippocampus in the AM network. Once recruited, the right hippocampus may then interact with posterior cortical regions that store experiential details. This explanation could account for my observation that performance did not differ between participants with left and right mtle on the behavioural task, even though fmri indicates that perceptual richness is mainly supported by right hemisphere regions that include the MTL. For now, little evidence can be found to address this hypothesis in the literature. If the left hippocampus is indeed a bottleneck in the recruitment of the right hippocampus during AM retrieval, one might expect hippocampal activation to occur sooner in the left than in the right hemisphere. A few recent fmri studies have split AM retrieval trials into an early, construction phase, and a later, elaboration phase using a button press to signal when a memory is accessed (Addis, Wong, & Schacter, 2007). Results from those studies indicate that activity peaks in both hippocampi during the early construction phase, while the AM is being established (Daselaar, et al., 2008; Rabin, et al., 2010; Schacter & Addis, 2009), although right hippocampal activation can be sustained throughout the elaboration phase if memories are vivid and detailed (St Jacques, et al., In Press). However, BOLD MRI may lack the temporal

102 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 90 resolution to capture small activation delays between two functionally interconnected structures. Of interest, right hippocampal activation has been shown to be sustained at the moment when posterior cortical regions that included visual and auditory cortices and the precuneus became engaged, and to decline as activity ramped up in those structures during elaboration (Daselaar, et al., 2008). This profile of activation is consistent with the idea that the right hippocampus works as an index that recruits regions supporting perceptual memory content during episodic memory retrieval (Moscovitch, et al., 2005; Ranganath, 2010; Teyler & Rudy, 2007). Overall, more work is needed in order to characterize the interaction between the left and the right MTL during AM retrieval. Analyses of effective connectivity could provide a more direct test of the interaction between the two hippocampi and other components from the AM retrieval network Differences between AM and Laboratory tasks I identified a set of regions whose activity differentiated between the AM condition and the two LE conditions. Regions such as the posterior cingulate and retrosplenial cortex, the medial prefrontal cortex, the left superior occipital gyrus and a region from the left middle temporal gyrus showed greater activity in the AM than in either of the two LE conditions. Both the medial prefrontal cortex and the retrosplenial cortex are known to be activated during tasks with high personal relevance (Buckner, et al., 2008; Buckner & Carroll, 2007; Burianova & Grady, 2007; Cavanna & Trimble, 2006; Craik, et al., 1999; Gusnard, Akbudak, Shulman, & Raichle, 2001; Kelley, et al., 2002; Rabin, et al., 2010; Spreng & Grady, 2010; St-Laurent, Abdi, Burianova, & Grady, 2011; St Jacques, Conway, et al., 2011; Svoboda, et al., 2006). Retrieving one s own memory as opposed to imagining someone else s memory leads to greater medial prefrontal and medial parietal activation (Rabin, et al., 2010; Spreng & Grady, 2010; St Jacques, Conway, et al., 2011). Also, the retrosplenial cortex, which contains head direction cells (Cho & Sharp, 2001; Clark, Bassett, Wang, & Taube, 2010), may play a role converting allocentric and egocentric spatial representations (Vann, et al., 2009). Evidence also shows that memory representations that involve imaging the self in a spatial context engage the retrosplenial cortex to a greater extent than imagining impersonal story characters in the same context (e.g., see Szpunar, Watson, & McDermott, 2007). In addition, both the posterior cingulate and the medial

103 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 91 prefrontal cortex are considered central nodes of the default mode network (Andrews-Hanna, et al., 2010; Buckner, et al., 2008; Gusnard, et al., 2001; Raichle, et al., 2001). They are robustly deactivated during stimulus-driven tasks that required outward attention, and they are considered to play an important role in self-projection, mind wandering and other cognitive processes for which attention is directed inward (Buckner, et al., 2008; Buckner & Carroll, 2007; Spreng, Mar, & Kim, 2009). While several features such as recency, rehearsal and emotionality distinguish AM from the two laboratory conditions, AM s self-relevance seems to be the main factor contributing to differences in their neural signature. I also identified regions that showed significantly greater activation in the two LE than in the AM condition, which included the supramarginal portion of the inferior parietal lobule bilaterally, the middle frontal gyrus (BA 6, 9, 10 and 46), the posterior portion of the precuneus and the middle cingulate cortex (BA 32). Some of these regions, such as the inferior parietal lobule, the cingulate cortex and the dorsolateral prefrontal cortex, have been identified as parts of the task-positive, or fronto-parietal attention network (Cabeza, et al., 2008; Corbetta & Shulman, 2002; Fox, et al., 2005; Toro, Fox, & Paus, 2008). Other regions from this network, such as the insula and the posterior portion of the middle temporal cortex (BA 37), were also activated to a greater extent during Clip (but not Narrative) retrieval, than during AM retrieval. The fronto-parietal network is anti-correlated with the default mode network during rest, and is thought to play a role in the top-down modulation of attention (Corbetta & Shulman, 2002; Toro, et al., 2008). Although the discrepancy typically observed between brain regions activated by laboratory and naturalistic tasks of episodic memory was reduced with the current paradigm (Gilboa, 2004; McDermott, et al., 2009), it is possible that the laboratory condition, during which very specific memory snippets were retrieved, required more response monitoring, while AM retrieval, which was more open-ended (e.g., participants could recall whichever part of the cued event came to mind), engaged more intuitive evaluation processes (e.g., feeling-ofrightness; Gilboa, 2004). Also, Svoboda and colleagues (2006) noted that familiarity and repeated exposure reduce activation in the dorsolateral prefrontal cortex at retrieval, and we know that AMs were more familiar to the participants than the laboratory events. Thus, while the patterns of brain activation elicited by the three memory conditions overlapped

104 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 92 extensively, these patterns also reveal that personal memories engaged self-projection processes to a greater extent than laboratory memories, which engaged more attentional processes, possibly due to increased monitoring requirements Encoding versus Retrieval The current imaging findings provide a partial answer to some of the questions raised in the previous chapter concerning encoding versus retrieval. I discussed earlier how my behavioural study could not establish with certainty whether damage to the MTL disrupted memory for perceptual content at encoding, at retrieval, or both. The behavioural results showed a clear paucity of perceptual memory details in the mtle group, which could have reflected a failure to encode perceptual elements, a failure to retain these details (i.e., details faded faster), or a failure to retrieve and integrate perceptual details into multi-modal memory experiences. Because brain activation was measured during retrieval, the current fmri results indicate that the MTL was certainly implicated at that point, although it is likely that it was also implicated at encoding. I observed a clear pattern of hippocampal activation at retrieval that was greater in the perceptually enriched conditions, and that correlated with a measure of perceptual richness in the enriched condition for which there was the most experimental control. These findings are consistent with evidence of retrograde AM impairments in patients with adult-onset trauma or pathology to the MTL (Cipolotti, et al., 2001; Gilboa, et al., 2006; Kapur, 1999; Rosenbaum, et al., 2005; Rosenbaum, et al., 2008), which suggests that the presence of MTL damage at retrieval is sufficient to cause the kind of memory deficits I observed behaviourally in the mtle group. Also, a growing literature links MTL function to the imagining of future and alternative scenes and scenarios, and suggests that MTL damage interferes with the integration of details into mental construct, a process that is essential to the recollection of complex memories (Buckner, 2010; Hassabis, Kumaran, Vann, & Maguire, 2007; Hassabis & Maguire, 2009; Kwan, Carson, Addis, & Rosenbaum, 2010; Schacter & Addis, 2009; but see Hurley, Maguire, & Vargha-Khadem, 2011). Because I did not measure brain activity at encoding, my fmri results cannot determine whether the hippocampus also played a role in the encoding of perceptually enriched memories, but they clearly demonstrate that it is engaged during the retrieval and integration of perceptual details into complex episodic memories. My

105 CHAPTER 3 NEURAL CORRELATES OF PERCEPTUAL RICHNESS 93 results also indicate that it is likely that MTL damage interfered with the retrieval of perceptual memory details in participants with mtle, which is consistent with the retrieval literature. 3.5 Conclusion I adapted the behavioural paradigm described in Chapter 2 to functional MRI in order to identify neural correlates of perceptual qualities of complex memory episodes. I identified regions whose activity was increased in all three memory conditions, which correspond to brain regions commonly engaged during long-term memory retrieval. Contrasts between the different task conditions also revealed a collection of posterior regions in the temporal, parietal and occipital lobe that were activated to a greater extent in the perceptually enriched AM and Clip conditions, than in the perceptually impoverished Narrative condition. The right hippocampus was among those regions, which supports my hypothesis that perceptual richness is an important determinant of hippocampal engagement. Additionally, parametric analyses revealed that a behavioural measure of perceptual richness correlated with right hippocampal activation in the Clip, but not in the Narrative condition. Finally, the contrasts indicated that default mode network regions such as the medial prefrontal and retrosplenial cortex showed more activation during the AM than the two LE conditions, suggesting that retrieving personal memories involves more reliving and self-projection than the retrieval of laboratory stimuli with no personal relevance. On the other hand, the two LE conditions engaged regions known to belong to the fronto-parietal task network to a greater extent than the AM condition, indicating that LE retrieval required more attentional monitoring and verification processes than the AM condition. The neural correlates of perceptual richness form a network of regions whose activation may be disrupted by damage to the hippocampus, in light of evidence that such damage reduces the perceptual richness of episodic memory. I tested this prediction in a study on right mtle, the results of which are presented in the next chapter.

106 Chapter 4 The Impact of Medial Temporal Lobe Epilepsy on the Neural Correlates of Perceptual Richness for Complex Episodic Memories 4.1 Introduction The goal of the current study was to assess how damage to the MTL affects the recruitment of brain regions engaged during the retrieval of perceptually rich memory episodes. In the General Introduction, I hypothesized that the hippocampus is a hub within a network of brain regions that support the retrieval of perceptual memory details. In the previous chapters, I showed a paucity of perceptual details in people with damage to the MTL. In a group of healthy adults, I also identified brain regions engaged by the perceptual richness of episodic memory, which included the right, but not the left hippocampus. With the current study, I followed up on these findings by comparing brain activity during episodic memory retrieval between healthy controls, and individuals with right medial temporal lobe epilepsy (RTLE), a condition that affects the integrity of the right MTL. I predicted that RTLE should interfere with the recruitment of brain regions that support the retrieval of perceptual memory details. Few other studies have assessed how damage to the MTL affects brain activation during AM retrieval. Maguire and colleagues (2001) measured brain activation in Jon, a young man who suffers from developmental amnesia related to perinatal bilateral damage to the hippocampus, using an AM recognition paradigm that Jon could perform. The authors reported mainly left-lateralized activation in controls, but observed contralateral compensatory activation in Jon. Also, they identified increased effective connectivity between the left retrosplenial cortex and the left hippocampus as well as the medial prefrontal cortex in Jon. Importantly, connectivity was not assessed in the right hemisphere in this study, and so several of the regions which I identified to be sensitive to perceptual memory content were left out of their model. Maguire et al. (2005) also tested another amnesic patient with adult-acquired bilateral hippocampal damage on the same paradigm. In contrast with Jon s results, they observed 94

107 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 95 reduced activity in the left hippocampus, parahippocampal gyrus and cingulate cortex, and increased activity in the lateral temporal cortex in that patient. Using a recall task during which AMs were cued with pre-selected titles, Addis et al. (2007) assessed changes in brain activity in a cohort of individuals with left-lateralized mtle (LTLE). Like the Maguire group, they reported increased connectivity between medial parietal regions and the medial prefrontal cortex in their patient group, which may have reflected an increased reliance on a suboptimal retrieval pathway that bypasses the MTL (see also McAndrews, In Press). Addis and colleagues also observed a significant activity reduction in the bilateral hippocampus and in the rest of the canonical AM retrieval network (Cabeza & St Jacques, 2007; Maguire, 2001; Svoboda, McKinnon, & Levine, 2006) in their LTLE group. They also reported reduced effective connectivity between the left hippocampus and the retrosplenial, parahippocampal and medial prefrontal cortex. Even though activation was reduced, effective connectivity was intact throughout the right hemisphere in LTLE participants. Discrepancies between the different studies may be mediated by the severity of memory impairment, and by differences between unilateral and bilateral lesions to the MTL, and between developmental and adult-onset pathology. Also, differences between paradigms may have influenced the results, as AM recognition tasks during which substantial retrieval support is provided help to engage medial temporal regions, and facilitate access to memory details in populations known to suffer from impoverished recollection (Browne, et al., 2011; Loveday & Conway, 2011; St Jacques, Conway, & Cabeza, 2011, November). Findings by Addis et al. (2007) indicate that damage to the left MTL should lead to reduced activation within the AM retrieval network, and to reduced connectivity between the MTL and the rest of the AM network during cued AM recall. However, no one has shown how right-lateralized MTL damage affects the neural signature of AM, nor whether it interferes with activation among regions engaged by perceptual richness, which was the goal of the current study. I tested a group of participants with RTLE on the fmri paradigm described in Chapter 3, which was designed to identify brain regions sensitive to perceptual content during episodic memory retrieval. I compared patterns of brain activation between participants with RTLE, and between an equal number of participants from the control group whose results were presented

108 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 96 in the previous chapter. All RTLE participants tested on this paradigm suffered from epileptic seizures of unilateral hippocampal origin, and were candidates for a medial temporal lobe resection to eliminate their seizures. Like Addis and colleagues (2007), I am reporting comparisons between the RTLE and the controls performed on contrasts between each memory condition and the baseline Counting condition. As mentioned, my recruitment of participants with right-lateralized pathology was motivated by the observation that the right hemisphere was especially engaged by perceptual richness in healthy controls. While I also tested a few participants with left mtle, recruitment difficulties prevented the formation of a full group, and these results will not be discussed here. Based on my paradigm, and on evidence from the literature and from the studies presented in the previous chapters, I made the following predictions: - I expected a LIWC analysis of the post-scan recall data to reveal group differences similar to those observed in the behavioural study reported in Chapter 2, although I anticipated that the lack of probing for perceptual memory content may limit the sensitivity of the current analysis. - I expected the increase in hippocampal activation observed during memory retrieval to be reduced in the RTLE group compared to the controls in all three memory conditions. - I expected to observe a similar pattern of group differences in other regions sensitive to perceptual richness, such as the rhinal and parahippocampal cortex, fusiform gyrus, inferior parietal lobule, middle occipital and medial parietal regions. - Finally, I expected the largest group differences in activation within the hippocampus and other regions sensitive to perceptual richness to be observed in the two perceptually enriched conditions, AM and Clips. I expected group differences to be less salient in the perceptually impoverished Narrative condition, to parallel results from the count of Perceptual details reported in the behavioural study.

109 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS Methods Participants Participants were recruited and tested in accordance with a protocol approved by the Research Ethics Board of the University Health Network. Ten participants with RTLE were recruited through the Epilepsy Clinic at the Toronto Western Hospital, and were tested on our procedure. I selected 10 participants among the healthy controls whose data were presented in Chapter 3 who were closest in age to participants in the RTLE group. These 10 controls data are contrasted with those of patients with RTLE in the current analysis. I elected to have equal group sizes because, based on previous results from Addis et al. (2007), I expected to observe reduced activation in participants with mtle. I wanted to guard against biasing the results by having a larger n, and thus greater statistical power, in the control group. Table 4.1 Mean Demographic and Neuropsychological Characteristics of the Control and the RTLE group Controls R-mTLE Norms (n = 10) (n = 10) Gender (M / F) 4M / 6F 6M / 4F n/a Age in Years 37.7 (11.7) 39.1 (15.0) n/a Years of Education 16.6 (3.3) 13.2 (2.3) n/a WASI Full Scale IQ n/a 107 (6.8) 100 (15) b Performance IQ n/a (8.9) 100 (15) b Verbal IQ n/a (9.1) 100 (15) b WASI Matrix Reasoning Subtest 13.4 (1.2) 11.5 (1.6) 10 (3) b RAVLT Total Recall Score n/a 44 (9.4) 51.1 (8.6) b RVDLT Total Recall Score n/a 38.2 (11.2) 44.4 (12.4) a Warrington Words n/a 47.4 (2.1) 45.5 (3.2) c Warrington Faces n/a 39.7 (5.3) 44.8 (3.3) c Note. Standard deviation is between parentheses. Norms were obtained from years old from a Spreen and Strauss (1991), b Strauss, Sherman and Spreen (2006), and c Warrington (1984). Domin = languagedominant; N-Domin = non-language-dominant; F = female; M = male; n/a = not applicable; IQ = Intellectual Quotient; RAVLT = Rey Auditory Verbal Learning Test, RVDLT = Rey Visual Design; WASI = Wechsler Abbreviated Scale of Intelligence.

110 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 98 Table 4.1 includes mtle participants performance on neuropsychological tests, as well as additional demographic information about the mtle participants and the control group. The Matrix Reasoning Subscale of the Wechsler Abbreviated Scale of Intelligence was administered to controls to estimate non-verbal intelligence and compare their scores to those of patients. In the RTLE group, two pairs of tests, the Warrington Recognition Memory Test for Words and for Faces (Warrington, 1984), and the Rey Auditory Verbal and Rey Visual Design Learning Tests (Spreen & Strauss, 1991; Strauss, Sherman, & Spreen, 2006), reflect memory for verbal and non-verbal material, respectively. The participants with RTLE were candidates for a unilateral temporal lobe resection. All had seizures originating from the right hippocampus. Only one participant was diagnosed with bilateral seizures based on intracranial recording: approximately 30% of his seizures originated from the left hippocampus, and the remaining 70% originated from the right hippocampus. The mean age of seizure onset was 25.6 years (Standard Deviation = 15.8 years) in the mtle group. Four of 10 participants began having recurrent seizures before reaching majority (at age 4, 12, 14 and 16, respectively), while the remaining participants started having seizures in adulthood. Five participants were diagnosed with right mesial temporal sclerosis (MTS) by a radiologist according to the following clinical criteria: atrophy on a T1-weighted MRI scan, and high intensity indicative of gliosis on a T2-weighted MRI scan. One participant with mtle had a small posterior temporal cavernoma on the left hemisphere, and one participant had right amygdala dysplasia. In all the other participants, no structural brain damage was observed beside MTS Procedure The task and scanning parameters were identical to those reported in Chapter 3. Postscan retrieval sessions were conducted in eight of the 10 participants with RTLE, during which they recalled the different memories retrieved in the scanner in their own words. Of the two participants who did not complete this portion of the study, one had a seizure, and the other had to leave due to time constraints. Also, two participants only completed a portion of the post-scan session due to fatigue, and their performance was averaged per condition over the trials they completed. Recall transcripts were analyzed with LIWC2007 (Pennebaker, Chung, Ireland, Gonzales, & Booth, 2007). A sample of eight participants was too small to conduct

111 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 99 parametric analyses, so mean word counts from the Perceptual Processes and Verbs categories are simply reported per group and condition. As discussed in the previous chapter, the absence of probing for perceptual details during post-scan recall resulted in the LIWC measure lacking sensitivity to perceptual memory content, most notably in the AM condition. For this reason, I also make inferences about memory content in the different conditions based on the more thorough behavioural analysis performed in a different cohort of patients with left and right mtle that is reported in Chapter Statistical Analysis I performed comparisons between patterns of brain activation elicited by each memory condition and the Counting condition within both the control and the RTLE group, at a threshold of p <.0001 uncorrected (cluster > 5 voxels). I also performed direct group comparisons on the contrasts between memory conditions and the baseline. For group comparisons, I adopted the more liberal threshold of p <.005 uncorrected (cluster > 5 voxels) to compensate for limited statistical power due to small sample sizes. Addis et al. (2007) also used this threshold for display purposes when reporting results from a group comparison between activity elicited by AM retrieval in individuals with LTLE and controls. Group comparisons performed on direct contrasts between the memory conditions lacked the statistical power to identify significant differences between the RTLE and the control groups, and they will not be discussed here. For all the reported contrasts, I also used a bilateral hippocampal MARINA mask (Bertram Walter Bender Institute of Neuroimaging, University of Giessen) to delineate and quantify significant voxels falling within the hippocampus proper. The mask was used for descriptive rather than for statistical purposes: I adopted the same uncorrected thresholds as for the full-brain contrasts, and no small volume correction was applied.

112 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS Results Behavioural Results The mean number of trials retrieved successfully (Story Content rating > 1), and the mean Story Content and Vividness ratings are plotted per memory condition for the RTLE group and their 10 matched controls in Figure 4.1. Vividness ratings were not available for one participant with RTLE because she did not provide her answers (button press) within the time window allotted. Trials with Story Content ratings of 1 were excluded from both the fmri analysis and the behavioural results. Only ratings between 2 and 4 were included in the averages plotted in Figure 4.1, so that mean ratings best characterized the perceived story content and vividness of trials that were included in the fmri analysis. However, excluding ratings of 1 from the mean rating calculations did not affect the pattern of results due to the low number of unsuccessful trials. The number of trials retrieved by the RTLE group was slightly reduced for the Narrative condition, which is consistent with what was observed in the behavioural analysis (Chapter 2), although this difference did not reach significance here (F(2, 18) = 2.76, p =.09). High rates of retrieval success in the RTLE group suggest that they benefitted from viewing the stories twice at encoding, as lower rates of success were observed among mtle participants in the behavioural study during which stories were only shown once. A two-way ANOVA with repeated measure over memory condition comparing ratings of Story Content between the control and the RTLE group revealed a significant main effect of condition (F(2, 36) = 9.35, p <.005), but no main effect of group (F(1, 18) < 1) or group x condition interaction effect (F(2, 36) = 1.70, p =.20). Series of paired sample t-tests revealed that Story Content ratings were significantly higher for the Clip than for the Narrative (t(19) = 5.21, p < 0.001) or the AM condition (t(19) = 2.42, p <.05), while they did not differ significantly between the Narrative and the AM condition (t(19) = 1.40, p =.18). A two-way ANOVA with repeated measure over memory condition comparing Vividness ratings between the two groups revealed a significant group effect, indicating that the RTLE group s Vividness ratings were significantly higher than those of their matched controls (F(1, 17) = 4.61, p <.05). This group difference is in the opposite direction from the one observed in the behavioural study, and it contradicts the LIWC results presented below, which suggests that vividness ratings were

113 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 101 inflated in the current mtle group. A significant main effect of condition was also observed (F(2, 34) = 26.09, p <.001), but no group x condition interaction effect (F(2, 34) < 1). Series of paired sample t-tests indicated that Vividness ratings differed significantly between each pair of memory conditions (p <.05). Overall, ratings were poor indicators of task performance, and they did not provide an accurate portrayal of the differences between groups and conditions observed with objective measures of memory content. Figure 4.1 Mean number of trials retrieved successfully, and mean Story Content and Vividness ratings (in-scan; 1-4 Likert scales), per group and experimental condition. AM = Autobiographical Memory; Narra = Narrative Story Condition; Clip = Film Clip Story Condition. Error bars represent standard errors of the mean. The mean number of words from LIWC2007 s Verbs and Perceptual Processes categories is plotted per memory condition for eight of the RTLE participants and their 10 matched controls in Figure 4.2. In the AM condition, independent-sample t-tests failed to reveal group differences in either the number of verbs or the number of Perceptual Processes words (both t(16) <1). The lack of group difference and the relatively lower number of words per categories contrast with the results reported in Chapter 2 and Appendix G, but may be explained by the absence of probing for perceptual details and by participant fatigue.

114 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 102 A two-way ANOVA comparing numbers of verbs between the mtle and the control group, with repeated measures over condition (Clip and Narrative), revealed a significant main effect of condition (F(1, 16) = 32.81, p <.001). Significantly more verbs were produced in the Clip than in the Narrative condition, consistent with the behavioural study (see Appendix G). Although verb counts offer an approximate measure of story content, unlike manual counts of Story details, their numbers are also inflated by perceptual content. Contrary to the behavioural study, no significant main effect of group (F(1, 16) < 1) or condition x group interaction effect (F(1, 16) = 1.10, p =.31) were observed. A two-way ANOVA comparing Perceptual Processes words between the two groups, with repeated measure over condition (Clip and Narrative), revealed significant main effects of group (F(1, 16) = 4.79, p =.04) and condition (F(1, 16) = 42.14, p <.001). As in the behavioural study, controls produced significantly more Perceptual Processes words than individuals with mtle, and more words were produced in the Clip than the Narrative condition. However, the group x condition interaction effect did not reach significance here (F(1, 16) = 2.02, p =.17), and indeed the mtle group produced more Perceptual Processes words in the Clip than the Narrative condition, just like the controls. Figure 4.2 Mean number of words per successful (Story Content rating > 1, complete recording) trial from LIWC2007 s Verbs and Visual Processes categories. Numbers of words are plotted per memory condition for each group. Error bars represent SEM. Note: AM = Autobiographical Memory; Narra = Narrative Condition; Clip = Film Clip Condition.

115 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 103 Again, discrepancies between the current results and those from the behavioural study are at least partially accounted for by the current lack of probing for perceptual content. As shown in Appendix F, numbers of Perceptual details recalled by the mtle group differed between the Clip and the Narrative condition after the first of two recordings, but this difference was reduced after mtle participants were probed for perceptual content in the second recording. These results suggest that, in the absence of probing, Perceptual details are more likely to be reported spontaneously in the Clip than in the Narrative condition. If I had performed probing for perceptual content in the current study, it may have reduced the difference in Perceptual Processes words between the Clip and the Narrative condition in the mtle group. Thus, caution is needed when using the current LIWC data to interpret results from the fmri analysis, as perceptual content that was available to participants minds may not have been reflected in their behavioural measures. Nevertheless, the current LIWC results suggest that the amount of retrieved story content, as estimated by the number of verbs, was more similar between the two groups than perceptual content, which was reduced in the mtle group in the two laboratory conditions Neural Correlates of Autobiographical Memory I first contrasted brain activation between the AM and the Counting condition within the RTLE and the control groups (see Figure 4.3, top and middle, respectively; p <.0001 uncorrected, cluster size > 5 voxels). Regions showing significantly greater activation in the AM condition in both the control and the RTLE group are listed in Table 4.2. These regions, which are commonly identified as parts of the neural correlates of autobiographical memory retrieval (Cabeza & St Jacques, 2007; Maguire, 2001; McDermott, Szpunar, & Christ, 2009; Svoboda, et al., 2006), included the hippocampus, the rhinal, parahippocampal and fusiform cortex, the middle temporal gyrus and the middle and inferior frontal gyri in the left hemisphere, and the posterior cingulate cortex, the precuneus, the angular portion of the inferior parietal lobule, the temporal pole, the medial prefrontal cortex, and the occipital pole (Brodmann area 18) bilaterally.

116 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 104 Figure 4.3 Contrast between the AM and the Counting condition in controls (top) and in the RTLE group (middle; p <.0001 uncorrected, cluster threshold > 5 voxels), and regions showing greater levels of activity for this contrast in the control than the RTLE group (bottom; p <.005 uncorrected; cluster threshold > 5 voxels).

117 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 105 Table 4.2 Brain Regions Activated in the Autobiographical Memory Condition, per Group MNI coordinates Region Hemi BA t value x y z Controls Retrosplenial Cortex L & R BA L BA 29 / Posterior Cingulate Cortex / Precuneus L BA 31 / Inferior Parietal Lobule (AG) R BA L BA Rhinal Cortex / Hippocampus L BA Rhinal Cortex / Parahippocampal Gyrus R BA 28 / Parahippocampal Gyrus L BA Fusiform Gyrus L BA Temporal Pole R BA L BA Amygdala L n/a Middle Temporal Gyrus L BA Medial Frontal Gyrus L BA 8 / Middle Frontal Gyrus L BA Inferior Frontal Gyrus L BA 45 / Middle Occipital Gyrus L BA R BA Calcarine Gyrus L BA Patients with RTLE Retrosplenial Cortex L BA 29 / Posterior Cingulate Cortex / Precuneus L BA 31 / R BA 31 / Inferior Parieral Lobule (AG) R BA L BA Rhinal Cortex / Hippocampus L BA Parahippocampal Gyrus L BA Fusiform Gyrus L BA Temporal Pole L BA R BA Middle Temporal Gyrus L BA Superior Frontal Gyrus L BA L BA Medial Frontal Gyrus R > L BA Middle Frontal Gyrus L BA Inferior Frontal Gyrus R BA L BA Cuneus L BA 17 / Lingual Gyrus L BA L BA Note. All activations are signification at p <.0001, uncorrected (cluster > 5 voxels). The coordinates (MNI space) and t value, and the corresponding BA of the voxel with the highest t value ("peak voxel") within a cluster are provided. AG = Angular Gyrus, BA = Brodmann area, Hemi = hemisphere, L = left, MNI = Montreal Neurological Institute, R = right, RTLE = patients with right temporal lobe epilepsy.

118 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 106 I also performed a direct comparison between the two groups for AM-related activation (AM > Counting contrast). Maps are shown in the bottom panel of Figure 4.3. I adopted a liberal threshold to compensate for lower statistical power due to the small sample sizes (p <.005 uncorrected, cluster size > 5 voxels). Regions showing significantly greater activation in the control than in the RTLE group are listed in Table 4.3. These regions included right temporal lobe structures such as the right hippocampus and rhinal, parahippocampal and fusiform cortex, the right middle temporal gyrus and the right temporal pole, which was predictable based on the RTLE group s pathology. Interestingly, activation was also decreased in the RTLE Table 4.3 Brain Regions Showing Greater Activaty for Controls than for Patients with RTLE in the Autobiographical Memory Condition MNI coordinates Region Hemi BA t value x y z Controls > Patients with RTLE Precuneus / Paracentral Lobule L & R BA 7 / Rhinal Cortex / Hippocampus R BA Rhinal Cortex L BA Parahippocampal Gyrus R BA Fusiform Gyrus L BA L BA Temporal Pole R BA Middle Temporal Gyrus R BA L BA Superior Frontal Gyrus L BA 9 / L BA R BA R BA Middle Frontal Gyrus R BA L BA Inferior Frontal Gyrus L BA Medial Frontal Gyrus L & R BA Anterior Cingulate Cortex L BA R BA 32 / R BA Middle Occipital Gyrus L BA R BA Note. All activations are signification at p <.005, uncorrected (cluster > 5 voxels). Difference in activity between the Autobiographical Memory and the Counting condition was compared between patients with RTLE and controls. No voxel showed significantly greater activation for patients than for controls at this threshold. The coordinates (MNI space) and t value, and the corresponding BA of the voxel with the highest t value ("peak voxel") within a cluster are provided. BA = Brodmann area, Hemi = hemisphere, L = left, MNI = Montreal Neurological Institute, R = right, RTLE = patients with right temporal lobe epilepsy.

119 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 107 group in other regions shown to be sensitive to perceptual richness in the analysis presented in Chapter 3, such as the middle occipital gyrus (Brodmann area 19), the left fusiform, and the anterior portion of the precuneus. Additionally, I observed a significant decrease in activation within regions such as the left middle temporal gyrus, the anterior cingulate cortex and the inferior, middle and superior frontal gyri, which are considered to be more general neural correlates of memory retrieval, whether episodic or otherwise. At this threshold, no voxels showed significantly greater activation in the RTLE group than in the control group Neural Correlates of Memory for Film Clips I then contrasted brain activation between the Film Clip and the Counting condition for each of the groups (see Figure 4.4, top and middle, respectively; p <.0001 uncorrected, cluster size > 5 voxels). Regions showing significantly greater activation in the Film Clip than in the Counting condition in controls and in people with RTLE, respectively, are listed in Table 4.4. The common regions activated by both participants with RTLE and controls were mostly left lateralized, and included the left hippocampus and rhinal cortex, the angular portion of the left inferior parietal lobule, the left temporal pole, the left middle temporal gyrus, the left lateral prefrontal cortex, the left cuneus and inferior occipital cortex, the posterior cingulate cortex, the precuneus, and the medial frontal cortex. Of note, the controls showed a much greater pattern of bilateral recruitment of these regions. These results suggest that damage restricted to the right MTL in people with mtle interferes with the recruitment of widespread extratemporal regions in the right hemisphere. To confirm the differences apparent in the single group analysis, I performed a direct comparison between brain activation elicited by the Film Clip condition in controls and in participants with RTLE (Film Clips > Counting contrast; see Figure 4.4, bottom panel), at the more liberal threshold I adopted for group comparisons (p <.005 uncorrected, cluster size > 5 voxels). Regions showing significantly greater activation in the control than in the RTLE group are listed in Table 4.5, and included right temporal lobe structures such as the right hippocampus, rhinal and parahippocampal cortex, the right temporal pole and the right middle temporal cortex.

120 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 108 Figure 4.4 Contrast between the Film Clip and the Counting condition in controls (top) and in people with RTLE (middle; p <.0001 uncorrected, cluster threshold > 5 voxels), and regions showing greater levels of activity for this contrast in the control than the RTLE group (bottom; p <.005 uncorrected; cluster threshold > 5 voxels).

121 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 109 Table 4.4 Brain Regions Activated in the Film Clip Memory Condition, per Group MNI coordinates Region Hemi BA t values x y z Controls Posterior Cingulate Gyrus L BA R BA Retrosplenial Cortex L & R BA 30/ Inferior Parietal Lobule (AG) R BA L BA Rhinal Cortex / Hippocampus R BA Hippocampus L n/a Fusiform Gyrus L BA L BA Temporal Pole R BA L BA Middle Temporal Gyrus R BA R BA L BA L BA L BA Inferior Temporal Gyrus L BA Superior Frontal Gyrus L BA L BA Inferior Frontal Gyrus L BA R BA R BA R BA Medial Frontal Gyrus L BA Gyrus Rectus R BA Cuneus R BA L BA Inferior Occipital Gyrus L BA Thalamus L n/a Patients with RTLE Posterior Cingulate Cortex / Precuneus L BA 31 / Posterior Cingulate Cortex R BA Inferior Parietal Lobule (AG) L BA Rhinal Cortex L BA L BA Temporal Pole L BA L BA Middle Temporal Gyrus L BA L BA Superior Temporal Gyrus L BA R BA Middle Frontal Gyrus L BA Medial Frontal Gyrus L BA Cuneus L BA Lingual Gyrus L BA Note. All activations are signification at p <.0001, uncorrected (cluster > 5 voxels). The coordinates (MNI space) and t value, and the corresponding BA of the voxel with the highest t value ("peak voxel") within a cluster are provided. AG = Angular Gyrus, BA = Brodmann area, Hemi = hemisphere, L = left, MNI = Montreal Neurological Institute, R = right, RTLE = patients with right temporal lobe epilepsy.

122 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 110 Table 4.5 Brain Regions Showing Greater Activaty for Controls than for Patients with RTLE in the Film Clip Memory Condition MNI coordinates Region Hemi BA t values x y z Controls > Patients with RTLE Posterior Cingulate Cortex L BA Middle Cingulate Cortex L BA Precuneus L BA Retrosplenial Cortex R BA 29 / Inferior Parietal Lobule (AG) R BA L BA Middle Temporal Gyrus L BA R BA Inferior Temporal Gyrus L BA Temporal Pole R BA Fusiform Gyrus L BA Rhinal Cortex R BA Parahippocampal Gyrus R BA Superior Frontal Gyrus L BA L BA R BA R BA Middle Frontal Gyrus L BA L BA L BA Inferior Frontal Gyrus L BA L BA R BA R BA Medial Frontal Gyrus L BA Superior Occipital Gyrus L BA Middle Occipital Gyrus L BA R BA Lingual Gyrus L BA R BA Cuneus L BA Note. All activations are signification at p <.005, uncorrected (cluster > 5 voxels). Difference in activity between the Film Clip and the Counting condition was compared between patients with RTLE and controls. No voxel showed significantly greater activation for patients than for controls at this threshold. The coordinates (MNI space) and t value, and the corresponding BA of the voxel with the highest t value ("peak voxel") within a cluster are provided. AG = Angular Gyrus, BA = Brodmann area, Hemi = hemisphere, L = left, MNI = Montreal Neurological Institute, R = right, RTLE = patients with right temporal lobe epilepsy. As in the AM condition, I observed significantly less activation in the RTLE group in regions sensitive to perceptual richness, such as the middle occipital gyrus, lingual gyrus and

123 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 111 cuneus, the left fusiform, the posterior cingulate cortex and the precuneus, and the angular portion of the right inferior parietal lobule. I also observed significantly reduced activation in the RTLE group in regions typically engaged during retrieval, such as the left middle temporal gyrus, the left thalamus, and the inferior, middle and superior frontal gyri. No voxels showed significantly greater activation in the RTLE group than in the control group at this threshold Neural Correlates of Memory for Narratives Finally, I contrasted brain activation between the Narrative and the Counting condition for each of the groups (see Figure 4.5, top and middle, respectively; p <.0001 uncorrected, cluster size > 5 voxels). Patterns of activation were much reduced in the RTLE group in comparison to the control group for this condition. Regions showing significantly greater activation in the Narrative than in the Counting condition in controls and in participants with RTLE, respectively, are listed in Table 4.6. Just like for the previous contrast, regions activated by both groups were mostly left lateralized. They included the posterior cingulate cortex and the precuneus, the anterior portion of the left inferior parietal lobule, the left temporal pole and middle temporal gyrus, the left middle frontal gyrus, and the left cuneus. In the controls, activation was more left-lateralized for the Narrative than for the Film Clip condition, which elicited more bilateral activation (see Chapter 3). Nevertheless, in both the Clip and the Narrative conditions, group comparisons revealed patterns of activation that were more leftlateralized in people with RTLE, and more bilateral in controls. I also performed a direct comparison between the brain activation elicited by the Narrative condition (Narrative > Counting) in the two groups (Figure 4.4, bottom), at the more liberal threshold I used for group comparisons (p <.005 uncorrected, cluster size > 5 voxels). Regions showing significantly greater activation in the control group than in the RTLE group are listed in Table 4.7, and included right temporal lobe structures such as the right hippocampus, rhinal and parahippocampal cortex, the right temporal pole, and the right middle temporal gyrus. Unlike what I observed for the AM and the Clip conditions, activation was also reduced in the left hippocampus and in the left rhinal and parahippocampal cortex in participants with mtle, although to a lesser extent than in the right hemisphere.

124 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 112 Figure 4.5 Contrast between the Narrative and the Counting condition in controls (top) and in people with RTLE (middle; p <.0001 uncorrected, cluster threshold > 5 voxels), and regions showing greater levels of activity for this contrast in the control than the RTLE group (bottom; p <.005 uncorrected; cluster threshold > 5 voxels).

125 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 113 Table 4.6 Brain Regions Activated in the Narrative Memory Condition, per Group MNI coordinates Region Hemi BA t values x y z Controls Precuneus L & R BA Posterior Cingulate Cortex L & R BA Inferior Parietal Lobule (AG) R BA L BA Hippocampus / Parahippocampal Gyrus L BA Parahippocampal Gyrus R BA Amygdala L n/a Rhinal Cortex R BA Fusiform Gyrus L BA L BA Temporal Pole R BA L BA Superior Temporal Gyrus L BA Middle Temporal Gyrus R BA L BA Inferior Temporal Gyrus L BA Middle Frontal Gyrus L BA Inferior Frontal Gyrus L BA L BA R BA Medial Frontal Gyrus L BA Anterior Cingulate Cortex L BA 9 / Gyrus Rectus L & R BA Inferior / Middle Occipital Gyrus L BA Lingual Gyrus L BA R BA Cuneus L BA R BA Patients with RTLE Posterior Cingulate Cortex / Precuneus R BA 31 / Posterior Cingulate Cortex / Precuneus L BA 31 / Inferior Parietal Lobule (AG) L BA Temporal Pole L BA Middle Temporal Gyrus L BA Middle Frontal Gyrus L BA Cuneus L BA Note. All activations are signification at p <.0001, uncorrected (cluster > 5 voxels). The coordinates (MNI space) and t value, and the corresponding BA of the voxel with the highest t value ("peak voxel") within a cluster are provided. AG = Angular Gyrus, BA = Brodmann area, Hemi = hemisphere, L = left, MNI = Montreal Neurological Institute, R = right, RTLE = patients with right temporal lobe epilepsy.

126 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 114 Table 4.7 Brain Regions Showing Greater Activaty for Controls than for Patients with RTLE in the Narrative Memory Condition MNI coordinates Region Hemi BA t values x y z Controls > Patients with RTLE Precuneus L & R BA Paracentral Lobule L & R BA Inferior Parietal Lobule (SG) R BA Inferior Parietal Lobule (AG) R BA Rhinal Cortex R BA Hippocampus R n/a Temporal Pole R BA Middle Temporal Gyrus R BA Superior Frontal Gyrus R BA R BA Middle Frontal Gyrus L BA Inferior Frontal Gyrus R BA R BA R BA L BA Middle Cingulate Cortex L BA Cuneus R BA L BA Middle Occipital Gyrus L BA L BA Inferior / Middle Occipital Gyrus R BA Thalamus L n/a R n/a Caudate R n/a Note. All activations are signification at p <.005, uncorrected (cluster > 5 voxels). Difference in activity between the Narrative and the Counting condition was compared between patients with RTLE and controls. No voxel showed significantly greater activation for patients than for controls at this threshold. The coordinates (MNI space) and t value, and the corresponding BA of the voxel with the highest t value ("peak voxel") within a cluster are provided. AG = Angular Gyrus, BA = Brodmann area, Hemi = hemisphere, L = left, MNI = Montreal Neurological Institute, R = right, RTLE = patients with right temporal lobe epilepsy, SG = Supramarginal Gyrus. Like for the other two conditions, I also observed significantly less activation in the RTLE group than in the controls in regions sensitive to perceptual richness, such as the middle occipital gyrus, lingual gyrus and cuneus (right > left), the left fusiform gyrus, the precuneus, and the angular portion of the right inferior parietal lobule. No voxels showed significantly greater activation in the RTLE group than in the control group at this threshold.

127 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS Differences in Hippocampal Activation Between the RTLE and the Control Groups I assessed group differences in levels of activation in the hippocampus proper in each memory condition compared to the Counting condition (see Figure 4.6). I used a bilateral hippocampal mask created in MARINA (Bertram Walter Bender Institute of Neuroimaging, University of Giessen) to circumscribe hippocampal voxels that reached significance during each of the whole-brain comparisons discussed above. The mask was used for descriptive rather than for statistical purposes: it served to identify a peak hippocampal voxel, and to quantify the spatial extent of activation through a hippocampal voxel count. No threshold adjustment or small volume correction was performed. Figure 4.6 Medial temporal voxels showing significantly greater activation in the controls than in participants with RTLE, per experimental conditions. Thresholding is identical to what was used for the group comparison on Figures 4.3, 4.4 and 4.5, respectively (p <.005 uncorrected, cluster > 5 voxels; no hippocampal mask was applied to the images). AM = Autobiographical Memory.

128 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 116 Table 4.8 Coordinates of Hippocampal Activation Differentiating Between Groups for Each Experimental Condition MNI Coordinates Contrast # voxels t value x y z AM > Counting Controls** Right Hippocampus Left Hippocampus RTLE Patients** Left Hippocampus Controls > RTLE Patients* Right Hippocampus Clip > Counting Controls** Right Hippocampus Left Hippocampus RTLE Patients** Left Hippocampus Controls > RTLE Patients* Right Hippocampus Narrative > Counting Controls** Right Hippocampus Left Hippocampus RTLE Patients** No Hippocampal Activation Controls > RTLE Patients* Right Hippocampus Left Hippocampus Note. We used a bilateral hippocampal mask to identify significant voxels from the whole brain analyses that fell within the hippocampus (clusters > 5 voxels; *p <.005 uncorrected; **p <.0001 uncorrected). The coordinates (MNI space) and t values of the hippocampal voxels with the largest t values are listed here, as well as the total number of significant voxels identified in each hippocampus. AM = Autobiographical Memory, MNI = Montreal Neurological Institute.

129 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 117 For each memory condition, bilateral hippocampal activation was observed in the controls, although activation was more widespread in the left hippocampus for the Narrative condition. In patients with RTLE, no significant right hippocampal activation was observed for any of the conditions, which is consistent with their right hippocampal pathology. By contrast, their left hippocampus was activated during the AM and the Film Clip conditions, but not the Narrative condition. This pattern of activation was unexpected, as I hypothesized that largest group differences in medial temporal activity should be observed in the perceptually enriched memory conditions. I explore possible explanations for this unanticipated finding in the discussion Discussion Summary of Findings In order to assess how damage to the MTL interferes with the activation of brain regions engaged during the retrieval of perceptually rich memory episodes, I tested people with right unilateral mtle, and an equal-sized sample of healthy controls, on an fmri paradigm designed to identify the neural correlates of perceptual episodic memory richness. As expected, the right hippocampus was significantly less activated in the RTLE group than in the control group, for all three memory conditions. This is consistent with the hippocampal neuronal loss and reduced MTL metabolism associated with mtle (Duncan, 1997; Trotta, et al., 2011; Walker, Chan, & Thom, 2007; Wieser, 2004), and with results from other brain imaging studies that assessed AM retrieval in individuals with unilateral mtle (Addis, 2005; Addis, et al., 2007; Giannoylis, Lin, & McAndrews, 2011, May). Also, reduced levels of activation were observed bilaterally throughout the brain in participants with RTLE, indicating that damage to the MTL has a widespread effect on the AM retrieval network. A similar bilateral reduction in activity has been reported by Addis and colleagues (2007) in individuals with LTLE during AM retrieval. In all three memory conditions, the RTLE group mainly activated brain regions from the left hemisphere. In the AM and the Film Clip conditions, such regions extended to the left MTL. In the Narrative condition, however, no MTL regions were activated above threshold in either

130 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 118 hemisphere, but regions from the left lateral temporal cortex were significantly activated. In all conditions, a widespread reduction in levels of activation was observed in right-lateralized temporal and extra-temporal brain regions in the RTLE group. Consistent with my previous findings, regions sensitive to perceptual richness, which I identified in healthy controls in Chapter 3, were among the ones expressing this decrease in individuals with mtle. Such regions included the rhinal, fusiform and parahippocampal cortex, the angular portion of the right inferior parietal lobule and the lateral occipital cortex in the right hemisphere, and the precuneus and retrosplenial cortex bilaterally. This reduced activation within brain regions sensitive to perceptual richness in individuals with RTLE is consistent with their reduced count of Perceptual Processes words, at least in the two laboratory event conditions. It also parallels the paucity of Perceptual details observed in the cohort of individuals with unilateral mtle tested in the behavioural study (see Chapter 2). In addition, some of the regions engaged during all three memory conditions in healthy participants were significantly less activated in the RTLE group than in the controls. These regions, which included the middle temporal gyrus, the right temporal pole, the inferior and middle frontal gyrus, and, in the narrative condition, the right caudate nucleus, are known to play a general role in declarative memory retrieval, episodic and otherwise (Burianova & Grady, 2007; Burianova, McIntosh, & Grady, 2010; Cabeza & St Jacques, 2007; St-Laurent, Abdi, Burianova, & Grady, 2011; Svoboda, et al., 2006; also, see Chapter 3 for a more in-depth discussion). These results suggest that damage to the hippocampus disrupts the engagement of brain regions that form a general purpose memory retrieval network, as well as regions that support the retrieval of the content of recollection Behavioural Results I observed similarities as well as discrepancies between the current behavioural results and those reported in Chapter 2. As expected based on the behavioural study, ratings were poor indicators of memory content. In fact, the mtle group rated the vividness of their memory higher than did the controls, which contradicts results from detail counts, and a large literature on memory deficits in patients with mtle. The LIWC analysis was also limited in its capacity to

131 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 119 provide insight into memory content. Unlike what I observed in the behavioural study, verb counts did not reveal significant group differences in any of the conditions. However, participant burden towards the end of a four hour testing session probably contributed to the low number of verbs produced by patients and controls. In addition, no difference in Perceptual Processes words was observed in the AM condition. It is possible that memory was less impaired in this particular cohort of participants with mtle, than in mtle participants tested on the behavioural study described in Chapter 2. The current group s mean performance was numerically higher than the previous group on the Rey Visual Design Learning Test, but not on the Faces version of Warrington s Recognition Memory Test, which provides partial support for this interpretation. However, it is most likely that the lack of group difference for Perceptual Processes words in the AM condition was due to the absence of probing for perceptual memory content. Given the clear pattern of group difference in brain activation observed during AM retrieval, it is unlikely that AM was in fact matched, perceptually and content-wise, between the control and the mtle group. Instead, it seems that memory details that were available to the controls mind were not properly captured by the post-scanning recall. In a previous study comparing AM retrieval between controls and participants with mtle, my colleagues and I observed that group differences in Perceptual detail counts widened following specific probing (unpublished data; see also Levine et al., 2002). Typically, participants tend not to mention perceptual details unless they are instructed to do so, because they consider them irrelevant to the story (i.e. what was the barmaid wearing?). With the current study, it is likely that a more thorough probing of perceptual content would have resulted in a better characterization of the patterns of brain activation reported here, although time constraints prevented me from collecting such data. Significant group differences in Perceptual Processes words were observed in the Clip and the Narrative conditions, with participants with mtle obtaining lower counts than controls. However, participants with mtle produced significantly more words in the Clip than in the Narrative condition, in contrast with results reported in Chapter 2. In the behavioural study, an interaction effect showed that the difference in Perceptual details between the Clip and the Narrative conditions was greater for controls than for participants with mtle. Here, only a non-

132 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 120 significant trend for such an interaction effect was observed. It is likely, however, that probing for perceptual content would have reduced discrepancies between the current findings and those from the behavioural study. Data from the behavioural study demonstrate that, in the mtle group, more Perceptual details are recalled during the first story content probing in the Clip condition, but that more details are then produced in the Narrative condition during the second perceptual content probing (see Appendix F), reducing the gap between the two conditions in the patients. It is plausible that perceptual details are verbalized more spontaneously in the Clip condition because images are central to the story, while other types of information are considered more central in the Narrative and AM conditions. Importantly, information that is available to mind may not always be captured by behavioural measures, but it will always be reflected in patterns of brain activity. Due to these important methodological limitations with the current LIWC analysis, I will rely on results from the more thorough behavioural study in order to interpret the current fmri findings Comparison Between the Behavioural and the Brain Imaging Results In participants with RTLE, the right hippocampus, which is sensitive to perceptual richness in healthy individuals, was not activated above threshold in any of the memory conditions. These reduced levels of activation, which were below threshold in both the perceptually enriched and the perceptually impoverished conditions, parallel the numbers of Perceptual details tallied in participants with mtle in the behavioural study: in the mtle group, the number of Perceptual details 1) was reduced in all conditions, and 2) did not differ significantly between enriched clips and impoverished narratives. Similar patterns of activation were observed in other right-lateralized regions sensitive to perceptual richness, such as the right fusiform, the right parahippocampal cortex, and the right lateral occipital region, whose level of activation failed to reach significance in any of the three memory conditions in the mtle group. However, contrary to my expectations, larger group differences in levels of activation were observed in the Narrative condition than in the AM or the Film Clip condition. This pattern was observed in the hippocampus proper, as well as in the whole-brain contrasts. In the

133 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 121 behavioural study, detail counts were most similar between controls and the mtle group in the Narrative condition, because the controls narratives had the lowest number of Perceptual details of all three memory conditions. On the other hand, detail counts reflected larger group differences in the Clip and AM conditions, because controls retrieved higher numbers of Perceptual details in these two conditions. Numerically, LIWC word counts were also most similar between the two groups in the Narrative condition in the current behavioural data. Based on these results, I expected larger group differences in patterns of brain activity in the AM and Clip conditions, but this hypothesis was not supported by the results. For example, although right hippocampal activation did not reach significance in any of the conditions in the RTLE group, and although it was lower for the Narrative condition in the controls (see Chapter 3), right hippocampal activation showed the largest difference between the mtle and the control group during narrative retrieval. This paradoxical result was at least partially caused by a slight deactivation in the right hippocampus observed in the mtle group during the Narrative condition (data not shown). As for the left hippocampus, it was activated significantly in the AM and the Clip conditions, but not in the Narrative condition in the RTLE group. This pattern differs from results observed in the controls, in whom left hippocampal activation was significant in all three memory conditions, and did not differ between the Narrative and the Clip conditions (see Chapter 3). In fact, no significant activation was observed in the entire MTL in either hemisphere during Narrative retrieval in the RTLE group. Group differences were also less prominent during the AM and the Clip conditions than during the Narrative condition in other regions from the left hemisphere. Regions such as the precuneus, the angular portion of the inferior parietal lobule, the rhinal cortex, the temporal pole, the middle temporal gyrus, the medial prefrontal cortex, the lateral prefrontal cortex and the occipital pole were significantly activated in both groups during the AM and the Clip conditions. Overall, activation was much reduced during the Narrative condition in the RTLE group, although some activation was observed in the left lateral prefrontal cortex, the left temporal pole, the left middle temporal gyrus and left inferior parietal lobule, and the left cuneus. With the exception of the cuneus, these regions were not shown to be sensitive to perceptual richness in healthy participants (see Chapter 3). Instead, the literature indicates that

134 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 122 the lateral temporal cortex and temporal pole play a role in semantic memory (Martin, 2007; Martin & Chao, 2001; Patterson, Nestor, & Rogers, 2007), while the left lateral prefrontal cortex plays a role in the selection, verification and online maintenance of information during strategic retrieval (see Svoboda et al., 2006 for a review). The absence of medial temporal and lateral occipital activation in the RTLE group in the Narrative condition suggests that people with mtle may have relied on a strategy that involved the retrieval of conceptual memory representations when recalling narratives. Also, even though the left MTL was engaged during the retrieval of clips and AMs in the RTLE group, the paucity of perceptual memory content observed behaviourally in these two conditions in individuals with mtle indicates that their attempt at retrieving experiential representations was not very successful, or, possibly, that the left MTL is insufficient to retrieve perceptually rich memories Laterality As mentioned in the introduction, few studies have assessed how damage to the MTL affects the neural correlates of AM and other forms of complex episodic memories. One such study was conducted in Jon, a young man who suffers from developmental amnesia due to bilateral hippocampal damage (Maguire, et al., 2001). During an AM fact-checking task, brain activation was more extensive in Jon than in a group of controls, especially in the right hemisphere, which is opposite to the widespread decrease in activation I observed in participants with RTLE. However, differences between fact-checking and extended cued recall tasks may have accounted for some of the discrepancy between results. Also, Jon s brain may have undergone dramatic plasticity due to the early age at which he suffered from brain damage, in comparison to my RTLE group whose mean age at seizure onset was 25.6 years. Support for this claim comes from a study by Maguire et al. (2005), who observed decreased activation throughout the AM retrieval network in an amnesic patient who suffered bilateral damage to the hippocampus in adulthood, which is more consistent with the current results. My findings are also more in line with evidence from Addis et al. (2007), who used a cued AM recall paradigm similar to my AM task, and who compared brain activation between individuals with LTLE and healthy controls. Like me, Addis et al. (2007) reported a significant decrease in activation in both hemispheres throughout the AM retrieval network in their mtle

135 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 123 group. Regions expressing group differences included right hemisphere regions shown to be sensitive to perceptual richness (see Chapter 3), including the hippocampus, the fusiform and the parahippocampal gyri, the retrospenial cortex, the precuneus, and the inferior parietal lobule. In my results, I also observed widespread bilateral reductions in activation in the RTLE group. Unlike Addis et al. (2007), who observed no significant activation within the MTL in their patient group, I also observed above-threshold activation in medial temporal structures in my mtle participants non-epileptogenic hemisphere in two conditions, AM and Film Clips. These discrepancies between the two studies may be due to differences in statistical power or in thresholding, as recent work comparing brain activation during AM retrieval in larger groups of patients with left and right unilateral mtle indicates that damage to either hemisphere leads to a bilateral reduction in activation throughout the AM retrieval network (Giannoylis, et al., 2011, May; McAndrews, In Press). In the previous chapter, I hypothesized that the left hippocampus is a more central node within the AM network than the right hippocampus. I suggested that the left MTL plays a dominant role in accessing the memory and establishing the core elements of the story, while the right MTL may enrich the memory by contributing to the retrieval of perceptual, and possibly emotional features (Buchanan, Tranel, & Adolphs, 2006; Daselaar, et al., 2008). AM retrieval studies typically report either left lateralized or bilateral profiles of neural activation, and bilateral activation is most often observed when retrieval is emotional or when more time is allotted at retrieval (Cabeza & St Jacques, 2007; Maguire, 2001; McDermott, et al., 2009; Svoboda, et al., 2006). My fmri results from healthy participants indicated that the neural correlates of perceptual richness are strongly right-lateralized. However, results from my behavioural results, and work from others (Herfurth, Kasper, Schwarz, Stefan, & Pauli, 2010; Lah, Grayson, Lee, & Miller, 2004; Lah, Lee, Grayson, & Miller, 2006; Noulhiane, et al., 2007, 2008; St-Laurent, et al., 2009; St-Laurent, Moscovitch, et al., 2011; Viskontas, et al., 2000; but see Voltzenlogel, et al., 2006), indicate that damage to the MTL in either hemisphere leads to an equivalent paucity of perceptual / context-specific memory content. The current findings, combined with evidence from others (Addis, et al., 2007; Giannoylis, et al., 2011, May; McAndrews, In Press), help to reconcile these conflicting results, by demonstrating that both

136 CHAPTER 4 mtle AND THE NEURAL CORRELATES OF PERCEPTUAL RICHNESS 124 left- and right-lateralized mtle lead to a widespread bilateral reduction in activation, which affects right-lateralized medial temporal and posterior cortical regions that are sensitive to perceptual memory content. Finally, Addis and colleagues (2007) also assessed effective connectivity amongst nodes in the AM network in their LTLE group, and connectivity behaved somewhat differently from activation per se. Although brain activity was significantly reduced in the right hemisphere in their LTLE group, effective connectivity amongst MTL components remained intact in the right hemisphere even though it was reduced in the left. Future work should assess whether effective connectivity mirrors those changes in participants with RTLE. Future connectivity models should also include regions sensitive to perceptual richness to determine how the MTL interacts with these at retrieval. 4.5 Conclusion I tested participants with RTLE on the fmri paradigm presented in Chapter 3, and I compared their brain activation with a group of healthy controls. In the RTLE group, activation was reduced significantly in the right hippocampus, as well as in brain regions sensitive to perceptual richness. These results provide a potential neural mechanism for the paucity of perceptual details observed in a different cohort of individuals with RTLE tested on a behavioural paradigm. I also observed reduced activation in brain regions known to play a more general role in declarative memory retrieval, indicating that damage to the MTL has a widespread impact on the neural correlates of retrieval. Patterns of neural activation that showed the greatest group difference were observed in the Narrative condition, which suggests that individuals with RTLE performed the task differently from controls, and may have accessed a more conceptual form of memory representation. The current study bridges the gap between evidence that the hippocampus is engaged during the retrieval of memory rich in multi-modal features, and behavioural evidence that damage to the hippocampus impedes the retrieval of perceptual memory features.

137 Chapter 5 General Discussion 5.1 Summary of Findings The body of work presented in the current thesis provides cohesive and complementary evidence that medial temporal lobe (MTL) structures that include the hippocampus play an important role in the retrieval and integration of context-specific event details, especially perceptual features, into multi-modal episodic memory experiences. The strength of my approach was to manipulate perceptual richness with a laboratory episodic memory task that captured the complexity of autobiographical memory (AM), while controlling for other memory characteristics such as story content, emotionality, rehearsal, age of the memory and personal relevance. Previous work had suggested that perceptual richness, an important dimension of AM (Brewer, 1995; Conway, 2009; Moscovitch, et al., 2005; Rubin, 2005; Rubin & Greenberg, 1998; Rubin, Schrauf, & Greenberg, 2003; Tulving, 1984), engages the MTL during retrieval (Addis, Moscovitch, Crawley, & McAndrews, 2004; Gilboa, Winocur, Grady, Hevenor, & Moscovitch, 2004; St Jacques, Rubin, & Cabeza, In Press; Viard, et al., 2007). However, in naturalistic memories, perceptual richness can be confounded with other characteristics such as emotionality or personal relevance (Daselaar, et al., 2008; Rubin, et al., 2003). My novel paradigm permitted to capture perceptual richness independently of other episodic memory features, and to assess how it is affected by damage to the MTL. I showed that individuals with unilateral mtle recalled fewer perceptual details than healthy participants. Unlike controls, their memory for enriched audio-visual film clips was as perceptually impoverished as their memory for bare-bones narratives. This paucity of perceptual details was also observed with an autobiographical memory (AM) condition in the mtle group, which replicated previous findings from another cohort of individuals with mtle tested on a different AM task (St-Laurent, Moscovitch, Levine, & McAndrews, 2009). In addition, I also observed that the hippocampus was implicated in memory for story details, though usually not to the same extent as for perceptual details. Together, these results confirm that the integrity of the MTL is crucial to the 125

138 CHAPTER 5 GENERAL DISCUSSION 126 perceptual richness and event-specific content of complex episodic memories, whether acquired in the laboratory or during everyday experiences. I also adapted my memory paradigm for functional MRI (fmri). In healthy participants, several overlapping brain regions were activated above threshold during AM and the laboratory tasks, indicating that the laboratory tasks, which were designed to resemble AM, were successful at engaging similar neural correlates. However, some differences were also observed between the different memory conditions. For example, midline structures such as the retrosplenial cortex and the medial prefrontal cortex were activated to a greater extent by AM, which is consistent with a literature indicating that these regions are engaged by personally relevant material (Andrews-Hanna, Reidler, Sepulcre, Poulin, & Buckner, 2010; Cabeza, et al., 2004; Rabin, Gilboa, Stuss, Mar, & Rosenbaum, 2010; St Jacques, Conway, Lowder, & Cabeza, 2011; Szpunar, Watson, & McDermott, 2007). In additions, contrasts between the memory conditions revealed that hippocampal engagement was greater during the retrieval of perceptually enriched than perceptually impoverished memories. This effect was strongly lateralized to the right hemisphere. A parametric analysis revealed that perceptual content correlated positively with right hippocampal activation in the film clip condition, although no such relationship was observed in the Narrative or the AM condition. Memories for film clips were well controlled for characteristics of no interest that were not controlled in the noisier AM condition, such as length, personal relevance, rehearsal and recency, which may have facilitated the observation of a significant correlation between perceptual richness and hippocampal activation in the Clip condition. On the other hand, variability within those dimensions may have modulated hippocampal activation during AM retrieval, making it more difficult to observe a clear relationship between hippocampal activation and perceptual content. As discussed previously, the behavioural measure entered as a parametric modulator underestimated perceptual memory content, especially in the AM condition, which may also have hindered the statistical power of the analysis. Importantly, evidence from the literature indicates a link between hippocampal activity and indicators of AM perceptual richness such as ratings of details, vividness, perceptual content and reliving (Addis, Moscovitch, et al., 2004; Gilboa, et al., 2004;

139 CHAPTER 5 GENERAL DISCUSSION 127 St Jacques, et al., In Press; Viard, et al., 2007), and my failure to replicate such findings in the AM condition was likely due to methodological limitations. Also, the significant correlation I observed in the Film Clip condition, which was well controlled and perceptually enriched, provides strong evidence that the right hippocampus is engaged during the retrieval of perceptual memory features. I also identified other neural correlates of perceptual richness in healthy participants. These correlates were mostly right-lateralized, and were comprised of regions from the secondary visual cortex (BA19), the ventral visual stream and medial temporal cortex, the posterior cortical midline, and the inferior parietal lobule. Much evidence from the literature implicates these regions in perception, in visual imagery and in the recollection of past events (Cabeza, Ciaramelli, Olson, & Moscovitch, 2008; Cavanna & Trimble, 2006; Daselaar, Porat, Huijbers, & Pennartz, 2010; Huijbers, Pennartz, Rubin, & Daselaar, 2011; Lehmann, Pascual- Marqui, Strik, & Koenig, 2010; Litman, Awipi, & Davachi, 2009; Stevens, Kahn, Wig, & Schacter, In Press; Vann, Aggleton, & Maguire, 2009; Vilberg & Rugg, 2008). In individuals with RTLE, who have focal damage to the right hippocampus, activation was reduced among these neural correlates during all three memory conditions, in comparison to activation measured in healthy participants. Even though profiles of detail counts were most similar between controls and RTLE participants in the Narrative condition, the largest group difference in activation within regions sensitive to perceptual memory content was observed in the Narrative condition, suggesting that RTLE participants relied more exclusively on brain regions that support conceptual rather than re-experiential memory representations in that condition. The widespread pattern of decreased activation observed in the RTLE group across memory conditions is consistent with evidence that hippocampal damage disrupts activity within the entire AM network (Addis, 2005; Addis, Moscovitch, & McAndrews, 2007; Giannoylis, Lin, & McAndrews, 2011, May), and it may provide a neural mechanism through which focal MTL damage can lead to the paucity of perceptual details I observed behaviourally in individuals with mtle. Together, results from the studies discussed here offer complementary evidence that the hippocampus is a hub within a network of brain regions that supports the retrieval and integration of perceptual and other event-specific memory features.

140 CHAPTER 5 GENERAL DISCUSSION 128 With my task, I also observed activity among neural regions known to play a role in memory retrieval whose activity was not modulated by perceptual memory content. These regions were significantly activated across the different memory tasks, and did not differ between the narrative and the clip condition. They included the left inferior frontal gyrus and temporal pole, the left lateral temporal cortex and the left angular gyrus. Some of these structures have been shown to be activated during the retrieval of episodic as well as semantic memory, and their activation at retrieval is resilient to normal, healthy aging (Burianova & Grady, 2007; Burianova, McIntosh, & Grady, 2010; Nyberg, Forkstam, Petersson, Cabeza, & Ingvar, 2002; Nyberg, et al., 2003; Rajah & McIntosh, 2005; St-Laurent, Abdi, Burianova, & Grady, 2011). In participants with RTLE, activation was decreased among most regions within the AM retrieval network, but activation among this particular set of regions, namely, the left angular gyrus, left temporal pole and left lateral temporal cortex, was observed consistently in all conditions, suggesting that their recruitment may be less readily compromised by focal right MTL damage. Activity in these regions may reflect retrieval processes such as cueing, search, and semantic processing, that are not as sensitive to perceptual memory content as those supported by temporal and posterior cortical regions, but that may work in orchestration with regions sensitive to perceptual richness to reconstruct recollective memories at retrieval. 5.2 Retrieving Naturalistic and Laboratory Events As mentioned, the paradigm I designed captured the complexity and richness of AM successfully, while benefitting from the experimental control provided by events created in the laboratory. My laboratory tasks were controlled for several factors of no interest to my research questions, but they also elicited patterns of brain activation that mimicked those observed during naturalistic autobiographical memory retrieval. These results are insightful given evidence that several laboratory episodic memory tasks activate brain regions that overlap minimally with those typically engaged by AM retrieval (Gilboa, 2004; McDermott, Szpunar, & Christ, 2009). Importantly, most laboratory episodic memory tasks are based on recognition paradigms, while AM is typically tested with recall paradigms, which may contribute to systematic differences between the two memory types. While the neural signature of AM

141 CHAPTER 5 GENERAL DISCUSSION 129 typically includes ventromedial prefrontal, medial parietal and medial temporal regions (Cabeza & St Jacques, 2007; Maguire, 2001a; Svoboda, McKinnon, & Levine, 2006), meta-analyses have shown that laboratory tasks consistently engage lateral frontal and lateral parietal cortical regions, as well as the anterior cingulate cortex and the precuneus (McDermott, et al., 2009; Spaniol, et al., 2009). McDermott and colleagues (2009) also found that very few brain areas, some of which included the right thalamus, the posterior cingulate cortex and the left inferior frontal cortex, overlapped between autobiographical and laboratory memory tasks. Except for the left parahippocampal gyrus, no medial temporal regions were found to be consistently activated by laboratory tasks, although a rich literature indicates that the hippocampus becomes engaged when emphasis is on the retrieval of contextual information (Slotnick, 2010), the experience of recollection (Cohn, Moscovitch, Lahat, & McAndrews, 2009; Spaniol, et al., 2009), or both (Eichenbaum, Yonelinas, & Ranganath, 2007; Yonelinas, Aly, Wang, & Koen, 2010). Importantly, discrepancies between the neural correlates of AM and laboratory episodic memory tasks can be mitigated when retrieval format is made more similar between the two conditions. For example, significant overlap has been observed between AM and laboratory tasks when both memory types are tested with multiple-choice questions (Burianova & Grady, 2007; St-Laurent, Abdi, et al., 2011; Stokes, Mazuz, Daselaar, Moscovitch, & Cabeza, 2011, April), with recognition paradigms using picture cues (Cabeza, et al., 2004), or when laboratory events are made more complex and take longer to retrieve (Stokes, et al., 2011, April). In my paradigm, several factors contributed to the tremendous overlap between AM and the laboratory conditions that was revealed with a conjunction analysis. First, the laboratory events were designed to resemble AM in their complexity, their narrative structure (Radvansky, Copeland, & Zwaan, 2005), and, in the case of film clips, their multi-modal imagery content. Also, both AM and laboratory events were cued with a title, and were recalled over the same time period. As expected, AM overlapped with the multimodal film clip condition to a greater extent than with the perceptually impoverished narrative condition. Although overlap was substantial, I still observed significant differences between AM and the two laboratory conditions in my paradigm. AM lead to significantly greater activation

142 CHAPTER 5 GENERAL DISCUSSION 130 among regions known to be part of the default mode network (DMN; Buckner, Andrews-Hanna, & Schacter, 2008; Gusnard, Akbudak, Shulman, & Raichle, 2001; Raichle, et al., 2001), while the laboratory events generated relatively greater activation among lateral frontal and parietal regions known to play a role in attention, and whose activity is anti-correlated with the DMN during rest (Fox, et al., 2005; Toro, Fox, & Paus, 2008). In other words, the pattern of distinction between laboratory and naturalistic tasks described by McDermott and colleagues (2009) was attenuated rather than eradicated in my data. As an example, McDermott reported distinctions along the posterior cortical midline that I also observed: the anterior portion of the precuneus and the posterior cingulate cortex were relatively more activated by AM (and, to a lesser extent, by the clip condition), while a more posterior portion of the precuneus was more activated by the two laboratory conditions. Similar patterns of overlap and differentiation between laboratory and personal memory episodes were also reported by Burianova and Grady (2007), who identified a common retrieval network, in addition to an activation profile within regions from the DMN that was unique to AM. Distinctions between the neural correlates of personal and laboratory memories may reflect the greater self-relevance of AM, and the greater sense of self-projection (Buckner & Carroll, 2007; Spreng, Mar, & Kim, 2009) it elicits at retrieval, as medial prefrontal and parietal regions that form the DMN are typically engaged by personally relevant material (Andrews- Hanna, et al., 2010; Cabeza, et al., 2004; Rabin, et al., 2010; St Jacques, et al., 2011; Szpunar, et al., 2007; Vann, et al., 2009). Also, it is possible that the evaluation and verification of personal memory is more intuitive (based on a feeling of rightness; Gilboa, 2004), and that the evaluation of laboratory events is more objective and accuracy oriented. Thus, both attentional and self-referential processes may underlie the patterns of brain activation that distinguish personal and laboratory episodic memory retrieval. Despite task differences, the degree of overlap between my conditions was substantial, and it validates the use of short laboratory stories as an alternative to naturalistic memories when assessing the neural correlates of event memory. Film clip retrieval was especially successful at engaging medial temporal and posterior cortical regions activated by the multimodal content of real-life episodes.

143 CHAPTER 5 GENERAL DISCUSSION Lesion Specificity: Hippocampus and the Medial Temporal Lobe While it is well established that mtle affects the integrity of the hippocampus, postsurgery mtle participants also had temporal lobe tissue resected outside the hippocampus proper, which included the amygdala, temporal pole and lateral and medial temporal cortex. In the behavioural study, most participants with mtle were post-surgery cases. However, preliminary analyses indicated no significant group differences based on surgical status, even at liberal levels. This observation is consistent with previous work in other cohorts of patients with mtle from my laboratory and from other groups (Herfurth, Kasper, Schwarz, Stefan, & Pauli, 2010; St-Laurent, et al., 2009; St-Laurent, Moscovitch, Tau, & McAndrews, 2011; Viskontas, McAndrews, & Moscovitch, 2000), who reported an absence of difference between pre and post-surgery cases on AM tasks. For example, Herfurth and colleagues (2010) reported a surgery-related loss of AM specificity for childhood, but not recent memories, in LTLE, and no surgery-related difference in RTLE. In a previous study, my colleagues and I also showed that the number of perceptual details produced during AM recall was not affected by surgical status (St-Laurent, et al., 2009). A volumetric study conducted by Rosenbaum and colleagues (2008) in patients with amnesia caused by infections (e.g., herpes encephalitis) showed that hippocampal volume loss was a better predictor of AM performance than neocortical MTL damage, although Noulhiane and colleagues (2007) reported that both spared hippocampal and medial temporal cortical volumes predicted AM performance in mtle patients who had undergone a temporal lobe resection. Note however that for the latter study, pre-surgical disease severity may have influenced both the extent of the resection and memory performance. More evidence of the widespread impact of hippocampal damage on the AM retrieval network comes from a longitudinal fmri study assessing the neural correlates of AM retrieval in a large sample of individuals with left- and right-lateralized mtle scanned before and after a unilateral temporal lobe resection. Surprisingly, the study revealed minimal changes in patterns of brain activation pre and post-surgery (Giannoylis, et al., 2011, May; McAndrews, In Press). This finding is concordant with the lack of impact of surgical status on behavioural performance during AM tasks, and with my own observation of a wide-spread reduction in activation among brain regions that support detailed memory retrieval in pre-surgical RTLE patients (see also

144 CHAPTER 5 GENERAL DISCUSSION 132 Addis, Moscovitch, et al., 2007 for evidence from LTLE cases). Together, this cumulative evidence suggests that the more restricted form of MTL damage present in pre-surgery mtle cases was sufficient to induce the paucity of details I observed in the behavioural study. Of course, although the most salient and consistent damage is found within the hippocampus proper (Keller & Roberts, 2008; Mathern, et al., 1996; Moran, Lemieux, Kitchen, Fish, & Shorvon, 2001), one must be mindful that mtle also leads to structural brain damage outside the hippocampus and medial temporal cortex. Within the past decade, improved imaging technology has contributed to a better mapping of the extra-temporal damage that accompanies mtle. Evidence from voxel-based morphometry and from diffusion tensor imaging has revealed grey matter atrophy and white matter abnormalities outside the temporal lobe in patients with mtle (Focke, et al., 2008; Gross, 2011; Gross, Concha, & Beaulieu, 2006; Keller & Roberts, 2008; Moran, et al., 2001), which could account for some of the cognitive changes associated with their condition (e.g., see Bell, Lin, Seidenberg, & Hermann, 2011, for a review). In my studies, I attempted to minimize these non-specific effects by recruiting participants with IQ within the normal range, with no or little visible structural brain damage outside the MTL, and whose language organization in the brain was typical (all but two participants tested behaviourally had language lateralized to the left hemisphere), all factors negatively associated with widespread unspecific changes in mtle (Bell, et al., 2011). Nonetheless, the brain operates as a network, and even focal lesions can lead to widespread changes in activation and metabolism in structurally intact afferent and efferent structures (e.g., Vann & Albasser, 2009; Vann, Brown, Erichsen, & Aggleton, 2000). Lesion studies have their limitations, and we gain from combining them with brain imaging studies conducted in people without neurological damage. In the current case, I observed that mtle disrupts activity among regions specifically identified to be sensitive to the perceptual richness of episodic memory in healthy controls, which included the hippocampus proper. Combined with the paucity of perceptual details I observed behaviourally in mtle participants, the brain imaging evidence from patients and controls provides converging support to my claim that the hippocampus is a key player in the retrieval of perceptually enriched memory episodes.

145 CHAPTER 5 GENERAL DISCUSSION Impact of mtle on Encoding versus Retrieval While the behavioural results clearly indicated a relationship between MTL function and the perceptual richness of memory episodes, it could not determine with certainty whether MTL lesions disrupted memory for perceptual features at encoding, at retrieval, or both. In the narrative and clip conditions, participants with mtle performed both encoding and retrieval with a defective MTL, and it is likely that their pathology affected both processes. With the exception of childhood and youth memories in individuals with late onset mtle, encoding was also performed with an epileptogenic MTL in the AM condition. It is well known that damage to the MTL interferes with the acquisition of new memories (Eichenbaum, Otto, & Cohen, 1992; Scoville & Milner, 1957; Squire, 1992), and evidence also suggests that it disrupts working memory tasks that require forming associations between sets of items, and between items and their context (Hannula & Ranganath, 2008; Hannula, Tranel, & Cohen, 2006; Henke, 2010; Nadel & Hardt, 2011; Ranganath, 2010). In the fmri study, the encoding portion of the paradigm was performed outside the scanner, and so I could not assess the hippocampus contribution to subsequent retrieval. However, imaging performed in healthy participants provided definitive evidence that the hippocampus was involved during the retrieval of perceptually enriched memories. Despite this finding, however, I cannot rule out that the decreased activity observed among regions sensitive to perceptual richness in individuals with RTLE could be explained at least partially by an encoding deficit. For example, studies during which people with mtle underwent fmri while encoding visual scenes have reported asymmetric activation that reflects dysfunction in the epileptogenic MTL, and that is linked with MTS and with poor memory performance (Binder, et al., 2010; Detre, et al., 1998; Mechanic-Hamilton, et al., 2009; Rabin, et al., 2004; Vannest, Szaflarski, Privitera, Schefft, & Holland, 2008). Nevertheless, two different pieces of experimental evidence contribute to my conviction that hippocampal damage interferes with the retrieval of perceptual memory features. Firstly, Viskontas and colleagues (2000) have compared memory for personal events from childhood and teenage years between individuals with childhood- and adult-onset mtle. Even though participants with adult-onset epilepsy were free of recurrent seizures at the time of encoding,

146 CHAPTER 5 GENERAL DISCUSSION 134 their AM was just as poor as AM from participants with childhood-onset seizures. In general, age of onset is a poor predictor of AM performance in the mtle literature (Bergin, Thompson, Baxendale, Fish, & Shorvon, 2000; Lah, Grayson, Lee, & Miller, 2004; Noulhiane, et al., 2007; Voltzenlogel, et al., 2006). In addition, retrograde AM deficits have been reported in individuals with MTL damage that is the result of adult-onset trauma or infection (Cipolotti, et al., 2001; Gilboa, et al., 2006; Kapur, 1999; Rosenbaum, et al., 2005; Rosenbaum, et al., 2008; Steinvorth, Levine, & Corkin, 2005). While perceptual memory content was not assessed in these studies per se, qualitative AM deficits that reflected a paucity of memory details were reported. The fact that these individuals brain was intact at encoding suggests that damage to the MTL at retrieval is sufficient to induce the kind of memory deficit I observed on my AM and laboratory episodic memory tasks. Secondly, amnesic individuals with bilateral hippocampal damage have been reported to struggle with the mental simulation of future or novel experiences, and to imagine scenes or events that lack details and coherence (Hassabis, Kumaran, Vann, & Maguire, 2007; Kwan, Carson, Addis, & Rosenbaum, 2010; Tulving, 1985, 2002; but see Maguire, Vargha-Khadem, & Hassabis, 2010). These kinds of mental constructs, which rely on the generation and binding of imagined features, have been shown to involve many of the same processes and brain regions as the retrieval of autobiographical memory episodes, and to engage the hippocampus proper (Addis, Wong, & Schacter, 2007; Buckner, 2010; Buckner & Carroll, 2007; Hassabis & Maguire, 2009; Rabin, et al., 2010; Schacter & Addis, 2009). In other words, hippocampal damage 1) disrupts construction mechanisms crucially engaged during episodic memory retrieval, and 2) is sufficient to cause significant AM impairment when present only at retrieval. Based on this evidence and on my own fmri findings, it is likely that mtle interfered with the recall and integration of perceptual memory features during the retrieval portion of my task, regardless of whether it also affected encoding. In the current study, the length of the protocol prevented me from scanning both at encoding and at retrieval. However, future studies should measure brain activity during the encoding and retrieval of complex laboratory events in individuals with MTL damage in order to portray fully how their pathology affects the relationship between brain activation and behavioural performance.

147 CHAPTER 5 GENERAL DISCUSSION Laterality of Hippocampal Function Functional MRI in healthy participants indicated that regions sensitive to perceptual richness were strongly lateralized to the right hemisphere. Based on this observation, one would expect right-lateralized mtle to be more detrimental to perceptual memory content than left-lateralized mtle, but my behavioural study revealed a deficit for perceptual memory details that was indistinguishable between the two groups. This discrepancy in the results is difficult to explain. In a group of participants with RTLE, brain imaging indicated reduced activation among right-lateralized regions sensitive to richness, including the right hippocampus. Although these neural changes are consistent with the behavioural decrease in perceptual memory content observed in mtle, pathology localization, rather than loss of perceptual memory content, probably accounted for the fact that these neural changes were right-lateralized (see also Giannoylis, et al., 2011, May). More specifically, results from my studies cannot explain how LTLE affects the recruitment and activation of right-lateralized neural correlates of perceptual richness, which I presume underlies their behavioural deficit in perceptual details. Although recruitment limitations prevented me from scanning participants with LTLE, fmri studies conducted in individuals with LTLE during AM retrieval (Addis, Moscovitch, et al., 2007; Giannoylis, et al., 2011, May; McAndrews, In Press) indicate that damage to the left MTL leads to a widespread reduction in activation that affects the entire AM network, including right-lateralized MTL and posterior cortical regions sensitive to perceptual memory content. Recent work by Giannoylis and colleagues (2011, May) showed that both left and right unilateral mtle lead to a bilateral decrease in activation throughout the AM retrieval network, although greater reductions are observed in the epileptogenic MTL. In my RTLE group, I observed greater changes in the right hemisphere, but I also observed reduced activation in the contralateral MTL. Although no significant differences between my control and my RTLE group were observed in the left hippocampus proper in the AM condition, positive findings may have emerged with greater statistical power. What these studies indicate is that unilateral damage to either the left or the right MTL is sufficient to interfere with right MTL activation and with the recruitment of right-lateralized posterior cortical regions that support the representation of

148 CHAPTER 5 GENERAL DISCUSSION 136 perceptual memory content. Given how important levels of cross-talk take place between the two hippocampi during AM retrieval (Addis, McIntosh, Moscovitch, Crawley, & McAndrews, 2004; Addis, Moscovitch, et al., 2007), it is plausible that MTL damage in either hemisphere has a similar impact on the retrieval of perceptual details, even if such details are represented overwhelmingly in the right hemisphere. If this interpretation is supported by evidence, it could explain how mtle lateralization had no impact on memory performance in my behavioural study. Nevertheless, my fmri results indicate that the right hippocampus is more directly involved in the retrieval of perceptual memory content than its contralateral counterpart. I speculated that the left MTL may play a more central or dominant role in accessing and establishing AM than the right MTL. Typically, AM retrieval elicits a pattern of activation that is either left lateralized or bilateral (Cabeza & St Jacques, 2007; Maguire, 2001a; Svoboda, et al., 2006). For example, fact-checking tasks (e.g., you were the matron-of-honour at your sister s wedding) generate a left-lateralized pattern of activation in healthy individuals (Maguire, 2001a, 2001b), but bilateral activation is more commonly reported when AMs are more recent, more emotional, or when more time is allotted to retrieve the memory (Cabeza & St Jacques, 2007; Maguire & Frith, 2003; Svoboda, et al., 2006). One possibility is that the left MTL is essential to access and reconstruct an episode, while the right MTL contributes to AM by adding flavor, or experiential qualities, to the retrieved memory (Maguire, 2001b; Spiers, et al., 2001). In support for this view, evidence from patients who have received a callosotomy, a surgical procedure that disconnects the two hemispheres almost entirely to control seizures, indicates that the left hemisphere can draw more elaborate inferences about the logical and causal structure of events than the right hemisphere (Roser, Fugelsang, Dunbar, Corballis, & Gazzaniga, 2005), which is essential to the construction of a coherent episodic memory representation. Thus, while the right MTL supports the retrieval of perceptual memory features, it is possible that the left MTL acts as a bottle-neck, so that perceptual details cannot be retrieved if the event s storyline is not properly reconstructed. If this explanation is correct, damage to either the right or the left MTL should lead to a reduction in perceptual memory content.

149 CHAPTER 5 GENERAL DISCUSSION 137 This hypothesis needs to be tested experimentally, as support from the literature is mixed. Electroencephalography performed during AM retrieval indicates a shift from the left to the right hemisphere over the time course of remembering (Conway, Pleydell-Pearce, & Whitecross, 2001; Conway, Pleydell-Pearce, Whitecross, & Sharpe, 2003). More specifically, activity appears to shift from an early left-lateralized anterior component that seems to involve the inferior frontal gyrus and temporal pole, to a late right-lateralized posterior component, indicating that retrieval is initiated by the left hemisphere. The authors could not measure subcortical activation with EEG, but it is possibly that this switch from the left to the right hemisphere also takes place in the MTL during retrieval. Also, the authors used conceptual cue words to elicit memory retrieval, and cues that probe perceptual memory content, such as pictures of the event, may facilitate the early engagement of the right hemisphere through bottom-up processes (Conway & Loveday, 2010; Loveday & Conway, 2011). New AM paradigms that make use of pictures taken by cameras worn by the participants (Browne, et al., 2011; Loveday & Conway, 2011; St Jacques, Conway, & Cabeza, 2011), as well as paradigms that modulate the time needed to access a memory representation (Addis, Knapp, Roberts, & Schacter, 2012) may help to answer some of these questions in the future. A few studies have used paradigms during which AM retrieval trials were split between an early construction phase, during which an AM was accessed, and a later elaboration phase, during which participants attempted to relive that AM in as many details as possible (Addis, Wong, et al., 2007). Typically, hippocampal activation was reported to peak in both hemispheres during the early construction phase (Daselaar, et al., 2008; Rabin, et al., 2010; Schacter & Addis, 2009; St Jacques, et al., In Press), challenging the idea that the left hippocampus becomes engaged before the right hippocampus, although fmri may lack the temporal resolution to capture rapid interactions between heavily interconnected structures such as the two hippocampi. However, it is also possible that both hippocampi are essential during construction, with the right hippocampus promoting access to perceptual memory content by recruiting posterior cortical regions that support its representation. Daselaar et al. (2008) showed that right hippocampal activation peaked when posterior cortical regions, which included visual and auditory cortices and the precuneus, became engaged, but declined while

150 CHAPTER 5 GENERAL DISCUSSION 138 activation remained sustained in the posterior cortical regions through elaboration. When memory is vivid and detailed, sustained right hippocampal activation can also be observed through the elaboration phase, possibly because participants engage in a reiterative retrieval process that leads to a continuous flow of recollection (St Jacques, et al., In Press). Clearly, more work is needed in order to settle issues of hemispheric specialization and AM representation. The current data advance our understanding of the role played by the hippocampus in AM by documenting interesting differences between the two hemispheres, which have been observed with simpler laboratory tasks, but are less well documented in the context of AM. Future work should probe these issues further, by assessing the impact of damage to the left MTL on the recruitment of right-lateralized regions sensitive to perceptual richness during AM retrieval. Also, effective connectivity should address the directionality of influence between the two hippocampi, both in healthy controls and in individuals with damage to the MTL. In patients with mtle implanted with depth electrodes for diagnostic purposes, intracranial EEG could provide the fine-grained temporal-resolution needed to quantify the interaction between the two hemispheres during AM retrieval. 5.6 Age of the Memory Theories of hippocampal function make different predictions about hippocampal involvement at retrieval depending on the age of the memory. Consolidation theory predicts that older memories can be retrieved independently from the hippocampus (Squire, 1992; Squire, et al., 2010; Squire & Wixted, 2011), whereas multiple trace theory (MTT; Moscovitch & Nadel, 1998; Moscovitch, et al., 2005; Nadel & Moscovitch, 1997) predicts that memory characteristics such as re-experiential features determine hippocampal engagement rather than age. Consistent with MTT, studies have shown that level of details, vividness, and other AM features can account for the reduced hippocampal activity observed during the recall of older memories (Addis, Moscovitch, et al., 2004; Gilboa, et al., 2004; St Jacques, et al., In Press; Viard, et al., 2007). Although I did not assess whether AM recency modulates hippocampal activity in the current study, I observed a striking lack of difference in levels of hippocampal activity between the two perceptually enriched conditions, AM and Clip, in a group of healthy

151 CHAPTER 5 GENERAL DISCUSSION 139 participants. While AMs ranged in age from one year to several decades old, memories for clips were much more recent, as they were encoded approximately minutes before participants underwent scanning. Despite this large difference in memory recency between the two conditions, only a small cluster of left parahippocampal voxels showed greater activation in the AM than in the clip condition across the entire MTL. These similar levels of medial temporal activation between the two conditions, both of which engaged the hippocampus to a greater extent than the perceptually impoverished narrative condition, provide clear evidence that the nature of the memory representation, rather than the memory s age, is what determines the engagement of the hippocampus. 5.7 Role of the Hippocampus in Memory Convergent evidence from different studies presented in this thesis reveals that the hippocampus plays a central role in the retrieval and integration of perceptual and other context-specific memory details into complex memory episodes, personal or otherwise. This is an important contribution to our understanding of the role of the hippocampus in memory. We know that long-term declarative memory retrieval s reliance on the hippocampus depends on the characteristics of the memory. As mentioned, MTT predicts that recollective memory is hippocampally-dependent no matter its age, while semantic memory can be retrieved successfully from the neocortex (Moscovitch & Nadel, 1998; Moscovitch, et al., 2005; Nadel & Moscovitch, 1997). Cumulative evidence from clinical cases and from brain imaging studies supports this theory (Gilboa, et al., 2006; Rosenbaum, et al., 2005; Rosenbaum, et al., 2008; Vargha-Khadem, et al., 1997; Viard, et al., 2007; Westmacott, Leach, Freedman, & Moscovitch, 2001). Research has also shown that some of the core characteristics that distinguish episodic memory from semantic memory, such as the retrieval of contextual details, and the experience of recollection (Moscovitch, et al., 2005; Nadel & Moscovitch, 1997; Tulving, 1972, 1984, 1985, 2002), engage the hippocampus (Addis, Moscovitch, et al., 2004; Aggleton & Brown, 2006; Dickerson & Eichenbaum, 2010; Eichenbaum, et al., 2007; Gilboa, et al., 2004; Hassabis, Kumaran, Vann, et al., 2007; Ranganath, 2010; Rosenbaum, Gilboa, Levine, Winocur, &

152 CHAPTER 5 GENERAL DISCUSSION 140 Moscovitch, 2009; St-Laurent, et al., 2009; St-Laurent, Moscovitch, et al., 2011; Yonelinas, et al., 2010). I hypothesized that perceptual richness is a major determinant of hippocampal engagement, because it captures both of these characteristics: details and recollection. Perceptual richness reflects the retrieval of sensory-based memory details, a subcategory of context-specific memory elements that depict one s sensory experience at encoding (Conway, 2009; Conway & Loveday, 2010; Larsen, 1998; Levine, Svoboda, Hay, Winocur, & Moscovitch, 2002), and that contributes to one s impression of reliving the past by rendering memory rich and life-like (Brewer, 1986, 1995; Conway, 2009; Rubin, et al., 2003). My behavioural and my brain imaging findings confirmed this hypothesis. Furthermore, by revealing a link between hippocampal function and the perceptual richness of episodic memory, I identified a potential cognitive mechanism through which recollection engages the hippocampus at retrieval. The retrieval of perceptual memory features facilitates recollection, and it depends on the hippocampus; damage to the hippocampus should therefore disrupt recollection by causing a paucity of perceptual details, which is what was observed in the current results. Recently, others have shown that the retrieval of episodic details contributes to one s subjective sense of recollection (Dudukovic & Knowlton, 2006; Noulhiane, et al., 2007; Piolino, et al., 2003), and mediates the relationship between hippocampal activation and recollection at retrieval (Hannula & Ranganath, 2009; Slotnick, 2010; but see the meta-analysis by Spaniol, et al., 2009, who report that hippocampal activation is more consistently predicted by subjective than by objective measures of recollection). My data support and expand on these results and on MTT, by demonstrating how the types of memory details that give rise to recollection, rather than any type of episodic details, depend especially on the integrity and engagement of the hippocampus at retrieval. My findings are also consistent with Martin Conway s theory of episodic memory (Conway, 2009; Conway & Loveday, 2010) which states that episodic memory is composed of experience-near episodic elements, EEs, which are often under the form of visual images. EEs are bundled together by a conceptual frame that provides the memory with its meaning or gist. The perceptual details that were measured with my task, as well as highly specific story details

153 CHAPTER 5 GENERAL DISCUSSION 141 (e.g., specific actions), fit under the definition of EEs. Conway suggests that the frame is supported by a fronto-temporal brain network, while EEs are supported by a temporo-occipitoparietal network. This is consistent with my brain imaging results, which reveal that medial temporal, lateral occipital, and lateral and medial parietal regions are sensitive to the perceptual content of episodic memory. Conway also predicts that, should damage to this temporo-occipito-parietal network occur, one would observe a dramatic loss of EEs, so that the gist or frame is accessed in the absence of experiential details (Conway & Loveday, 2010; Greenberg, Eacott, Brechin, & Rubin, 2005; Greenberg & Rubin, 2003; Rubin & Greenberg, 1998). During my task, memories for clips and narratives were cued at the level of the frame, using a title that reflected their gist (e.g., Crashing the bicycle ). Consistent with Conway s prediction, perceptual details and specific story elements were drastically reduced in individuals with mtle. In the AM condition, I also observed that story details were more resilient than perceptual details to MTL damage (see also St-Laurent, et al., 2009). Those AMs corresponded to events spreading over a timeline of a few minutes up to several hours, which was longer than the laboratory memories timeline, and many AM elements scored as story details would have corresponded to conceptual frames rather than to EEs, following Conway s definition (Conway, 2009). Conway proposes that episodic memories are reconstructed from long-term knowledge (Conway, 1995; Conway & Pleydell-Pearce, 2000), and that the integration of EEs into a recollective memory experience is based on the interaction between temporal, parietal and occipital regions (Conway, 2009; Conway & Loveday, 2010). Based on this model and on experimental evidence, including mine, one could think of the human hippocampus as forming a bridge between the frame and the EEs during episodic memory retrieval. Several models of episodic memory suggest that EEs are not stored in the MTL per se, but rather are indexed, accessed and integrated into a memory through the actions of the MTL (Buckner, 2010; Conway & Loveday, 2010; Greenberg & Rubin, 2003; Hassabis & Maguire, 2007, 2009; McClelland, McNaughton, & O'Reilly, 1995; Moscovitch, et al., 2005; Teyler & DiScenna, 1986; Teyler & Rudy, 2007). Storage may instead take place in posterior cortical regions, as evidenced in cases of severe retrograde amnesia from patients with extensive bilateral posterior regions that

154 CHAPTER 5 GENERAL DISCUSSION 142 include visual cortices (Greenberg, et al., 2005; Greenberg & Rubin, 2003; Rubin & Greenberg, 1998). I suggest that the hippocampus plays the functional role of linking EEs to conceptual long-term memory knowledge, at encoding through its well documented role in the formation of new memories, and at retrieval by supporting the assemblage of experiential memory elements into recollective memory episodes (see also Moscovitch, submitted, for a related view). 5.8 Nature of the Memory Representation My paradigm contrasted memory for audio-visual film clips to memory for written narratives. I made different assumptions about the nature of the memory representation in both these and the AM task condition. For example, I assumed that AMs had a strong imagery component, based on previous evidence based on similar paradigms (Levine, et al., 2002; Rubin, et al., 2003; St-Laurent, et al., 2009; Viard, et al., 2007). I also assumed that, in healthy controls, clips had greater imagery content, and were richer in visual images, spatial information, sounds, motion, texture and colours, than narratives, which I assumed to be perceptually impoverished. Crucially, these assumptions were confirmed by the respective detail profiles of the task conditions. However, I did not speculate whether the nature of the narratives representation was schematic but imagery-based, imageless and purely conceptual, or language-based. In the literature, Brewer (1986; 1995) makes a distinction between memory for life episodes that is based on imagery (recollective memories) and memory that is conceptual or propositional (autobiographical facts), although he also recognizes that recollective memories have some propositional content, and vice versa (see also Greenberg & Verfaellie, 2010). This distinction between recollective and factual memory is influenced by the work of philosophers such as Broad (1925), who distinguished between image-based recollective memory, skill or rote implicit memory, and propositional memory, the conscious representation of knowledge in a non-image form (Brewer, 1995, but see Pylyshyn, 2003). In its early formulation, propositional memory was considered conceptual, logical, reflective, and language-based, although some contemporary theorists suggest that abstract memories can be represented in a conceptual

155 CHAPTER 5 GENERAL DISCUSSION 143 manner that is non-verbal (a sort of mentalese ; e.g., Conway, Singer & Tagini, 2004). In the current study, I quantified how perceptual, but not how abstract or conceptual a memory was, and I can make more definitive conclusions about perceptual than about conceptual content in my conditions. Of note, the narrative condition elicited some imagery in most participants, controls and mtle, indicating that memory for narratives was not process-pure. My paradigm could not determine whether this imagery was elicited mostly at retrieval, or whether it was elicited at encoding and then became integrated into the memory representation. Because narratives were presented verbally, it is likely that some of them were recalled verbatim, in full or in part, while others were recalled in a non-literal conceptual manner. As a side note, participants with left-lateralized mtle typically suffer from a verbal memory deficit (Djordjevic, et al., 2010; Kalviainen, et al., 1997; Kilpatrick, et al., 1997; Stewart, et al., 2009). Although no difference in performance was observed between participants with right and left mtle in the narrative condition, it is possible that narrative recall was less verbatim in the left mtle group, which would be interesting to assess in a future analysis. More importantly, perceptual content was equally low in the Clip and the Narrative condition in individuals with mtle, suggesting that their memory for both clips and narratives was equally gist-like (Winocur & Moscovitch, 2011; Winocur, Moscovitch, & Bontempi, 2010). Brain imaging also revealed that neural structures implicated in semantic memory and longterm memory retrieval processes, but not regions engaged by perceptual memory content, were activated during narrative retrieval in individuals with RTLE. Although less pronounced, a similar pattern of results, namely, greater reduction in activity among regions supporting perceptual memory content than among those supporting semantic memory retrieval, was also observed in the Clip condition. These findings support claims that memory is more gist-like in individuals with mtle (see also St-Laurent, et al., 2009; St-Laurent, Moscovitch, et al., 2011) and in other individuals with damage to the MTL (Rosenbaum, et al., 2009), than in healthy controls. 5.9 The Role of Imagery in Episodic Memory I showed that the hippocampus is essential to the integration of imagery-based content into memory representations of past episodes. This contribution to human memory is adaptive:

156 CHAPTER 5 GENERAL DISCUSSION 144 Brewer (1995) suggests that recollection could have evolved to provide adaptive insight into the workings of one s own memory. Research shows that a memory s imagery content is associated with its accuracy, and with one s confidence in its accuracy (Brewer, 1995; Brown & Kulik, 1977; Johnson, 2006; Mitchell & Johnson, 2009; Neisser, et al., 1996; Rubin, et al., 2003; Tulving, 1985; Yonelinas, et al., 2010). While accuracy and confidence can sometimes be dissociated (e.g., see Neisser & Harsch, 1992), in most circumstances the association between imagery, confidence and accuracy reflects a reasonable approximation of reality (Brewer, 1995). Similarly, Conway (Conway, 2009; Conway & Loveday, 2010) suggests that episodic memory serves to keep an adaptive record of recent goal processing (e.g., did I pack my lunch?), and that people use images derived from experience (e.g., I can see myself putting a sandwich in my lunch bag) as specific evidence that certain goals or sub-goals have been achieved. People then use this insight to constrain subsequent goals (e.g., I should eat the sandwich I packed rather than eat out), and thus to plan future actions. Recent work has also revealed similarities between the cognitive mechanisms and neural substrates engaged during recollection, and those that support the anticipation of neverencountered scenes and scenarios (Addis, Wong, et al., 2007; Buckner, 2010; Buckner, et al., 2008; Buckner & Carroll, 2007; Hassabis, Kumaran, & Maguire, 2007; Sheldon, McAndrews, & Moscovitch, 2011; Spreng, et al., 2009). Among other things, both recollection and prospection, the life-like mental simulation of alternative or novel situations, require the assemblage of a mental construct, and some believe that new scenarios are amalgamations of elements that compose past events (Buckner, 2010; Johnson & Sherman, 1990; Suddendorf, Addis, & Corballis, 2009; Suddendorf & Corballis, 1997). Support for this view comes from evidence that patients with amnesia, including hippocampal amnesics, have difficulty anticipating novel or future scenes and scenarios (Buckner, 2010; Hassabis, Kumaran, Vann, et al., 2007; Klein & Loftus, 2002; Kwan, et al., In Press; Suddendorf, et al., 2009; Suddendorf & Corballis, 1997; Tulving, 1985, 2002; but see Hurley, Maguire, & Vargha-Khadem, 2011; Maguire, et al., 2010; Squire, et al., 2010). What these data suggest is that the hippocampus contributes both to the retrieval of past episodes, and to the anticipation of future ones.

157 CHAPTER 5 GENERAL DISCUSSION 145 While my work did not assess the pragmatics of recollection per se, nor its applications to prospective thinking, I show that the hippocampus contributes to the retrieval of imageryladen memory episodes, and this may be one of the mechanisms through which it contributes to anticipating the future, by supporting the rich and life-like simulation of new realities. Bartlett (1932) draws a parallel between the evolutionary advantage provided by far senses such as vision and hearing, and imagery, which allows one to simulate future situations from afar, and thus to anticipate them better. He also suggests that one s capacity to recollect is linked to the propensity to solve problems in a flexible, creative manner, which is supported by recent findings by Sheldon et al., In other words, by supporting the retrieval of perceptually rich memory episodes, the hippocampus may enhance confidence in one s memory, and may provide the cognitive flexibility to anticipate the future and solve problems Limitations Although the body of work presented in the current thesis provides convincing evidence of a relationship between MTL function and the perceptual richness of episodic memory, some methodological and practical limitations must be acknowledge that influenced my results. As discussed, the behavioural measures I collected with my fmri paradigm lacked sensitivity to perceptual memory content, mainly because time restrictions prevented me from probing participants for perceptual details in the same manner they were probed in the behavioural study. It is likely that this factor undermined the statistical power of tests that assessed 1) group differences on these measures, and 2) the parametric modulation of hippocampal activation by these measures. Practicality also prevented me from collecting brain imaging data at encoding, and so the extent to which RTLE disrupted activation during that phase could not be linked to subsequent memory performance on my task. This factor is of importance, and its influence should be assessed in the future. In addition, recruitment limitations prevented me from testing and reporting fmri data from a complete group of participants with LTLE. My RTLE group was also small, but it was sufficiently large to reveal clear patterns of group differences that were consistent with results from my behavioural study.

158 CHAPTER 5 GENERAL DISCUSSION 146 At last, mtle is a condition that evolves gradually over several years, often from the time of an early childhood insult, even though recurrent seizure onset can be delayed until adulthood. Unlike stroke, encephalitis or other lesion-inducing insults with a clear adult onset, mtle is accompanied by slowly evolving neural changes, which vary between individuals. In my mtle groups, factors such as plasticity and compensatory activity are likely to have contributed to memory performance and to patterns of brain activation in ways that are difficult to control for. Thus, although mtle can inform us about the impact of damage to the MTL, there are always limitations to the knowledge one can acquire about the healthy adult brain from patients with lesioned brains. However, combining lesion studies with brain imaging performed in healthy individuals is powerful, as it can provide multi-faceted evidence linking brain activation, brain integrity and cognitive function Future Directions: mtle s Impact on Functional Connectivity The brain imaging results I report are based on contrasts between levels of BOLD signal performed at the voxel level, and I did not assess connectivity between voxels or brain regions. However, voxels correspond to brain regions that form functional networks, and patterns of neural interactions can provide additional information about cognitive function, as well as brain integrity and efficiency. In future analyses, I would like to assess how functional connectivity changes within the different conditions, and how it is affected by mtle. MTLE has been shown to have brain-wide impact on brain connectivity during language and memory tasks (Addis, Moscovitch, et al., 2007; Protzner & McAndrews, 2011; Voets, et al., 2009) as well as during rest (Bettus, et al., 2009). Importantly, changes in effective connectivity within regions that form an AM retrieval network have been reported in patients with left-lateralized pathology (Addis, Moscovitch, et al., 2007), but not yet with right-lateralized pathology. As mentioned, my results have led me to speculate that the left MTL may play a more dominant role in initial retrieval and reconstruction of AM than the right MTL, and I would like to test this hypothesis 1) by assessing the level and direction of influence between the two hippocampi over time during AM retrieval, and 2) by contrasting network connectivity between individuals with right- and left-lateralized mtle. If, as I hypothesized, the left hippocampus plays a greater role in

159 CHAPTER 5 GENERAL DISCUSSION 147 searching and establishing the AM, and the right hippocampus contributes to AM through the recruitment of posterior cortical regions that support perceptual memory content, then the left hippocampus may be a bottleneck in the recruitment of the right hemisphere, and, indirectly, of right hemisphere regions that support perceptual richness during AM retrieval. If so, LTLE s impact on functional connectivity during AM retrieval may be more widespread and bilateral than the impact of RTLE, especially if AM is cued conceptually, as with a title, or indirectly through a thematic cue (Addis, et al., 2012; although see Giannoylis, et al., 2011, May). Additionally, my analyses were designed to assess the neural correlates of perceptual richness, with an emphasis on hippocampal function. However, other memory dimensions differed between my conditions, and I did not explore their neural correlates in great detail. For example, I observed that AM engages the default mode network (Raichle, et al., 2001), which is thought to play a role in self-projection (Buckner & Carroll, 2007), to a greater extent than conditions without personal relevance, which are associated with relatively more activation among regions from the fronto-parietal attentional network (Fox, et al., 2005; Toro, et al., 2008). This observation is consistent with other brain imaging studies that contrasted the neural correlates of self-relevant and impersonal memory tasks and other forms of mental projection, and that revealed greater levels of activation within the medial prefrontal and medial parietal cortex in the personally relevant conditions (Andrews-Hanna, et al., 2010; Cabeza, et al., 2004; Gilboa, 2004; Hassabis, Kumaran, & Maguire, 2007; McDermott, et al., 2009; Rabin, et al., 2010; Spreng & Grady, 2010; St Jacques, Conway, Lowder, et al., 2011; Summerfield, Hassabis, & Maguire, 2009). I also observed activation among mainly leftlateralized brain regions known to play a more general role in long-term declarative memory retrieval, such as the inferior frontal gyrus, the temporal pole, and the lateral temporal cortex (Burianova & Grady, 2007; Burianova, et al., 2010; Nyberg, et al., 2002; Rajah & McIntosh, 2005). Activity among these regions was insensitive to perceptual richness, and it was relatively preserved in participants with mtle. It is likely that regions sensitive to perceptual richness and to personal-relevance, as well as regions that play a more general role in memory retrieval, interact with each other during the retrieval of personal and laboratory events. In the future, I would like to perform functional and effective connectivity analyses to determine how the

160 CHAPTER 5 GENERAL DISCUSSION 148 magnitude and direction of these interactions is shaped by different memory characteristics. More specifically, I would like to explore how regions that form a core memory retrieval network interact with regions that are engaged by personal relevance and perceptual content during my different task conditions, and to assess how these interactions are affected by mtle. Such analyses would help to characterize how the healthy brain generates perceptually enriched memory experiences, and how pathology disrupts these mechanisms and gives rise to memory impairments like the ones I documented in the current body of work Conclusion I presented results from a series of experiments designed to test whether perceptual richness is a determinant of hippocampal engagement at retrieval, and to identify the neural mechanisms through which damage to the MTL disrupts the recall of perceptual episodic memory features. My approach benefitted from the combination of lesion studies and of functional brain imaging. I report convergent evidence that supports my hypothesis that the hippocampus plays a central role in the retrieval and integration of perceptual and other context-specific memory features into enriched recollective memories through its interaction with medial temporal, occipital and parietal brain regions. My results advance our understanding of the neural mechanisms through which the medial temporal lobe supports the recollection of episodic memories. By supporting the retrieval of sensory and fine-grained story details and integrating them into rich memory representations, the hippocampus contributes to the impression that one travels mentally back in time, which characterizes episodic memory. My findings also advance our understanding of the neural correlates of autobiographical memory retrieval. To the best of my knowledge, no consistent evidence has been provided as of yet for hemispheric specialization in the medial temporal lobe during AM retrieval, and my observation of a clear pattern of hemispheric specialization during the retrieval of perceptually enriched memories is a novel and important contribution to the literature. Finally, my results provide evidence for a neural mechanism through which epilepsy from hippocampal origin affects the vividness and experiential feel of memory for personal episodes. I observed that temporal lobe epilepsy disrupts activity among brain regions that contribute to the retrieval of

161 CHAPTER 5 GENERAL DISCUSSION 149 perceptual details, which is consistent with the perceptually impoverished manner in which they retrieve the past. Future work should address how functional connectivity among brain regions engaged during memory retrieval is affected by episodic memory features, and by structural damage in the network. In addition, more work should be directed at characterizing the dynamic interaction between the left and the right hemisphere during retrieval to better understand their unique contribution to autobiographical memory.

162 References Addis, D. R. (2005). Investigating the engagement of the hippocampus and related structures during autobiographical memory retrieval in healthy individuals and temporal lobe epilepsy patients. PhD thesis, University of Toronto, Toronto, Canada. Addis, D. R., Knapp, K., Roberts, R. P., & Schacter, D. L. (2012). Routes to the past: Neural substrates of direct and generative autobiographical memory retrieval. Neuroimage. doi: S (11) [pii] /j.neuroimage Addis, D. R., McIntosh, A. R., Moscovitch, M., Crawley, A. P., & McAndrews, M. P. (2004). Characterizing spatial and temporal features of autobiographical memory retrieval networks: a partial least squares approach. Neuroimage, 23(4), doi: S (04) [pii] /j.neuroimage Addis, D. R., Moscovitch, M., Crawley, A. P., & McAndrews, M. P. (2004). Recollective qualities modulate hippocampal activation during autobiographical memory retrieval. Hippocampus, 14(6), doi: /hipo Addis, D. R., Moscovitch, M., & McAndrews, M. P. (2007). Consequences of hippocampal damage across the autobiographical memory network in left temporal lobe epilepsy. Brain, 130(Pt 9), doi: awm166 [pii] /brain/awm166 Addis, D. R., & Schacter, D. L. (2008). Constructive episodic simulation: temporal distance and detail of past and future events modulate hippocampal engagement. Hippocampus, 18(2), doi: /hipo Addis, D. R., Wong, A. T., & Schacter, D. L. (2007). Remembering the past and imagining the future: common and distinct neural substrates during event construction and elaboration. Neuropsychologia, 45(7), doi: S (06) [pii] / j.neuropsychologia Addis, D. R., Wong, A. T., & Schacter, D. L. (2008). Age-related changes in the episodic simulation of future events. Psychol Sci, 19(1), doi: PSCI2043 [pii] /j x 150

163 151 Aggleton, J. P., & Brown, M. W. (1999). Episodic memory, amnesia, and the hippocampal-anterior thalamic axis. Behav Brain Sci, 22(3), ; discussion Aggleton, J. P., & Brown, M. W. (2006). Interleaving brain systems for episodic and recognition memory. Trends Cogn Sci, 10(10), doi: S (06) [pii] /j.tics Amaral, D., & Lavenex, P. (2007). Hippocampal Neuroanatomy. In P. Andersen, R. Morris, D. Amaral, T. Bliss & J. O'Keefe (Eds.), The Hippocampus Book (pp ). Oxford: Oxford University Press, Inc. Andrews-Hanna, J. R., Reidler, J. S., Sepulcre, J., Poulin, R., & Buckner, R. L. (2010). Functional-anatomic fractionation of the brain's default network. Neuron, 65(4), doi: S (10) [pii] /j.neuron Arruda, F., Cendes, F., Andermann, F., Dubeau, F., Villemure, J. G., Jones-Gotman, M., et al. (1996). Mesial atrophy and outcome after amygdalohippocampectomy or temporal lobe removal. Ann Neurol, 40(3), doi: /ana Barense, M. D., Henson, R. N., & Graham, K. S. (2011). Perception and conception: temporal lobe activity during complex discriminations of familiar and novel faces and objects. J Cogn Neurosci, 23(10), doi: /jocn_a_00010 Barnett, M. P., Newman, H. W., Richardson, J. T., Thompson, P., & Upton, D. (2000). The constituent structure of autobiographical memory: autobiographical fluency in people with chronic epilepsy. Memory, 8(6), doi: / Barr, W. B., Goldberg, E., Wasserstein, J., & Novelly, R. A. (1990). Retrograde amnesia following unilateral temporal lobectomy. Neuropsychologia, 28(3), doi: (90)90018-J [pii] Bartlett, F. C. (1932). Remembering: AStudy in experimental and social psychology. London: Cambridge University Press. Baxendale, S., Thompson, P., Harkness, W., & Duncan, J. (2006). Predicting memory decline following epilepsy surgery: a multivariate approach. Epilepsia, 47(11), doi: EPI810 [pii] /j x 151

164 152 Baxendale, S. A., Thompson, P. J., & Van Paesschen, W. (1998). A test of spatial memory and its clinical utility in the pre-surgical investigation of temporal lobe epilepsy patients. Neuropsychologia, 36(7), doi: S [pii] Baxendale, S. A., van Paesschen, W., Thompson, P. J., Connelly, A., Duncan, J. S., Harkness, W. F., et al. (1998). The relationship between quantitative MRI and neuropsychological functioning in temporal lobe epilepsy. Epilepsia, 39(2), Bell, B., Lin, J. J., Seidenberg, M., & Hermann, B. (2011). The neurobiology of cognitive disorders in temporal lobe epilepsy. Nat Rev Neurol, 7(3), doi: nrneurol [pii] /nrneurol Ben-Yakov, A., & Dudai, Y. (2011). Constructing realistic engrams: poststimulus activity of hippocampus and dorsal striatum predicts subsequent episodic memory. J Neurosci, 31(24), doi: 31/24/9032 [pii] /JNEUROSCI Bergin, P. S., Thompson, P. J., Baxendale, S. A., Fish, D. R., & Shorvon, S. D. (2000). Remote memory in epilepsy. Epilepsia, 41(2), Bernasconi, N., Natsume, J., & Bernasconi, A. (2005). Progression in temporal lobe epilepsy: differential atrophy in mesial temporal structures. Neurology, 65(2), doi: 65/2/223 [pii] /01.wnl fa Bernhardt, B. C., Worsley, K. J., Kim, H., Evans, A. C., Bernasconi, A., & Bernasconi, N. (2009). Longitudinal and cross-sectional analysis of atrophy in pharmacoresistant temporal lobe epilepsy. Neurology, 72(20), doi: 01.wnl f5 [pii] / 01.wnl f5 Bettus, G., Guedj, E., Joyeux, F., Confort-Gouny, S., Soulier, E., Laguitton, V., et al. (2009). Decreased basal fmri functional connectivity in epileptogenic networks and contralateral compensatory mechanisms. Hum Brain Mapp, 30(5), doi: /hbm Binder, J. R., Sabsevitz, D. S., Swanson, S. J., Hammeke, T. A., Raghavan, M., & Mueller, W. M. (2008). Use of preoperative functional MRI to predict verbal memory decline after temporal lobe epilepsy surgery. Epilepsia, 49(8), doi: EPI1625 [pii] /j x 152

165 153 Binder, J. R., Swanson, S. J., Sabsevitz, D. S., Hammeke, T. A., Raghavan, M., & Mueller, W. M. (2010). A comparison of two fmri methods for predicting verbal memory decline after left temporal lobectomy: language lateralization versus hippocampal activation asymmetry. Epilepsia, 51(4), doi: EPI2340 [pii] /j x Bird, C. M., Capponi, C., King, J. A., Doeller, C. F., & Burgess, N. (2010). Establishing the boundaries: the hippocampal contribution to imagining scenes. J Neurosci, 30(35), doi: 30/35/11688 [pii] /JNEUROSCI Birn, R. M., Cox, R. W., & Bandettini, P. A. (2002). Detection versus estimation in eventrelated fmri: choosing the optimal stimulus timing. Neuroimage, 15(1), doi: /nimg S [pii] Bohbot, V. D., Kalina, M., Stepankova, K., Spackova, N., Petrides, M., & Nadel, L. (1998). Spatial memory deficits in patients with lesions to the right hippocampus and to the right parahippocampal cortex. Neuropsychologia, 36(11), doi: S [pii] Bonilha, L., Martz, G. U., Glazier, S. S., & Edwards, J. C. (2012). Subtypes of medial temporal lobe epilepsy: Influence on temporal lobectomy outcomes? Epilepsia, 53(1), 1-6. doi: /j x Brainerd, C. J., & Reyna, V. F. (2001). Fuzzy-trace theory: dual processes in memory, reasoning, and cognitive neuroscience. Adv Child Dev Behav, 28, Brewer, W. F. (1986). What is autobiographical memory?. In D. C. Rubin (Ed.), Autobiographical Memory (pp ). Cambridge, UK.: Cambridge University Press. Brewer, W. F. (1995). What is recollective memory?. In D. C. Rubin (Ed.), Remembering our past: Studies in autobiographical memory (pp. pp ). Cambridge, England: Cambridge University Press. Brown, R., & Kulik, J. (1977). Flashbulb memories. Cognition, 5(1), Browne, G., Berry, E., Kapur, N., Hodges, S., Smyth, G., Watson, P., et al. (2011). SenseCam improves memory for recent events and quality of life in a patient with memory retrieval difficulties. Memory, 19(7), doi: / Buchanan, T. W. (2007). Retrieval of emotional memories. Psychol Bull, 133(5), doi: [pii] /

166 154 Buchanan, T. W., Tranel, D., & Adolphs, R. (2006). Memories for emotional autobiographical events following unilateral damage to medial temporal lobe. Brain, 129(Pt 1), doi: awh672 [pii] /brain/awh672 Buchsbaum, B., Fang, C., Lemire-Rodger, S., & Young, J. (2011, April). Similarity of patterns of neural activity during perception and vivid remembering of short movie clips. Paper presented at the Annual Meeting of the Cognitive Neuroscience Society, San Francisco, CA. Buckner, R. L. (2010). The role of the hippocampus in prediction and imagination. Annu Rev Psychol, 61, 27-48, C doi: /annurev.psych Buckner, R. L., Andrews-Hanna, J. R., & Schacter, D. L. (2008). The brain's default network: anatomy, function, and relevance to disease. Ann N Y Acad Sci, 1124, doi: 1124/1/1 [pii] /annals Buckner, R. L., & Carroll, D. C. (2007). Self-projection and the brain. Trends Cogn Sci, 11(2), doi: S (06) [pii] /j.tics Burianova, H., & Grady, C. L. (2007). Common and unique neural activations in autobiographical, episodic, and semantic retrieval. J Cogn Neurosci, 19(9), doi: /jocn Burianova, H., McIntosh, A. R., & Grady, C. L. (2010). A common functional brain network for autobiographical, episodic, and semantic memory retrieval. Neuroimage, 49(1), doi: S (09) [pii] /j.neuroimage Burnstine, T. H., Lesser, R. P., Hart, J., Jr., Uematsu, S., Zinreich, S. J., Krauss, G. L., et al. (1990). Characterization of the basal temporal language area in patients with left temporal lobe epilepsy. Neurology, 40(6), Cabeza, R., Ciaramelli, E., Olson, I. R., & Moscovitch, M. (2008). The parietal cortex and episodic memory: an attentional account. Nat Rev Neurosci, 9(8), doi: nrn2459 [pii] /nrn2459 Cabeza, R., Mazuz, Y. S., Stokes, J., Kragel, J. E., Woldorff, M. G., Ciaramelli, E., et al. (2011). Overlapping parietal activity in memory and perception: evidence for the attention to memory model. J Cogn Neurosci, 23(11), doi: /jocn_a_

167 155 Cabeza, R., Prince, S. E., Daselaar, S. M., Greenberg, D. L., Budde, M., Dolcos, F., et al. (2004). Brain activity during episodic retrieval of autobiographical and laboratory events: an fmri study using a novel photo paradigm. J Cogn Neurosci, 16(9), Cabeza, R., & St Jacques, P. (2007). Functional neuroimaging of autobiographical memory. Trends Cogn Sci, 11(5), doi: S (07) [pii] /j.tics Cavanna, A. E., & Trimble, M. R. (2006). The precuneus: a review of its functional anatomy and behavioural correlates. Brain, 129(Pt 3), doi: awl004 [pii] /brain/awl004 Cho, J., & Sharp, P. E. (2001). Head direction, place, and movement correlates for cells in the rat retrosplenial cortex. Behav Neurosci, 115(1), Cipolotti, L., Shallice, T., Chan, D., Fox, N., Scahill, R., Harrison, G., et al. (2001). Longterm retrograde amnesia...the crucial role of the hippocampus. Neuropsychologia, 39(2), doi: S [pii] Clark, B. J., Bassett, J. P., Wang, S. S., & Taube, J. S. (2010). Impaired head direction cell representation in the anterodorsal thalamus after lesions of the retrosplenial cortex. J Neurosci, 30(15), doi: 30/15/5289 [pii] /JNEUROSCI Cohn, M., McAndrews, M. P., & Moscovitch, M. (2009). Associative reinstatement: a novel approach to assessing associative memory in patients with unilateral temporal lobe excisions. Neuropsychologia, 47(13), doi: S (09) [pii] /j.neuropsychologia Cohn, M., Moscovitch, M., Lahat, A., & McAndrews, M. P. (2009). Recollection versus strength as the primary determinant of hippocampal engagement at retrieval. Proc Natl Acad Sci U S A, 106(52), doi: [pii] /pnas Conway, M. A. (1995). Autobiographical knowledge and autobiographical memories. In D. C. Rubin (Ed.), Remembering our past: studies in autobiographical memory (pp ). Cambridge: Cambridge University Press. Conway, M. A. (2009). Episodic memories. Neuropsychologia, 47(11), doi: S (09) [pii] /j.neuropsychologia

168 156 Conway, M. A., & Loveday, C. (2010). Accessing autobiographical memories. In J. H. Mace (Ed.), The act of remembering: toward an understanding of how we recall the past. (pp ). Malden, MA.: Wiley-Blackwell. Conway, M. A., & Pleydell-Pearce, C. W. (2000). The construction of autobiographical memories in the self-memory system. Psychol Rev, 107(2), Conway, M. A., Pleydell-Pearce, C. W., & Whitecross, S. E. (2001). THe neuroanatomy of autobiographical memory: a slow cortical potential study of autobiographical memory retrieval. Journal of Memory and Language, 45(3), Conway, M. A., Pleydell-Pearce, C. W., Whitecross, S. E., & Sharpe, H. (2003). Neurophysiological correlates of memory for experienced and imagined events. Neuropsychologia, 41(3), doi: S [pii] Conway, M. A., Singer, J. A., & Tagini, A. (2004). The self and autobiographical memory: correspondence and coherence. Social Cognition, 22(5), Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci, 3(3), doi: /nrn755 nrn755 [pii] Corkin, S. (2002). What's new with the amnesic patient H.M.? Nat Rev Neurosci, 3(2), doi: /nrn726 nrn726 [pii] Coward, L. A. (2010). The hippocampal system as the cortical resource manager: a model connecting psychology, anatomy and physiology. Adv Exp Med Biol, 657, doi: / _18 Craik, F. I., Moroz, T. M., Moscovitch, M., Stuss, D. T., Winocur, G., Tulving, E., et al. (1999). In search of the self: a position emission tomography study. Psychological Science, 10(1), Daselaar, S. M., Fleck, M. S., Dobbins, I. G., Madden, D. J., & Cabeza, R. (2006). Effects of healthy aging on hippocampal and rhinal memory functions: an event-related fmri study. Cereb Cortex, 16(12), doi: bhj112 [pii] /cercor/bhj112 Daselaar, S. M., Porat, Y., Huijbers, W., & Pennartz, C. M. (2010). Modality-specific and modality-independent components of the human imagery system. Neuroimage, 52(2), doi: S (10) [pii] /j.neuroimage

169 157 Daselaar, S. M., Rice, H. J., Greenberg, D. L., Cabeza, R., LaBar, K. S., & Rubin, D. C. (2008). The spatiotemporal dynamics of autobiographical memory: neural correlates of recall, emotional intensity, and reliving. Cereb Cortex, 18(1), doi: bhm048 [pii] /cercor/bhm048 Derdikman, D., & Moser, E. I. (2010). A manifold of spatial maps in the brain. Trends Cogn Sci, 14(12), doi: S (10) [pii] /j.tics Detre, J. A., Maccotta, L., King, D., Alsop, D. C., Glosser, G., D'Esposito, M., et al. (1998). Functional MRI lateralization of memory in temporal lobe epilepsy. Neurology, 50(4), Dickerson, B. C., & Eichenbaum, H. (2010). The episodic memory system: neurocircuitry and disorders. Neuropsychopharmacology, 35(1), doi: npp [pii] /npp Djordjevic, J., Smith, M. L., Sziklas, V., Piper, D., Penicaud, S., & Jones-Gotman, M. (2010). The Story Learning and Memory (SLAM) test: equivalence of three forms and sensitivity to left temporal lobe dysfunction. Epilepsy Behav, 20(3), doi: S (11) [pii] /j.yebeh Dudukovic, N. M., & Knowlton, B. J. (2006). Remember-Know judgments and retrieval of contextual details. Acta Psychol (Amst), 122(2), doi: S (05) [pii] /j.actpsy Duncan, J. S. (1997). Imaging and epilepsy. Brain, 120 ( Pt 2), Eichenbaum, H., & Lipton, P. A. (2008). Towards a functional organization of the medial temporal lobe memory system: role of the parahippocampal and medial entorhinal cortical areas. Hippocampus, 18(12), doi: /hipo Eichenbaum, H., Otto, T., & Cohen, N. J. (1992). The hippocampus--what does it do? Behav Neural Biol, 57(1), Eichenbaum, H., Yonelinas, A. P., & Ranganath, C. (2007). The medial temporal lobe and recognition memory. Annu Rev Neurosci, 30, doi: /annurev.neuro Engel, J., Jr., Wiebe, S., French, J., Sperling, M., Williamson, P., Spencer, D., et al. (2003). Practice parameter: temporal lobe and localized neocortical resections for epilepsy: report of 157

170 158 the Quality Standards Subcommittee of the American Academy of Neurology, in association with the American Epilepsy Society and the American Association of Neurological Surgeons. Neurology, 60(4), Epstein, R., & Kanwisher, N. (1998). A cortical representation of the local visual environment. Nature, 392(6676), doi: /33402 Epstein, R. A. (2008). Parahippocampal and retrosplenial contributions to human spatial navigation. Trends Cogn Sci, 12(10), doi: S (08)00199-X [pii] /j.tics Epstein, R. A., Parker, W. E., & Feiler, A. M. (2007). Where am I now? Distinct roles for parahippocampal and retrosplenial cortices in place recognition. J Neurosci, 27(23), doi: 27/23/6141 [pii] /JNEUROSCI Ettlinger, G. (1990). "Object vision" and "spatial vision": the neuropsychological evidence for the distinction. Cortex, 26(3), Farah, M. J. (1995). Current issues in the neuropsychology of image generation. Neuropsychologia, 33(11), doi: (95)00075-E [pii] Fletcher, P. C., Frith, C. D., Baker, S. C., Shallice, T., Frackowiak, R. S., & Dolan, R. J. (1995). The mind's eye--precuneus activation in memory-related imagery. Neuroimage, 2(3), doi: S (85) [pii] /nimg Focke, N. K., Yogarajah, M., Bonelli, S. B., Bartlett, P. A., Symms, M. R., & Duncan, J. S. (2008). Voxel-based diffusion tensor imaging in patients with mesial temporal lobe epilepsy and hippocampal sclerosis. Neuroimage, 40(2), doi: S (07) [pii] /j.neuroimage Fortin, N. J., Agster, K. L., & Eichenbaum, H. B. (2002). Critical role of the hippocampus in memory for sequences of events. Nat Neurosci, 5(5), doi: /nn834 nn834 [pii] Fox, M. D., Snyder, A. Z., Vincent, J. L., Corbetta, M., Van Essen, D. C., & Raichle, M. E. (2005). The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc Natl Acad Sci U S A, 102(27), doi: [pii] /pnas

171 159 Friederici, A. D. (2011). The brain basis of language processing: from structure to function. Physiol Rev, 91(4), doi: 91/4/1357 [pii] /physrev Frisk, V., & Milner, B. (1990). The role of the left hippocampal region in the acquisition and retention of story content. Neuropsychologia, 28(4), doi: (90)90061-R [pii] Furman, O., Dorfman, N., Hasson, U., Davachi, L., & Dudai, Y. (2007). They saw a movie: long-term memory for an extended audiovisual narrative. Learn Mem, 14(6), doi: 14/6/457 [pii] /lm Galton, F. (1880). Statistics of mental imagery. Mind, 5(19), Giannoylis, I., Lin, S. Y., & McAndrews, M. P. (2011, May). Autobiographical memory networks in patients with unilateral temporal lobe epilepsy and excisions: An fmri study.. Paper presented at the Toronto Western Research Institute Research Day, Toronto, ON, CA. Gilboa, A. (2004). Autobiographical and episodic memory--one and the same? Evidence from prefrontal activation in neuroimaging studies. Neuropsychologia, 42(10), doi: /j.neuropsychologia S [pii] Gilboa, A., Winocur, G., Grady, C. L., Hevenor, S. J., & Moscovitch, M. (2004). Remembering our past: functional neuroanatomy of recollection of recent and very remote personal events. Cereb Cortex, 14(11), doi: /cercor/bhh082 bhh082 [pii] Gilboa, A., Winocur, G., Rosenbaum, R. S., Poreh, A., Gao, F., Black, S. E., et al. (2006). Hippocampal contributions to recollection in retrograde and anterograde amnesia. Hippocampus, 16(11), doi: /hipo Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action. Trends Neurosci, 15(1), doi: (92) [pii] Graham, K. S., Barense, M. D., & Lee, A. C. (2010). Going beyond LTM in the MTL: a synthesis of neuropsychological and neuroimaging findings on the role of the medial temporal lobe in memory and perception. Neuropsychologia, 48(4), doi: S (10) [pii] /j.neuropsychologia

172 160 Greenberg, D. L., Eacott, M. J., Brechin, D., & Rubin, D. C. (2005). Visual memory loss and autobiographical amnesia: a case study. Neuropsychologia, 43(10), doi: S (05) [pii] /j.neuropsychologia Greenberg, D. L., & Rubin, D. C. (2003). The neuropsychology of autobiographical memory. Cortex, 39(4-5), Greenberg, D. L., & Verfaellie, M. (2010). Interdependence of episodic and semantic memory: evidence from neuropsychology. J Int Neuropsychol Soc, 16(5), doi: S [pii] /S Grill-Spector, K., Kourtzi, Z., & Kanwisher, N. (2001). The lateral occipital complex and its role in object recognition. Vision Res, 41(10-11), doi: S (01) [pii] Gross, D. W. (2011). Diffusion tensor imaging in temporal lobe epilepsy. Epilepsia, 52 Suppl 4, doi: /j x Gross, D. W., Concha, L., & Beaulieu, C. (2006). Extratemporal white matter abnormalities in mesial temporal lobe epilepsy demonstrated with diffusion tensor imaging. Epilepsia, 47(8), doi: EPI603 [pii] /j x Gusnard, D. A., Akbudak, E., Shulman, G. L., & Raichle, M. E. (2001). Medial prefrontal cortex and self-referential mental activity: relation to a default mode of brain function. Proc Natl Acad Sci U S A, 98(7), doi: /pnas [pii] Hannula, D. E., & Ranganath, C. (2008). Medial temporal lobe activity predicts successful relational memory binding. J Neurosci, 28(1), doi: 28/1/116 [pii] /JNEUROSCI Hannula, D. E., & Ranganath, C. (2009). The eyes have it: hippocampal activity predicts expression of memory in eye movements. Neuron, 63(5), doi: S (09) [pii] /j.neuron Hannula, D. E., Tranel, D., & Cohen, N. J. (2006). The long and the short of it: relational memory impairments in amnesia, even at short lags. J Neurosci, 26(32), doi: 26/32/8352 [pii] /JNEUROSCI Harvey, D. J., Naugle, R. I., Magleby, J., Chapin, J. S., Najm, I. M., Bingaman, W., et al. (2008). Relationship between presurgical memory performance on the Wechsler Memory 160

173 161 Scale-III and memory change following temporal resection for treatment of intractable epilepsy. Epilepsy Behav, 13(2), doi: S (08) [pii] /j.yebeh Hassabis, D., Kumaran, D., & Maguire, E. A. (2007). Using imagination to understand the neural basis of episodic memory. J Neurosci, 27(52), doi: 27/52/14365 [pii] /JNEUROSCI Hassabis, D., Kumaran, D., Vann, S. D., & Maguire, E. A. (2007). Patients with hippocampal amnesia cannot imagine new experiences. Proc Natl Acad Sci U S A, 104(5), doi: [pii] /pnas Hassabis, D., & Maguire, E. A. (2007). Deconstructing episodic memory with construction. Trends Cogn Sci, 11(7), doi: S (07) [pii] /j.tics Hassabis, D., & Maguire, E. A. (2009). The construction system of the brain. Philos Trans R Soc Lond B Biol Sci, 364(1521), doi: 364/1521/1263 [pii] /rstb Hasson, U., Furman, O., Clark, D., Dudai, Y., & Davachi, L. (2008). Enhanced intersubject correlations during movie viewing correlate with successful episodic encoding. Neuron, 57(3), doi: S (07) [pii] /j.neuron Helmstaedter, C., Grunwald, T., Lehnertz, K., Gleissner, U., & Elger, C. E. (1997). Differential involvement of left temporolateral and temporomesial structures in verbal declarative learning and memory: evidence from temporal lobe epilepsy. Brain Cogn, 35(1), doi: S (97) [pii] /brcg Henke, K. (2010). A model for memory systems based on processing modes rather than consciousness. Nat Rev Neurosci, 11(7), doi: nrn2850 [pii] /nrn2850 Herfurth, K., Kasper, B., Schwarz, M., Stefan, H., & Pauli, E. (2010). Autobiographical memory in temporal lobe epilepsy: Role of hippocampal and temporal lateral structures. Epilepsy Behav. doi: S (10) [pii] /j.yebeh Hoscheidt, S. M., Nadel, L., Payne, J., & Ryan, L. (2010). Hippocampal activation during retrieval of spatial context from episodic and semantic memory. Behav Brain Res, 212(2), doi: S (10) [pii] /j.bbr

174 162 Huijbers, W., Pennartz, C. M., Rubin, D. C., & Daselaar, S. M. (2011). Imagery and retrieval of auditory and visual information: Neural correlates of successful and unsuccessful performance. Neuropsychologia. doi: S (11) [pii] /j.neuropsychologia Hurley, N. C., Maguire, E. A., & Vargha-Khadem, F. (2011). Patient HC with developmental amnesia can construct future scenarios. Neuropsychologia, 49(13), doi: S (11) [pii] /j.neuropsychologia Insausti, R., Amaral, D. G., & Cowan, W. M. (1987). The entorhinal cortex of the monkey: II. Cortical afferents. J Comp Neurol, 264(3), doi: /cne Jackson, G. D., Connelly, A., Duncan, J. S., Grunewald, R. A., & Gadian, D. G. (1993). Detection of hippocampal pathology in intractable partial epilepsy: increased sensitivity with quantitative magnetic resonance T2 relaxometry. Neurology, 43(9), Johnson, J. D., & Rugg, M. D. (2007). Recollection and the reinstatement of encodingrelated cortical activity. Cereb Cortex, 17(11), doi: bhl156 [pii] /cercor/ bhl156 Johnson, M. K. (2006). Memory and reality. Am Psychol, 61(8), doi: [pii] / X Johnson, M. K., & Sherman, S. J. (1990). Constructing and reconstructing the past and the future in the present. In E. T. Higgins & R. M. Sorrentino (Eds.), Handbook of Motivation and Cognition: Foundations of Social Behavior. New York: The Guilford Press. Jones-Gotman, M., & Milner, B. (1978). Right temporal-lobe contribution to imagemediated verbal learning. Neuropsychologia, 16(1), Jones-Gotman, M., Smith, M. L., Risse, G. L., Westerveld, M., Swanson, S. J., Giovagnoli, A. R., et al. (2010). The contribution of neuropsychology to diagnostic assessment in epilepsy. Epilepsy Behav, 18(1-2), doi: S (10) [pii] /j.yebeh Kalviainen, R., Partanen, K., Aikia, M., Mervaala, E., Vainio, P., Riekkinen, P. J., Sr., et al. (1997). MRI-based hippocampal volumetry and T2 relaxometry: correlation to verbal memory performance in newly diagnosed epilepsy patients with left-sided temporal lobe focus. Neurology, 48(1),

175 163 Kalviainen, R., Salmenpera, T., Partanen, K., Vainio, P., Riekkinen, P., & Pitkanen, A. (1998). Recurrent seizures may cause hippocampal damage in temporal lobe epilepsy. Neurology, 50(5), Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: a module in human extrastriate cortex specialized for face perception. J Neurosci, 17(11), Kapur, N. (1999). Syndromes of retrograde amnesia: a conceptual and empirical synthesis. Psychol Bull, 125(6), Keller, S. S., & Roberts, N. (2008). Voxel-based morphometry of temporal lobe epilepsy: an introduction and review of the literature. Epilepsia, 49(5), doi: EPI1485 [pii] /j x Kelley, W. M., Macrae, C. N., Wyland, C. L., Caglar, S., Inati, S., & Heatherton, T. F. (2002). Finding the self? An event-related fmri study. J Cogn Neurosci, 14(5), doi: / Kilpatrick, C., Murrie, V., Cook, M., Andrewes, D., Desmond, P., & Hopper, J. (1997). Degree of left hippocampal atrophy correlates with severity of neuropsychological deficits. Seizure, 6(3), Klein, S. B., & Loftus, J. (2002). Memory and temporal experience: the effects of episodic memory loss on an amnesic patient's ability to remember the past and imagine the future. Social Cognition, 20(5), Konkel, A., Warren, D. E., Duff, M. C., Tranel, D. N., & Cohen, N. J. (2008). Hippocampal amnesia impairs all manner of relational memory. Front Hum Neurosci, 2, 15. doi: /neuro Kosslyn, S. M. (2005). Mental images and the Brain. Cogn Neuropsychol, 22(3), doi: [pii] / Kosslyn, S. M., Koenig, O., Barrett, A., Cave, C. B., Tang, J., & Gabrieli, J. D. (1989). Evidence for two types of spatial representations: hemispheric specialization for categorical and coordinate relations. J Exp Psychol Hum Percept Perform, 15(4),

176 164 Kosslyn, S. M., Maljkovic, V., Hamilton, S. E., Horwitz, G., & Thompson, W. L. (1995). Two types of image generation: evidence for left and right hemisphere processes. Neuropsychologia, 33(11), doi: (95)00077-G [pii] Kosslyn, S. M., Thompson, W. L., Sukel, K. E., & Alpert, N. M. (2005). Two types of image generation: evidence from PET. Cogn Affect Behav Neurosci, 5(1), Kwan, D., Carson, N., Addis, D. R., & Rosenbaum, R. S. (2010). Deficits in past remembering extend to future imagining in a case of developmental amnesia. Neuropsychologia, 48(11), doi: S (10)00240-X [pii] /j.neuropsychologia Lah, S., Grayson, S., Lee, T., & Miller, L. (2004). Memory for the past after temporal lobectomy: impact of epilepsy and cognitive variables. Neuropsychologia, 42(12), doi: /j.neuropsychologia S [pii] Lah, S., Lee, T., Grayson, S., & Miller, L. (2006). Effects of temporal lobe epilepsy on retrograde memory. Epilepsia, 47(3), doi: EPI476 [pii] /j x Lambon Ralph, M. A., Cipolotti, L., Manes, F., & Patterson, K. (2010). Taking both sides: do unilateral anterior temporal lobe lesions disrupt semantic memory? Brain, 133(11), doi: awq264 [pii] /brain/awq264 Larsen, S. F. (1998). What is it like to remember? On phenomenal qualities of memory. In C. P. Thompson, Herrmann, D.J., Bruce, D., Read, J.D., Payne, D.G., & Toglia, M.P. (Ed.), Autobiographical Memory. Theoretical and Applied Perspectives. (pp. pp ). Mahwah, NJ.: Lawrence Erlbau Associates, Publishers. Lehmann, D., Pascual-Marqui, R. D., Strik, W. K., & Koenig, T. (2010). Core networks for visual-concrete and abstract thought content: a brain electric microstate analysis. Neuroimage, 49(1), doi: S (09) [pii] /j.neuroimage Levine, B., Svoboda, E., Hay, J. F., Winocur, G., & Moscovitch, M. (2002). Aging and autobiographical memory: dissociating episodic from semantic retrieval. Psychol Aging, 17(4),

177 165 Li, J., Zhang, Z., & Shang, H. (2012). A meta-analysis of voxel-based morphometry studies on unilateral refractory temporal lobe epilepsy. Epilepsy Res. doi: S (11)00307-X [pii] /j.eplepsyres Litman, L., Awipi, T., & Davachi, L. (2009). Category-specificity in the human medial temporal lobe cortex. Hippocampus, 19(3), doi: /hipo Liu, T. T., Frank, L. R., Wong, E. C., & Buxton, R. B. (2001). Detection power, estimation efficiency, and predictability in event-related fmri. Neuroimage, 13(4), doi: /nimg S (00) [pii] Loveday, C., & Conway, M. A. (2011). Using SenseCam with an amnesic patient: accessing inaccessible everyday memories. Memory, 19(7), doi: / Luby, M., Spencer, D. D., Kim, J. H., delanerolle, N., & McCarthy, G. (1995). Hippocampal MRI volumetrics and temporal lobe substrates in medial temporal lobe epilepsy. Magn Reson Imaging, 13(8), doi: X(95)02014-K [pii] Maguire, E. A. (2001a). Neuroimaging studies of autobiographical event memory. Philos Trans R Soc Lond B Biol Sci, 356(1413), doi: /rstb Maguire, E. A. (2001b). Neuroimaging, memory and the human hippocampus. Rev Neurol (Paris), 157(8-9 Pt 1), doi: MDOI-RN C ART9 [pii] Maguire, E. A., & Frith, C. D. (2003). Lateral asymmetry in the hippocampal response to the remoteness of autobiographical memories. J Neurosci, 23(12), doi: 23/12/5302 [pii] Maguire, E. A., Frith, C. D., Rudge, P., & Cipolotti, L. (2005). The effect of adult-acquired hippocampal damage on memory retrieval: an fmri study. Neuroimage, 27(1), doi: S (05) [pii] /j.neuroimage Maguire, E. A., Vargha-Khadem, F., & Hassabis, D. (2010). Imagining fictitious and future experiences: evidence from developmental amnesia. Neuropsychologia, 48(11),

178 166 Maguire, E. A., Vargha-Khadem, F., & Mishkin, M. (2001). The effects of bilateral hippocampal damage on fmri regional activations and interactions during memory retrieval. Brain, 124(Pt 6), Malach, R., Reppas, J. B., Benson, R. R., Kwong, K. K., Jiang, H., Kennedy, W. A., et al. (1995). Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Natl Acad Sci U S A, 92(18), Manns, J. R., Howard, M. W., & Eichenbaum, H. (2007). Gradual changes in hippocampal activity support remembering the order of events. Neuron, 56(3), doi: S (07) [pii] /j.neuron Martin, A. (2007). The representation of object concepts in the brain. Annu Rev Psychol, 58, doi: /annurev.psych Martin, A., & Chao, L. L. (2001). Semantic memory and the brain: structure and processes. Curr Opin Neurobiol, 11(2), doi: S (00) [pii] Mathern, G. W., Babb, T. L., Leite, J. P., Pretorius, K., Yeoman, K. M., & Kuhlman, P. A. (1996). The pathogenic and progressive features of chronic human hippocampal epilepsy. Epilepsy Res, 26(1), doi: S [pii] McAndrews, M. P. (In Press). Remote Memory in Temporal Lobe Epilepsy. In A. Zeman, N. Kapur & M. Jones-Gotman (Eds.), Epilepsy and Memory. Oxford, UK.: Oxford University Press. McAndrews, M. P., & Cohn, M. (In Press). Neuropsychology in temporal lobe epilepsy: influences from cognitive neuroscience and functional neuroimaging. Epilepsy Research and Treatment. McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol Rev, 102(3), McCormick, C., Moscovitch, M., Protzner, A. B., Huber, C. G., & McAndrews, M. P. (2010). Hippocampal-neocortical networks differ during encoding and retrieval of relational 166

179 167 memory: functional and effective connectivity analyses. Neuropsychologia, 48(11), doi: S (10) [pii] /j.neuropsychologia McDermott, K. B., Szpunar, K. K., & Christ, S. E. (2009). Laboratory-based and autobiographical retrieval tasks differ substantially in their neural substrates. Neuropsychologia, 47(11), doi: S (08) [pii] / j.neuropsychologia McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), Mechanic-Hamilton, D., Korczykowski, M., Yushkevich, P. A., Lawler, K., Pluta, J., Glynn, S., et al. (2009). Hippocampal volumetry and functional MRI of memory in temporal lobe epilepsy. Epilepsy Behav, 16(1), doi: S (09) [pii] /j.yebeh Milner, B. (1970). Memory and the medial temporal regions of the brain. In K. H. Pribam & D. E. Broadbent (Eds.), Biology of memory (pp ). New York: Academic Press. Mitchell, K. J., & Johnson, M. K. (2009). Source monitoring 15 years later: what have we learned from fmri about the neural mechanisms of source memory? Psychol Bull, 135(4), doi: [pii] /a Moran, N. F., Lemieux, L., Kitchen, N. D., Fish, D. R., & Shorvon, S. D. (2001). Extrahippocampal temporal lobe atrophy in temporal lobe epilepsy and mesial temporal sclerosis. Brain, 124(Pt 1), Morris, R. G., Abrahams, S., Baddeley, A., & Polkey, C. E. (1995). Doors and people: visual and vernal memory after unilateral temporal lobectomy. Neuropsychology, 9(4), Morris, R. G., Pickering, A., Abrahams, S., & Feigenbaum, J. D. (1996). Space and the hippocampal formation in humans. Brain Res Bull, 40(5-6), doi: (96) [pii] Moscovitch, D. A., & McAndrews, M. P. (2002). Material-specific deficits in "remembering" in patients with unilateral temporal lobe epilepsy and excisions. Neuropsychologia, 40(8), doi: S [pii] 167

180 168 Moscovitch, M. (1995). Recovered consciousness: a hypothesis concerning modularity and episodic memory. J Clin Exp Neuropsychol, 17(2), doi: / Moscovitch, M. (2008). The hippocampus as a "stupid," domain-specific module: Implications for theories of recent and remote memory, and of imagination. Can J Exp Psychol, 62(1), doi: [pii] / Moscovitch, M. (Submitted). The contribution of research on autobiographical memory to past and present theories of memory consolidation. Moscovitch, M., & Nadel, L. (1998). Consolidation and the hippocampal complex revisited: in defense of the multiple-trace model. Curr Opin Neurobiol, 8(2), doi: S (98) [pii] Moscovitch, M., Rosenbaum, R. S., Gilboa, A., Addis, D. R., Westmacott, R., Grady, C., et al. (2005). Functional neuroanatomy of remote episodic, semantic and spatial memory: a unified account based on multiple trace theory. J Anat, 207(1), doi: JOA421 [pii] /j x Mueller, S. G., Laxer, K. D., Scanlon, C., Garcia, P., McMullen, W. J., Loring, D. W., et al. (2011). Different structural correlates for verbal memory impairment in temporal lobe epilepsy with and without mesial temporal lobe sclerosis. Hum Brain Mapp. doi: /hbm Nadel, L., & Hardt, O. (2011). Update on memory systems and processes. Neuropsychopharmacology, 36(1), doi: npp [pii] /npp Nadel, L., & Moscovitch, M. (1997). Memory consolidation, retrograde amnesia and the hippocampal complex. Curr Opin Neurobiol, 7(2), doi: S (97) [pii] Neisser, U., & Harsch, N. (1992). Phantom flashbulbs: false recollections of hearing the news about Challenger. In E. Winograd & U. Neisser (Eds.), Affect and accuracy in recall: studies of "flashbulb" memories. (pp ). New York: Cambridge University Press. Neisser, U., Winograd, E., Bergman, E. T., Schreiber, C. A., Palmer, S. E., & Weldon, M. S. (1996). Remembering the earthquake: direct experience vs. hearing the news. Memory, 4(4),

181 169 Nestor, A., Plaut, D. C., & Behrmann, M. (2011). Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis. Proc Natl Acad Sci U S A, 108(24), doi: [pii] /pnas Nichols, T., Brett, M., Andersson, J., Wager, T., & Poline, J. B. (2005). Valid conjunction inference with the minimum statistic. Neuroimage, 25(3), doi: S (04) [pii] /j.neuroimage Noulhiane, M., Piolino, P., Hasboun, D., Clemenceau, S., Baulac, M., & Samson, S. (2007). Autobiographical memory after temporal lobe resection: neuropsychological and MRI volumetric findings. Brain, 130(Pt 12), doi: awm258 [pii] /brain/awm258 Noulhiane, M., Piolino, P., Hasboun, D., Clemenceau, S., Baulac, M., & Samson, S. (2008). Autonoetic consciousness in autobiographical memories after medial temporal lobe resection. Behav Neurol, 19(1-2), Nyberg, L., Forkstam, C., Petersson, K. M., Cabeza, R., & Ingvar, M. (2002). Brain imaging of human memory systems: between-systems similarities and within-system differences. Brain Res Cogn Brain Res, 13(2), doi: S [pii] Nyberg, L., Marklund, P., Persson, J., Cabeza, R., Forkstam, C., Petersson, K. M., et al. (2003). Common prefrontal activations during working memory, episodic memory, and semantic memory. Neuropsychologia, 41(3), doi: S [pii] O'Keefe, J., & Dostrovsky, J. (1971). The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Res, 34(1), doi: (71) [pii] O'Keefe, J., & Nadel, L. (1978). The hippocampus as a cognitive map. Oxford, UK: Clarendon Press. Olson, I. R., Plotzker, A., & Ezzyat, Y. (2007). The Enigmatic temporal pole: a review of findings on social and emotional processing. Brain, 130(Pt 7), doi: awm052 [pii] /brain/awm052 Pardoe, H. R., Pell, G. S., Abbott, D. F., & Jackson, G. D. (2009). Hippocampal volume assessment in temporal lobe epilepsy: How good is automated segmentation? Epilepsia, 50(12), doi: EPI2243 [pii] /j x 169

182 170 Park, L., St-Laurent, M., McAndrews, M. P., & Moscovitch, M. (2011). The immediacy of recollection: the use of the historical present in narratives of autobiographical episodes by patients with unilateral temporal lobe epilepsy. Neuropsychologia, 49(5), doi: S (11) [pii] /j.neuropsychologia Patterson, K., Nestor, P. J., & Rogers, T. T. (2007). Where do you know what you know? The representation of semantic knowledge in the human brain. Nat Rev Neurosci, 8(12), doi: nrn2277 [pii] /nrn2277 Pell, G. S., Briellmann, R. S., Pardoe, H., Abbott, D. F., & Jackson, G. D. (2008). Composite voxel-based analysis of volume and T2 relaxometry in temporal lobe epilepsy. Neuroimage, 39(3), doi: S (07) [pii] /j.neuroimage Pennebaker, J. W., Chung, C. K., Ireland, M., Gonzales, A., & Booth, R. J. (2007). The development and psychometric properties of LIWC2007. [software manual]. Austin, TX. Piolino, P., Desgranges, B., Belliard, S., Matuszewski, V., Lalevee, C., De la Sayette, V., et al. (2003). Autobiographical memory and autonoetic consciousness: triple dissociation in neurodegenerative diseases. Brain, 126(Pt 10), doi: /brain/awg222 awg222 [pii] Piolino, P., Desgranges, B., & Eustache, F. (2009). Episodic autobiographical memories over the course of time: cognitive, neuropsychological and neuroimaging findings. Neuropsychologia, 47(11), doi: S (09) [pii] / j.neuropsychologia Protzner, A. B., & McAndrews, M. P. (2011). Network alterations supporting word retrieval in patients with medial temporal lobe epilepsy. J Cogn Neurosci, 23(9), doi: /jocn Pylyshyn, Z. (2003). Return of the mental image: are there really pictures in the brain? Trends Cogn Sci, 7(3), doi: S [pii] Rabin, J. S., Gilboa, A., Stuss, D. T., Mar, R. A., & Rosenbaum, R. S. (2010). Common and unique neural correlates of autobiographical memory and theory of mind. J Cogn Neurosci, 22(6), doi: /jocn

183 171 Rabin, M. L., Narayan, V. M., Kimberg, D. Y., Casasanto, D. J., Glosser, G., Tracy, J. I., et al. (2004). Functional MRI predicts post-surgical memory following temporal lobectomy. Brain, 127(Pt 10), doi: /brain/awh281 awh281 [pii] Race, E., Keane, M. M., & Verfaellie, M. (2011). Medial temporal lobe damage causes deficits in episodic memory and episodic future thinking not attributable to deficits in narrative construction. J Neurosci, 31(28), doi: 31/28/10262 [pii] / JNEUROSCI Radvansky, G. A., Copeland, D. E., & Zwaan, R. A. (2005). A novel study: investigating the structure of narrative and autobiographical memories. Memory, 13(8), doi: Q4L662202H [pii] / Raichle, M. E., MacLeod, A. M., Snyder, A. Z., Powers, W. J., Gusnard, D. A., & Shulman, G. L. (2001). A default mode of brain function. Proc Natl Acad Sci U S A, 98(2), doi: /pnas /2/676 [pii] Rajah, M. N., & McIntosh, A. R. (2005). Overlap in the functional neural systems involved in semantic and episodic memory retrieval. J Cogn Neurosci, 17(3), doi: / Ranganath, C. (2010). A unified framework for the functional organization of the medial temporal lobes and the phenomenology of episodic memory. Hippocampus, 20(11), doi: /hipo Rausch, R., & Babb, T. L. (1993). Hippocampal neuron loss and memory scores before and after temporal lobe surgery for epilepsy. Arch Neurol, 50(8), Ribot, T. (Ed.). (1887). Diseases of the Memory: An Essay in the Positive Psychology. New York, NY.: D. Appleton and Compagny. Richardson, M. P., Strange, B. A., Thompson, P. J., Baxendale, S. A., Duncan, J. S., & Dolan, R. J. (2004). Pre-operative verbal memory fmri predicts post-operative memory decline after left temporal lobe resection. Brain, 127(Pt 11), doi: /brain/awh293 awh293 [pii] Rosenbaum, R. S., Gilboa, A., Levine, B., Winocur, G., & Moscovitch, M. (2009). Amnesia as an impairment of detail generation and binding: evidence from personal, fictional, and 171

184 172 semantic narratives in K.C. Neuropsychologia, 47(11), doi: S (08) [pii] /j.neuropsychologia Rosenbaum, R. S., Kohler, S., Schacter, D. L., Moscovitch, M., Westmacott, R., Black, S. E., et al. (2005). The case of K.C.: contributions of a memory-impaired person to memory theory. Neuropsychologia, 43(7), doi: S (04) [pii] /j.neuropsychologia Rosenbaum, R. S., McKinnon, M. C., Levine, B., & Moscovitch, M. (2004). Visual imagery deficits, impaired strategic retrieval, or memory loss: disentangling the nature of an amnesic person's autobiographical memory deficit. Neuropsychologia, 42(12), doi: /j.neuropsychologia S [pii] Rosenbaum, R. S., Moscovitch, M., Foster, J. K., Schnyer, D. M., Gao, F., Kovacevic, N., et al. (2008). Patterns of autobiographical memory loss in medial-temporal lobe amnesic patients. J Cogn Neurosci, 20(8), doi: /jocn Rosenbaum, R. S., Stuss, D. T., Levine, B., & Tulving, E. (2007). Theory of mind is independent of episodic memory. Science, 318(5854), doi: 318/5854/1257 [pii] /science Roser, M. E., Fugelsang, J. A., Dunbar, K. N., Corballis, P. M., & Gazzaniga, M. S. (2005). Dissociating processes supporting causal perception and causal inference in the brain. Neuropsychology, 19(5), doi: [pii] / Rubin, D. C. (2005). A basic-systems approach to autobiographical memory. Current Directions in Psychological Science. Rubin, D. C., & Greenberg, D. L. (1998). Visual memory-deficit amnesia: a distinct amnesic presentation and etiology. Proc Natl Acad Sci U S A, 95(9), Rubin, D. C., Schrauf, R. W., & Greenberg, D. L. (2003). Belief and recollection of autobiographical memories. Mem Cognit, 31(6), Ryan, L., Lin, C. Y., Ketcham, K., & Nadel, L. (2010). The role of medial temporal lobe in retrieving spatial and nonspatial relations from episodic and semantic memory. Hippocampus, 20(1), doi: /hipo

185 173 Sass, K. J., Buchanan, C. P., Kraemer, S., Westerveld, M., Kim, J. H., & Spencer, D. D. (1995). Verbal memory impairment resulting from hippocampal neuron loss among epileptic patients with structural lesions. Neurology, 45(12), Schacter, D. L., & Addis, D. R. (2009). On the nature of medial temporal lobe contributions to the constructive simulation of future events. Philos Trans R Soc Lond B Biol Sci, 364(1521), doi: 364/1521/1245 [pii] /rstb Schramm, J. (2008). Temporal lobe epilepsy surgery and the quest for optimal extent of resection: a review. Epilepsia, 49(8), doi: EPI1604 [pii] /j x Schulz, R., Woermann, F. G., & Ebner, A. (2007). When written words become moving pictures: complex visual hallucinations on stimulation of the lateral occipital lobe. Epilepsy Behav, 11(1), doi: S (07) [pii] /j.yebeh Scoville, W. B., & Milner, B. (1957). Loss of recent memory after bilateral hippocampal lesions. J Neurol Neurosurg Psychiatry, 20(1), Seidenberg, M., Geary, E., & Hermann, B. (2005). Investigating temporal lobe contribution to confrontation naming using MRI quantitative volumetrics. J Int Neuropsychol Soc, 11(4), Sheldon, S., McAndrews, M. P., & Moscovitch, M. (2011). Episodic memory processes mediated by the medial temporal lobes contribute to open-ended problem solving. Neuropsychologia. doi: S (11) [pii] /j.neuropsychologia Slotnick, S. D. (2010). Does the hippocampus mediate objective binding or subjective remembering? Neuroimage, 49(2), doi: S (09) [pii] /j.neuroimage Smith, M. L., & Milner, B. (1981). The role of the right hippocampus in the recall of spatial location. Neuropsychologia, 19(6), Smith, M. L., & Milner, B. (1989). Right hippocampal impairment in the recall of spatial location: encoding deficit or rapid forgetting? Neuropsychologia, 27(1), doi: (89) [pii] 173

186 174 Spaniol, J., Davidson, P. S., Kim, A. S., Han, H., Moscovitch, M., & Grady, C. L. (2009). Event-related fmri studies of episodic encoding and retrieval: meta-analyses using activation likelihood estimation. Neuropsychologia, 47(8-9), doi: S (09) [pii] /j.neuropsychologia Spiers, H. J., Burgess, N., Maguire, E. A., Baxendale, S. A., Hartley, T., Thompson, P. J., et al. (2001). Unilateral temporal lobectomy patients show lateralized topographical and episodic memory deficits in a virtual town. Brain, 124(Pt 12), Spiers, H. J., Maguire, E. A., & Burgess, N. (2001). Hippocampal amnesia. Neurocase, 7(5), Spreen, O., & Strauss, E. (1991). A compendium of neuropsychological tests: administration, norms and commentary. New York.: Oxford University Press. Spreng, R. N., & Grady, C. L. (2010). Patterns of brain activity supporting autobiographical memory, prospection, and theory of mind, and their relationship to the default mode network. J Cogn Neurosci, 22(6), doi: /jocn Spreng, R. N., Mar, R. A., & Kim, A. S. (2009). The common neural basis of autobiographical memory, prospection, navigation, theory of mind, and the default mode: a quantitative meta-analysis. J Cogn Neurosci, 21(3), doi: /jocn Squire, L. R. (1992). Memory and the hippocampus: a synthesis from findings with rats, monkeys, and humans. Psychol Rev, 99(2), Squire, L. R., Ojemann, J. G., Miezin, F. M., Petersen, S. E., Videen, T. O., & Raichle, M. E. (1992). Activation of the hippocampus in normal humans: a functional anatomical study of memory. Proc Natl Acad Sci U S A, 89(5), Squire, L. R., van der Horst, A. S., McDuff, S. G., Frascino, J. C., Hopkins, R. O., & Mauldin, K. N. (2010). Role of the hippocampus in remembering the past and imagining the future. Proc Natl Acad Sci U S A, 107(44), doi: [pii] /pnas Squire, L. R., & Wixted, J. T. (2011). The cognitive neuroscience of human memory since H.M. Annu Rev Neurosci, 34, doi: /annurev-neuro

187 175 St-Laurent, M., Abdi, H., Burianova, H., & Grady, C. L. (2011). Influence of aging on the neural correlates of autobiographical, episodic, and semantic memory retrieval. J Cogn Neurosci, 23(12), doi: /jocn_a_00079 St-Laurent, M., Moscovitch, M., Levine, B., & McAndrews, M. P. (2009). Determinants of autobiographical memory in patients with unilateral temporal lobe epilepsy or excisions. Neuropsychologia, 47(11), doi: S (09) [pii] / j.neuropsychologia St-Laurent, M., Moscovitch, M., Tau, M., & McAndrews, M. P. (2011). The temporal unraveling of autobiographical memory narratives in patients with temporal lobe epilepsy or excisions. Hippocampus, 21(4), doi: /hipo St Jacques, P., Conway, M., & Cabeza, R. (2011, November). Age-Related Effects on the Neural Correlates of Autobiographical Memory Retrieval Under Varying Levels of Retrieval Support. Paper presented at the Annual Scientific Meeting of the Gerontological Society of America, Boston, MA. St Jacques, P., Conway, M., Lowder, M. W., & Cabeza, R. (2011, April). Two ways of accessing the personal past: an fmri study examining the functional connectivity of the hippocampus during autobiographical memory retrieval. Paper presented at the Annual Meeting of the Cognitive Neuroscience Society, San Francisco, CA. St Jacques, P. L., Conway, M. A., & Cabeza, R. (2011). Gender differences in autobiographical memory for everyday events: retrieval elicited by SenseCam images versus verbal cues. Memory, 19(7), doi: [pii] / St Jacques, P. L., Conway, M. A., Lowder, M. W., & Cabeza, R. (2011). Watching My Mind Unfold versus Yours: An fmri Study Using a Novel Camera Technology to Examine Neural Differences in Self-projection of Self versus Other Perspectives. J Cogn Neurosci. doi: /jocn St Jacques, P. L., Kragel, P. A., & Rubin, D. C. (2011). Dynamic neural networks supporting memory retrieval. Neuroimage. doi: S (11) [pii] /j.neuroimage

188 176 St Jacques, P. L., Rubin, D. C., & Cabeza, R. (In Press). Age-related effects on the neural correlates of autobiographical memory retrieval. Neurobiol Aging. doi: S (10) [pii] /j.neurobiolaging Stangalino, C., Semenza, C., & Mondini, S. (1995). Generating visual mental images: deficit after brain damage. Neuropsychologia, 33(11), doi: (95)00076-F [pii] Staresina, B. P., & Davachi, L. (2008). Selective and shared contributions of the hippocampus and perirhinal cortex to episodic item and associative encoding. J Cogn Neurosci, 20(8), doi: /jocn Staresina, B. P., & Davachi, L. (2009). Mind the gap: binding experiences across space and time in the human hippocampus. Neuron, 63(2), doi: S (09) [pii] /j.neuron Stark, C. E., & Squire, L. R. (2001). When zero is not zero: the problem of ambiguous baseline conditions in fmri. Proc Natl Acad Sci U S A, 98(22), doi: /pnas [pii] Steinvorth, S., Corkin, S., & Halgren, E. (2006). Ecphory of autobiographical memories: an fmri study of recent and remote memory retrieval. Neuroimage, 30(1), doi: S (05) [pii] /j.neuroimage Steinvorth, S., Levine, B., & Corkin, S. (2005). Medial temporal lobe structures are needed to re-experience remote autobiographical memories: evidence from H.M. and W.R. Neuropsychologia, 43(4), doi: S (05) [pii] / j.neuropsychologia Stevens, W. D., Kahn, I., Wig, G. S., & Schacter, D. L. (In Press). Hemispheric Asymmetry of Visual Scene Processing in the Human Brain: Evidence from Repetition Priming and Intrinsic Activity. Cereb Cortex. doi: bhr273 [pii] /cercor/bhr273 Stewart, C. C., Griffith, H. R., Okonkwo, O. C., Martin, R. C., Knowlton, R. K., Richardson, E. J., et al. (2009). Contributions of volumetrics of the hippocampus and thalamus to verbal memory in temporal lobe epilepsy patients. Brain Cogn, 69(1), doi: S (08)00198-X [pii] /j.bandc

189 177 Stokes, J., Mazuz, Y., Daselaar, S. M., Moscovitch, M., & Cabeza, R. (2011, April). Similarities and differences between the neural mechanisms of episodic and autobiographical memory recall. Paper presented at the Annual Meeting of the Cognitive Neuroscience Society, San Francisco, CA. Strauss, E., Sherman, E. M. S., & Spreen, O. (2006). A compendium of neuropsychological tests: administration, norms, and commentary. 3rd Edition. (3rd Edition ed.). New York.: Oxford University Press. Suddendorf, T., Addis, D. R., & Corballis, M. C. (2009). Mental time travel and the shaping of the human mind. Philos Trans R Soc Lond B Biol Sci, 364(1521), doi: 364/1521/1317 [pii] /rstb Suddendorf, T., & Corballis, M. C. (1997). Mental time travel and the evolution of the human mind. Genet Soc Gen Psychol Monogr, 123(2), Summerfield, J. J., Hassabis, D., & Maguire, E. A. (2009). Cortical midline involvement in autobiographical memory. Neuroimage, 44(3), doi: S (08) [pii] /j.neuroimage Summerfield, J. J., Hassabis, D., & Maguire, E. A. (2010). Differential engagement of brain regions within a 'core' network during scene construction. Neuropsychologia, 48(5), doi: S (10) [pii] /j.neuropsychologia Suzuki, W. A. (2010). Untangling memory from perception in the medial temporal lobe. Trends Cogn Sci, 14(5), doi: S (10) [pii] /j.tics Svoboda, E., McKinnon, M. C., & Levine, B. (2006). The functional neuroanatomy of autobiographical memory: a meta-analysis. Neuropsychologia, 44(12), doi: S (06) [pii] /j.neuropsychologia Szpunar, K. K., Watson, J. M., & McDermott, K. B. (2007). Neural substrates of envisioning the future. Proc Natl Acad Sci U S A, 104(2), doi: [pii] /pnas Teyler, T. J., & DiScenna, P. (1986). The hippocampal memory indexing theory. Behav Neurosci, 100(2),

190 178 Teyler, T. J., & Rudy, J. W. (2007). The hippocampal indexing theory and episodic memory: updating the index. Hippocampus, 17(12), doi: /hipo Toro, R., Fox, P. T., & Paus, T. (2008). Functional coactivation map of the human brain. Cereb Cortex, 18(11), doi: bhn014 [pii] /cercor/bhn014 Trotta, N., Goldman, S., Legros, B., Ligot, N., Guerry, N., Baete, K., et al. (2011). Metabolic evidence for episodic memory plasticity in the nonepileptic temporal lobe of patients with mesial temporal epilepsy. Epilepsia, 52(11), doi: /j x Tulving, E. (1972). Episodic and semantic memory. In E. Tulving & W. Donaldson (Eds.), Organization of memory (pp ). New York: Academic Press. Tulving, E. (1984). Elements of episodic memory. New York: Oxford University Press. Tulving, E. (1985). Memory and consciousness. Canadian psychology, 26(1), Tulving, E. (2002). Episodic memory: from mind to brain. Annu Rev Psychol, 53, doi: /annurev.psych /1/1 [pii] Tulving, E., & Markowitsch, H. J. (1998). Episodic and declarative memory: role of the hippocampus. Hippocampus, 8(3), doi: /(SICI) (1998)8:3<198:: AID-HIPO2>3.0.CO;2-G Vann, S. D., Aggleton, J. P., & Maguire, E. A. (2009). What does the retrosplenial cortex do? Nat Rev Neurosci, 10(11), doi: nrn2733 [pii] /nrn2733 Vann, S. D., & Albasser, M. M. (2009). Hippocampal, retrosplenial, and prefrontal hypoactivity in a model of diencephalic amnesia: Evidence towards an interdependent subcortical-cortical memory network. Hippocampus, 19(11), doi: /hipo Vann, S. D., Brown, M. W., Erichsen, J. T., & Aggleton, J. P. (2000). Using fos imaging in the rat to reveal the anatomical extent of the disruptive effects of fornix lesions. J Neurosci, 20(21), doi: 20/21/8144 [pii] Vann, S. D., Tsivilis, D., Denby, C. E., Quamme, J. R., Yonelinas, A. P., Aggleton, J. P., et al. (2009). Impaired recollection but spared familiarity in patients with extended hippocampal 178

191 179 system damage revealed by 3 convergent methods. Proc Natl Acad Sci U S A, 106(13), doi: [pii] /pnas Vannest, J., Szaflarski, J. P., Privitera, M. D., Schefft, B. K., & Holland, S. K. (2008). Medial temporal fmri activation reflects memory lateralization and memory performance in patients with epilepsy. Epilepsy Behav, 12(3), doi: S (07) [pii] /j.yebeh Vargha-Khadem, F., Gadian, D. G., Watkins, K. E., Connelly, A., Van Paesschen, W., & Mishkin, M. (1997). Differential effects of early hippocampal pathology on episodic and semantic memory. Science, 277(5324), Viard, A., Piolino, P., Desgranges, B., Chetelat, G., Lebreton, K., Landeau, B., et al. (2007). Hippocampal activation for autobiographical memories over the entire lifetime in healthy aged subjects: an fmri study. Cereb Cortex, 17(10), doi: bhl153 [pii] /cercor/bhl153 Vilberg, K. L., & Rugg, M. D. (2008). Memory retrieval and the parietal cortex: a review of evidence from a dual-process perspective. Neuropsychologia, 46(7), doi: S (08) [pii] /j.neuropsychologia Viskontas, I. V., McAndrews, M. P., & Moscovitch, M. (2000). Remote episodic memory deficits in patients with unilateral temporal lobe epilepsy and excisions. J Neurosci, 20(15), doi: 20/15/5853 [pii] Voets, N. L., Adcock, J. E., Stacey, R., Hart, Y., Carpenter, K., Matthews, P. M., et al. (2009). Functional and structural changes in the memory network associated with left temporal lobe epilepsy. Hum Brain Mapp, 30(12), doi: /hbm Voltzenlogel, V., Despres, O., Vignal, J. P., Kehrli, P., & Manning, L. (2007). One-year postoperative autobiographical memory following unilateral temporal lobectomy for control of intractable epilepsy. Epilepsia, 48(3), doi: EPI970 [pii] /j x Voltzenlogel, V., Despres, O., Vignal, J. P., Steinhoff, B. J., Kehrli, P., & Manning, L. (2006). Remote memory in temporal lobe epilepsy. Epilepsia, 47(8), doi: EPI555 [pii] /j x 179

192 180 Walker, M., Chan, D., & Thom, M. (2007). Hippocampus and Human Disease: Mesial Temporal Lobe Epilepsy and Hippocampal Sclerosis. In P. Andersen, R. Morris, D. Amaral, T. Bliss & J. O'Keefe (Eds.), The Hippocampus Book (pp ). Oxford: Oxford University Press, Inc. Warrington, E. K. (1984). Recogniton Memory Test: Manual. Berkshire, UK.: NFER- Nelson. Wendel, J. D., Trenerry, M. R., Xu, Y. C., Sencakova, D., Cascino, G. D., Britton, J. W., et al. (2001). The relationship between quantitative T2 relaxometry and memory in nonlesional temporal lobe epilepsy. Epilepsia, 42(7), doi: epi25600 [pii] Westmacott, R., Leach, L., Freedman, M., & Moscovitch, M. (2001). Different patterns of autobiographical memory loss in semantic dementia and medial temporal lobe amnesia: a challenge to consolidation theory. Neurocase, 7(1), Wheeler, M. A., Stuss, D. T., & Tulving, E. (1997). Toward a theory of episodic memory: the frontal lobes and autonoetic consciousness. Psychol Bull, 121(3), Whitfield-Gabrieli, S., & Mozes, S. (2010). ARtifact Detection Tools: Retrieved from Wiebe, S. (2004). Effectiveness and safety of epilepsy surgery: what is the evidence? CNS Spectr, 9(2), , Wiebe, S., Blume, W. T., Girvin, J. P., & Eliasziw, M. (2001). A randomized, controlled trial of surgery for temporal-lobe epilepsy. N Engl J Med, 345(5), doi: /NEJM Wieser, H. G. (2004). ILAE Commission Report. Mesial temporal lobe epilepsy with hippocampal sclerosis. Epilepsia, 45(6), doi: /j x EPI09004 [pii] Wiltgen, B. J., Zhou, M., Cai, Y., Balaji, J., Karlsson, M. G., Parivash, S. N., et al. (2010). The hippocampus plays a selective role in the retrieval of detailed contextual memories. Curr Biol, 20(15), doi: S (10)00816-X [pii] /j.cub Winocur, G., & Moscovitch, M. (2011). Memory transformation and systems consolidation. J Int Neuropsychol Soc, 17(5), doi: S [pii] /S

193 181 Winocur, G., Moscovitch, M., & Bontempi, B. (2010). Memory formation and long-term retention in humans and animals: convergence towards a transformation account of hippocampal-neocortical interactions. Neuropsychologia, 48(8), doi: S (10) [pii] /j.neuropsychologia Wood, E. R., Dudchenko, P. A., & Eichenbaum, H. (1999). The global record of memory in hippocampal neuronal activity. Nature, 397(6720), doi: /17605 Yonelinas, A. P., Aly, M., Wang, W. C., & Koen, J. D. (2010). Recollection and familiarity: examining controversial assumptions and new directions. Hippocampus, 20(11), doi: /hipo

194 Appendix A Instructions - Behavioural Task This study has [two] three parts. - In the first parts, we will select a number of events from your personal life. [If AMs were selected in advance (before testing day, e.g. on the phone), read the selected AM titles with the participant to make sure they remember the event, and skip to part 2, encoding] - In the second phase, you will be shown stories which you will need to remember for later. - In the third phase, you will be required to remember the events from your life that we selected together, and the stories you were shown, and you will answer some questions about each of them. The first two phases are fairly short, and the third phase takes the longest, and it requires the most input from you. 1. Selecting Autobiographical Events [To reduce testing time in the lab, AMs can be selected before testing day, e.g. over the phone] I need you to select 11 events from your personal life. Each of the events must have taken place at least one year ago. The events must be unique, and they should be capsules of your life that are very specific in time: they should have taken place within a few minutes, to about an hour. For example, your memory of a full weekend away would be too long, but your memory of having lunch somewhere special would be good, or it could be even more specific, like your memory of ordering something, or paying the bill at the restaurant. Keep in mind that you will need to answer questions about the events you select, so just make sure you choose events that you feel comfortable talking about. I will give you a list of suggestions for events*, which you can use for inspiration. You will read down the list, and as soon as you can think of a memory, let me know. The events you select do not need to be on the list, so if examples make you think of other personal events, that s good too. Once you have a memory, we will select a title for it. This title is important because it will be shown to you later in the study, and it will be your cue to think about this specific event from your life. So we need to make sure that by reading this title, you will know which event we are referring to. *[If over the phone, send in the list by . Ask participant to have it opened in front of them for the phone interview, and ask them to read down the list. If they don t have the internet at home, read them suggestions from the list, one at a time, over the phone.] [When participants think of an event, ask them to give you the gist of that event, and suggest a title for it. Some participants will come up with good titles, but a lot of people will pick vague titles that may not be good retrieval cues (e.g. dinner with a friend, instead of dinner at Frank s Kitchen with Gene ). Write down title, how long ago the event took place, and its duration (e.g. did it take place over 5 minutes, 2 hours, a full day?)]. 182

195 APPENDICES Encoding of the stories You will be shown short stories in two different formats. The first kind of stories will be shown to you as film clips. You will need to pay attention to the story that is taking place in the clip, what each of the characters is doing, what the situation is, etc, as you will be required to remember that story later, in your own words. The second kind of stories will be written on the computer screen. You will need to read them, one sentence at a time, and a voice-over will also play that will take you through the story. Again, pay attention to what is going on, what each of the characters is doing, as you will be asked to remember this story for later, in your own words. Both the clips and the written stories will have a title, shown before and after the story is played. Pay attention to this title, as it will be shown to you later as your cue that this is the story you need to retrieve. While most people find that the voiceover helps, some people find that reading the stories and listening to the voice-over at the same time is distracting, because they read at a different speed than the voice [N.B. Only 1-2 people have made that complaint]. If that is the case for you, you may choose to listen to the voice only instead of trying to read at the same time; do whatever works best so that you get to understand the story. 3. Retrieval [There are a lot of instructions; if at any point participants start feeling overwhelmed, you can interrupt giving the instructions, and mention that you will do a practice with them to make sure they know what to do.] You will be shown titles of the personal memories we ve selected earlier, and titles of the stories you ve studied. For each title, you will be given 16 seconds to retrieve the personal memory or the story that corresponds to this title. For the personal memories, I want you try and put yourself back in the event, and try to re-live it in your mind, in as much details as possible, as if the scene was playing in your mind s eye. For the stories, tell yourself what happened in the story, from beginning to end, as if you were playing it in your mind. You don t need to speak during the 16s, just think. After the 16 seconds is over, you will be asked questions about what you had time to think about within the 16 seconds when you thought of the story or personal memory. First, you will need to rate your retrieval success, on a scale of 1-5 [Point to keypad]. By retrieval success, I mean how much information you remembered about what happened during the personal memory or the story. While using the scale, keep in mind that you are using a different scale for the personal memories and for the stories. A 5 for a personal memory might mean something different than a 5 for a story, since you have a lot more information about your own life.

196 APPENDICES For the personal memories, 1 means that you did not retrieve any information about the memory, while 5 means that this is one of the personal memories for which you remembered the most information, within the context of this task. - For the stories, 1 means you did not retrieve any information about the story, while 5 means you think you retrieved all the story details. Then, you will need to rate the vividness of your memory on a scale of 1 to 5. By vividness, I mean how much visual, auditory and other sensory details were contained in your memory. For both the stories and the memories, you might have an image of what the room looked like, what was posted on the wall, pieces of furniture, what people were eating, people s faces and expressions, where people were sitting, whether it was dark or light, what people were wearing. You might also have re-experienced sounds, like laughter, or voices or street sounds. For personal memories, you might have also re-experienced smells (e.g. food, cooking, rain ) or you might have remembered being in pain or being tired, feeling on your skin, or other physical sensations. I would like you to rate how much of these visual and other sensory details you retrieved with the memory, on a scale of 1-5. Again, keep in mind that you are using a different scale for the personal memories and for the stories, so a 5 may mean something different for a personal memory than for a story. HOWEVER, you are comparing the videos and the written stories on the same scale. - For the personal memories, 1 means no visual and other sensory details, while 5 means this is one of the personal memories which evoked the most visual and sensory imagery within the 16s of retrieval. - For the stories, 1 means no visual details, while 5 means this is one of the stories for which you retrieved the most visual and sensory details. Now, you are likely to retrieve visual details for the videos, since they were shown to you visually. Also, some of the written stories may or may not evoke images in your mind when you tell yourself the story. For example, thinking of the story may evoke the image of a person, their face or shape, the setting in which it is taking place, even though you were not shown anything at encoding. Or you may not have any image at all, there is no right or wrong. I am not asking you to make an effort to evoke visual images when retrieving the written story, I simply want you to try and remember the story. But, if images or sounds naturally pop in your mind during the 16s when you tell yourself that story, I would like you to report how vivid these were, using the scale. The next two questions will require you to speak into the microphone. Firstly, I would like you to tell me, in your own words, all the memory details you had time to retrieve within the 16 seconds period, more specifically in terms of what happened over the course of the personal memory or what happened in the story. Lastly, I will ask you to tell me all the visual and other sensory details that were evoked in your mind s eye during the 16s. For example, like I said earlier, you may have imagined people s faces, clothes, what the environment or the scenery looked like, features of buildings, furniture, objects in the room, whether it was dark or light, sounds, etc. For the personal

197 APPENDICES 185 memories, you may also have re-experienced smells, temperature, feelings on your skin, things like that. Please list every single one of the details you saw or perceived in your mind s eye. 4. Practise Trial [Sit with participant through practice trial. They will be shown 1 new clip, and 1 new narrative to encode. Then, they will be tested on these two stories, as well as on one of their AMs selected for practice. For the first practice trial, tell them what to do at each step:] This is the 16s when you need to think of your memory / tell yourself what happened in the story, in your mind. Now press the button. From 1-5, how much information did you retrieve about the memory / story. Now tell me, did you have an image in your mind while thinking of the memory / story, and how vivid was it, from 1-5? Now it s recording. Tell me, in your own words, all the story / memory details you had time to think about during the 16s, in terms of who did what, what happened Now tell me, in your own words, any visual or perceptual details you saw in your mind while thinking of the story / memory, in terms of images, sounds, things like that. [If participants are a bit confused with the steps at the end of the practice trial, sit with them through the first few trials of retrieval until they get the hang of it].

198 Appendix B Autobiographical Event Cues N.B. The following cues are suggestions; they may help you think of personal events that are not on the list, but that still meet our criteria. CELEBRATIONS A wedding toast Pronouncing wedding vows The highlight of your / someone s wedding reception Bride and groom walking down the aisle Christmas / holiday / family dinner A special New Years eve (e.g. year 2000) A birthday celebration / a surprise party A special church event Confirmation/bar mitzvah A holiday celebration A Halloween party The highlight of your prom night Graduation ceremony FAMILY Getting together with a relative Leaving a child with the babysitter for the first time A special moment with a child First day of school for a child / yourself Getting a pet Bringing a pet home for the first time Taking care of a sick child or spouse Staying home with a sick child / picking a sick or hurt child from school Providing care to an aging parent or relative Taking care of a sick pet Child walking for the first time Finding out you are / someone is pregnant Yourself / your partner giving birth to a child Birth of your relative s or friend s children Meeting a friend s baby or child for the first time Meeting / spending time with your in-laws Going out to a favorite place with spouse, partner, friend, group of friends (e.g. favorite restaurant, local pub ) Your children/grandchildren graduation 186

199 APPENDICES 187 Your child s first day at school Cooking with the family WORK Babysitting a young child Receiving a promotion Making a mistake at work Realizing you forgot to do something important (e.g. paying your taxes, or rent, or writing up a report at work, or a class assignment) A work dinner A business meeting Accepting / refusing a job A job interview Incident at work First day at a new job Last day of work Your / someone s retirement party Making a good / bad business deal Being fired or resigning from a job TRAVELLING Leaving on a trip Boarding into a plane Taking a boat / cruise / ferry Buying plane tickets / booking through a travelling agency (e.g. final decision) Checking in at a bad / ok / amazing hotel Loosing luggage Going on a road trip Taking a ferry A memorable boat tour / cruise Loosing your camera Taking a special picture A special / memorable / unpleasant commute Meeting someone strange / interesting / scary looking on the subway or bus Seeing a famous landmark for the first time (e.g. Eiffel Tower) Driving in the country side Wine tasting A road trip with family / friends Camping outdoors Hiking in the mountains Skiing A memorable train ride Walking down the streets of a new city or down Walking down the beach / by the ocean

200 APPENDICES 188 Getting a sun burn First time driving/first driving lesson FOOD and ENTERTAINEMENT Buying concerts ticket Lining-up for a concert, show, etc. A memorable concert (e.g. when they played a favorite song) Going to see someone famous Meeting someone famous Inviting / being invited by someone to the concert, movie, dinner, etc Going to the movie / watching a memorable movie at home Visiting a famous museum, a special exhibit Cooking something special for the first time Someone cooking for your / Dinner invitation Having a BBQ with friends or family An interesting / amazing / horrible meal Trying out a new restaurant Sipping on good wine Smoking pot Trying a cigarette/drugs/alcohol for the first time Taking a picture of someone / something (e.g. place) special Winning / loosing at the casino A night at the comedy club Watching an important sporting event (at the stadium, on tv, at the bar, etc) Your team winning / loosing an important game Going clubbing A well deserved cigarette Going to a memorable opera, ballet, play ROMANTIC Giving/receiving a romantic gift Getting engaged Meeting a friend or partner for the first time (e.g. first impression) Asking someone out on a date Meeting someone online Being asked on a date Good date / bad date Your first date / kiss with a new partner Your best valentine s day SOCIAL Saying goodbye to someone Goodbye party Dropping / welcoming someone at the airport

201 APPENDICES 189 Giving out something you made yourself (e.g. knitted mitts, baby blanket) Visiting a relative or a friend s place Attending a club s meeting / social event Attending a partner s club s meeting / social event Bumping into an old friend in unexpected circumstances Hurting someone s feelings by telling them the truth Contacting someone you lost touch with Catching up with a friend Getting to know someone better Finding someone on Facebook Being visited by parents / children Cheering up someone Meeting someone interesting on the plane / train Helping someone to build something Disagreeing about house chores Being angry at a partner or friend without telling them Telling a lie / hiding the truth Finding out someone lied to you See someone steal something Stealing something Sharing a cigarette with a friend HEALTH & SICKNESS / INJURY Cutting / injuring yourself Breaking something Waiting in the hospital Visiting someone in the hospital Learning about / being present for someone s death Attending a funeral Getting sick (e.g. food poisoning) Drinking too much Having a hangover Being hospitalized Staying home sick Doing something dumb / risky / unsafe Being scolded for doing something dumb / unsafe Learning about someone s condition (disease, pregnancy, death, birth) Witnessing an accident / someone getting hurt / doing something risky Getting one or more cavities fixed Getting a root canal Waking up from having surgery Being in a car accident HASSLES

202 APPENDICES 190 Shoveling snow / getting a car unstuck / pushing a car out of the snow bank A stressful time driving A scary car ride Fixing a car Being lost Breaking / damaging something you borrowed Staining a library book, a friend s book Being arrested by the police Burning a meal Driving into a storm Loosing something important (e.g. document, jewelry) Getting your wallet stolen Being caught in bad weather / a storm / natural disaster MATERIAL STUFF Putting together IKEA furniture Buying a car (e.g. trying out a car; driving it home) Buying technology (your first i-pod, cell phone, blackberry) Receiving something special Purchasing something special (e.g. a favorite pair of shoes, clothing, a piece of furniture) Purchasing a gift for someone Giving a gift to someone Getting a bike for a child Building / constructing / renovating a house or cottage MOVING Arriving into a new city (e.g. getting off the plane) Buying / selling a house (e.g. signing the papers, or visiting the house for the first time) Visiting a house / apartment Cleaning day before moving out / after moving in Moving out of your parents house Moving out of an apartment / house First time grocery shopping in new neighborhood SCHOOL A special day at school (e.g. giving a presentation) Getting a good / bad mark A memorable discussion with a teacher / professor Signing up for a class (e.g. gym class, cooking class, language class, etc). First/Last day of primary/ middle/ high school/university Cheating on a test Taking an important exam/standardized test SURPRISE

203 APPENDICES 191 Opening an important letter Receiving an important Sending an important Winning the lottery Winning an award / prize MONEY Receiving a much needed paycheck Paying debts Paying someone else back Getting someone to pay you back what they owe you A big purchase FITNESS Jogging in an interesting place Playing a competitive sport Playing sports in the backyard with family / friends A great play (sports) Sports performance (e.g. winning / loosing in the finals of a tournament; counter performance) Finding your dream yoga studio Getting a massage Playing golf Incident while walking the dog Pool tournament POLITICS Being involved in / witnessing a protest rally Being involved in an intense political argument Watching a political debate on tv Voting Voting for the first time A political celebration / gathering PUBLIC PERFORMANCE Being on TV, radio, Newspaper Speaking in public (e.g. wedding toast, speech, teaching a class, giving training to colleagues) Being parts of a play / performance Giving a presentation / talk

204 Appendix C List of Laboratory Events - Narratives and Clips Narratives included in both the Behavioural and fmri protocols: Story 1 Boy Getting into Dad's Car Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 2 A boy opens the passenger's car door and gets in, his mom standing behind him. The mom grabs the boy's briefcase, wipes it with a rag and hands it back to him. She closes the car's door and starts wiping the car as it slowly drives away. She follows behind, still wiping as the car drives into the street. She finally stops and waves goodbye with her dusty rag. Boy, Girl and Balloons Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 A little girl holding a blue balloon stands by a little boy holding a red balloon. They both start walking in opposite directions with their balloon. The girl lets go of her blue balloon, which floats towards the boy. She runs up to the boy as he catches her balloon. He hands it back to her, and they walk away from each other again. 192

205 APPENDICES 193 Story 3 Boys Faking Car Accident Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 4 A group of bored kids are walking on the sidewalk. Cars on the street have stopped at a red light. One boy signals his friend to get ready. With his foot, he pushes down on a car's bumper. Simultaneously, his friend hits a garbage lid with a stick, making a car crash sound. The car's driver storms out and yells at the driver behind him, while the boys watch in glee. Boys Watching Television Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 5 Two boys are sitting inches away from a large TV screen, captivated by a soccer game. Their team scores, and they cheer loudly. One boy reaches for the TV, and rewinds what seems to be a video tape of the match. The two boys watch the goal all over again, seemingly caught up in the suspense. As the ball goes in, they cheer just as loud as the first time. Bullied Boy and Mimes

206 APPENDICES 194 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 A young boy is standing by himself on the street. Two bullies run up to him, push him to the ground and run away. The young boy gets up, holding the glasses he lost. He notices two mimes are watching him. The mimes are frowning, but both pass their hands in front of their face, turning their frowns into smiles. The boy puts on his glasses and smiles back. He turns around and starts running. Story 6 Couple Taking a Photo Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 7 A man sets up a camera on a fence. He points it towards a woman waiting for him on the opposite side of the street. He runs across the street towards her, then turns around so they both face the camera. She tries to hold his arm, but he yanks it away. They wait. A car drives between them and the camera, just as the camera is triggered. They look at each other, then start to laugh. Crashing the Bicycle

207 APPENDICES 195 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 8 A boy and his dad are riding a bike, with the boy sitting on the handlebars. They are going down a hill. The father squeezes the brakes to slow them down. He realizes that the brakes are broken. They both scream as the bike accelerates. The dad tries to brake with his shoes, without success. They hit a tree on the side of the road at full speed and fall off the bike. Cyclist Waiting for Her Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 9 An awkward young man is standing by his bicycle, waiting outside a building. The young woman he was hoping to see walks out. She says hello and walks away. He starts after her, forgetting his bike. He returns for the bike, then runs behind her, calling her name. They start chatting as she walks and he cycles next to her. Girl Chasing her Shoe

208 APPENDICES 196 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 10 As a little girl jumps over a narrow street gutter, her shoe falls into the water. The shoe is quickly taken away by the running water. She chases the shoe, avoiding trees and obstacles along the sidewalk. The shoe disappears under a small bridge and gets lodged in some debris. The little girl bends over but cannot reach the shoe. She is ready to cry. Girl Meets Boy at the Mosque Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 11 A young woman walks out of a mosque. A boy her age sitting across the street recognizes her. They make eye contact and smile at each other. Just then an older man walks out behind her. The boy's face drops. The older man grabs the girl by the shoulders, then notices the boy. The man whisks the girl away. Grandfather Babysitting Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 12 An old man is standing outside by his grandson in a baby carriage. He is holding one of the baby's squeaky toys, which he stuffs into his back pocket. As he sits down, the toy in his pocket squeaks. He lights up a cigarette. Accidentally, he breathes smoke in the baby's face. The baby starts screaming. The man swears grumpily, unsure what to do. Gunman at the Window

209 APPENDICES 197 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 13 A middle-aged man withdraws a hunting gun from his cabinet. Determined, the armed man walks to a nearby window and opens it wide. Outside on the side-walk, a young woman is playing with an automated stuffed puppy. Coldly, the gunman aims at the toy and shoots it. Startled and unsure, the woman looks towards the window. Man Against Locked Door Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 14 A nervous-looking orchestra conductor knocks against a locked backstage door. He steps back, then runs into the door, shoulder first. Inside, two men are startled by the noise. As the conductor steps back to renew his assault, they pull the door open. The conductor tumbles through and falls to the floor. Embarrassed, he gets up and rubs his arm. The two men shake hands with the conductor awkwardly. Man Stuck in Phone Booth

210 APPENDICES 198 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 15 A man reaches for coins in his back pocket. He walks into a phone booth and tries to place a call. As he fiddles with the phone, the booth's door closes behind him. He realizes the phone is broken. He hangs up and turns around to get out. The booth's door is stuck. He pushes the door with both hands but can't open it. No Paper in the Washroom Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 16 A man just washed his hands in a public washroom. He presses the handle of the paper distributing machine, but it's empty. He attempts to use the hand dryer, but it is very weak and short lasting. After 3 attempts, he gives up and reaches for toilet paper. The toilet paper distributor is also empty. Parents Arguing Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 17 A couple is arguing ferociously inside their house. Their young daughter is sitting at a piano in the room next door. She overhears everything. Upset, she starts to pound on the piano with all her strength. The fight intensifies. The girl keeps pounding on the piano. The parents notice the noise and stop screaming. The dad grabs the girl's hand, softly. Penalty Kick

211 APPENDICES 199 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 18 A soccer player has been awarded a penalty shot. He adjusts his collar while all the other players watch him. He looks the keeper in the eyes. The referee blows the whistle. The man runs towards the ball and kicks it so hard that it flies way above the net. The player follows the ball with his gaze, covering his mouth in shock. Son Leaving Parental Home Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 19 A middle-aged couple watches their son as he finishes packing his car. They smile awkwardly as he grabs the last box and puts it in the car. He sits behind the wheel, honks and waves goodbye. They wave back, playing it cool. The son puts the car into gear. As he drives off, the moving trailer he was pulling gets detached and stays behind. The parents cannot hide their concern. The Teens and the Fruit Seller

212 APPENDICES 200 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 20 A teenage boy happily shows his pal what seems like a romantic note. His friend starts teasing him. The first boy shoves him off. Laughing, the friend walks away. As he passes a fruit stand he grabs a piece of fruit. The stall owner scolds him, but he throws the fruit over his shoulder. The boy with the note catches the fruit, laughs, and gives it back to the stall owner. Woman Squeezing Food Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 An elderly woman is in a food store, handling a peach. She squeezes the fruit so hard that is bursts and splatters her in the face. The man behind the counter gives her an angry look. Embarrassed, she vanishes down one of the aisles, while he follows her. She starts squeezing a soft cheese with her thumbs, looking delighted.

213 APPENDICES Additional narratives included in the fmri protocol only: Story 21 Boy Making a Soccer Jersey Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 22 A shirtless boy is laying on the floor, drawing on a t-shirt with a big marker pen. He puts the shirt on. The drawing makes it look like a soccer jersey. Proud of his work, the boy looks at himself in the mirror from all angles. In front of the mirror, he makes a little victory dance. He folds the bottom of his shirt over his head to cover his face, like a soccer pro. Boys Chasing the Balloon Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 23 Three bullies in ambush watch as a young boy walks by holding a red balloon. The bullies start chasing the boy. The boy tries to escape them by running up a staircase. Out of nowhere, another group of boys starts running down the stairs. He is caught between them. He lets go of the red balloon. He runs through the crowd of boys who watch the balloon float away. He reaches the top of the stairs and waits as the balloon descends towards him. He grabs it and runs. Couple by the Sea

214 APPENDICES 202 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 24 A couple is hanging out by the sea shore. As he takes her picture, a small wave splashes her. She closes her eyes and waits for him to dry her face with a Kleenex. A gigantic wave hits them both. It takes away her hat and rips their clothes off. To their astonishment, the Kleenex he is holding is still intact. Couple Meeting at the Café Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 25 A woman walks into a café. She skips towards a man waiving at her from his table. He gets up to welcome her, and kisses her on the cheek. She closes her eyes for a moment. They sit down facing each other. She sighs and smiles softly. He seems lost in a daze, distant. He suddenly thinks of sharing his drink, and offers her his glass. She takes a huge sip. Dog Escaping

215 APPENDICES 203 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 26 A woman stands on the street as she unlocks the door to her apartment. As she opens the door, her massive dog pushes his way out. The dog runs off. She turns and screams his name, irritated. She chases after him, but cannot catch up with him. Annoyed, she stomps back towards the apartment and disappears inside. Drink for the Pianist Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 27 A barmaid at the opera is bringing a drink to the pianist's lounge. She knocks on his door a few times. Through the door, she can hear him practicing. The sound of the piano covers her knocking. She decides to enter the lounge. Without him noticing her, she tiptoes across the room. She places the drink on a table, loudly. The pianist stops playing and turns around, startled. Family Painting a Cottage

216 APPENDICES 204 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 28 Two women and a child are painting a cottage. An energetic man brings them a huge bucket of paint. He grabs the paint brush from one of the women who is pregnant. He starts painting frantically. She shrugs her shoulders and walks off, annoyed by his presence. Girl Breaking an Egg Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 29 In school, a little girl cracks an egg against the side of a bowl while her classmates watch. As she opens the cracked egg, some of it splatters in her eye. The rest falls outside the bowl onto the table. The kids laugh. She rubs her eye, while her teacher takes the empty shells from her. The teacher uses a spoon to scoop the egg from the table into the bowl. Girl Falling in front of Boys (this story was always presented as a narrative due to sound issues with the video) Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 30 Three young men are sitting and chatting as a girl walks by. She trips over a rock and falls to the ground, dropping her bag. Two of the boys laugh, while the third one looks on concerned. He runs over to her, picks up her bag, and helps her up. His friends make fun of him but he gives them a dirty look. Late Boy Sitting in School

217 APPENDICES 205 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 31 A boy walks into his classroom. Everyone is already sitting down. He closes the door behind him and tiptoes to his bench, shyly. His two bench mates get up to let him sit down. He crawls into his seat, and the other two boys sit back. Quietly, he gets his books and material out from his schoolbag. Meeting the Flute Player Slide 1 A teenage boy is sitting at the window of an abandoned house. He is playing the flute. Slide 2 His date walks into the house to meet up with him. She hears the flute and smiles. Slide 3 He sees her enter, and decides to hide to scare her. Slide 4 While he looks cautiously inside one of the rooms, she tiptoes behind him. Slide 5 All of a sudden, she grabs his flute and takes it away, coyly. Story 32 Old Man at the Movie

218 APPENDICES 206 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 33 An elderly man rides his bicycle on a road that leads to an old movie theater. He parks his bike in front of the theater, which looks abandoned. Walking inside, he purchases a single ticket from a person behind the ticket booth. He is the only viewer in the theatre. He smokes, waiting for the movie to start. The technician shouts, the lights go off and the movie begins. Picking a Hitchhiker Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 34 A young man is driving alone on a road in the desert. He passes an attractive female hitchhiker holding a sign that says "West". The driver hesitates for a split-second, then tells himself "why not!" He pulls over and waits for the hitchhiker, who runs up to him. She takes off her shades as he excitedly opens the passenger's window. Policeman Flirting Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 35 A traffic officer sees a beautiful woman and her little girl, waiting to cross the street. He takes off his shades and walks towards them, smoothly. With confidence, he blows his whistle to stop the traffic for them. He tries to help the woman with the little cart she is pushing. Unimpressed, she brushes him off and walks away. Sleeping by the Road

219 APPENDICES 207 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 36 A poor boy is pulling a cart on a dusty deserted road. He comes across another child sleeping by the side of the road. The boy approaches the sleeper and pokes his cheek. He pours a little bit of water into the sleeper's face. The sleeper opens his eyes suddenly. They are both startled. Tender Moment in the Kitchen Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 37 A man seems lost in his thoughts. He looks at his wife across the room who is cooking. She sings to herself as she spoons pasta onto a dish. He walks towards her but she doesn't hear him approaching. He embraces her from behind, and kisses her on the cheek. She smiles and stops working for a moment. The Husband's Cigarette

220 APPENDICES 208 Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 38 A man stands on his porch as his wife hands him a pack of cigarettes. He brings one to his lips. As he waits there expectantly, she frantically wipes a window, then rushes inside the house. Calmly, he puts the pack of cigarettes inside his vest, as she comes back with his lighter. He lights up as she adjusts his vest. She disappears inside the house as he enjoys his cigarette. The Woman and the Shell Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Story 39 A woman walks into a museum. She throws her coat on a chair. She walks over to a table showing a sea shell exhibit. She brings one to her ear. She accidentally drops the shell, which makes a loud noise as it shatters. Boldly, she reaches into her bag for a shell-shaped pasta, which she leaves as replacement. She writes a descriptive note for the pasta shell, grabs her coat and runs off. Two Men Toasting Slide 1 Slide 2 Slide 3 A middle-aged man enters his living room. He turns on a lamp. A younger man is already sitting on the couch, looking nervous. Smoothly, the older man puts two wine glasses on the coffee table.

221 APPENDICES 209 Slide 4 Slide 5 Story 40 He sits by the young man, and pours wine into both glasses. They toast, the young man timidly, the older man with more assurance. Woman Buying Lettuce Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 An elderly woman walks up to an outdoors vegetable stand. She picks up a head of lettuce. The vegetable stall owner is sitting at a nearby terrace, drinking with a friend. From a distance, she shows him the lettuce. He shakes his head in disapproval and points to a different lettuce, which she picks instead. He nods in approval. She picks one more head of lettuce and leaves a few coins in his money jar.