Multiple Signals of Recognition Memory in the Medial Temporal Lobe

Transcription

1 Multiple Signals of Recognition Memory in the Medial Temporal Lobe Michael A. Yassa 1,2 and Craig E.L. Stark 1,2 * HIPPOCAMPUS 18: (2008) ABSTRACT: The medial temporal lobe (MTL) is known to play an essential role in recognition memory (the ability to judge the prior occurrence of a stimulus). Electrophysiological studies in nonhuman primates have suggested the presence of more than one type of recognition signal in the medial temporal lobe (e.g., novelty, familiarity, and recency). It has also been suggested that the perirhinal cortex plays an essential role in visual recognition memory. Here, we present fmri results from 16 college-aged participants who underwent a continuous yes/no recognition task of novel and familiar pictures with multiple stimulus presentations. Our goal was to understand the dynamics of recognition in the MTL over multiple trials. We hypothesized that we could see changes in signal with repeated exposure that carry information related to novelty, familiarity, and recency. Whole brain activation maps demonstrated a strong novelty effect, marked by activity in several frontal and occipital regions that decreases with increasing number of presentations. The opposite pattern was observed in several other regions that include the supramarginal gyrus and inferior parietal lobule. In the MTL region, we observed monotonic decreases in activity across trials in the parahippocampal cortex as well as the anterior perirhinal cortex. We also observed monotonic increases in activity in the posterior perirhinal cortex with increasing memory strength. In addition, we observed clear effects of pre-experimental familiarity with the stimulus in several regions. Consistent with previously reported electrophysiological data, we found evidence for several medial temporal lobe signals carrying recency, familiarity, and novelty information. VC 2008 Wiley-Liss, Inc. KEY WORDS: fmri; familiarity; recency; novelty; perirhinal cortex; parahippocampal cortex INTRODUCTION The medial temporal lobe (MTL) plays an essential role in declarative (fact and event) memory processing (Milner et al., 1998). It consists of the hippocampal region (dentate gyrus (DG), CA subfields and subiculum) and surrounding perirhinal, entorhinal, and parahippocampal cortices. Recognition memory, or the ability to judge the prior occurrence of a stimulus (Brown, 1996), is one type of declarative memory thought to be subserved by MTL structures. For example, patients such as H.M. and E.P., who have MTL damage have severe deficits in the ability to 1 Center for Neurobiology of Learning and Memory, Department of Neurobiology and Behavior, University of California, Irvine, California; 2 Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, Maryland Grant sponsor: National Science Foundation; Grant number: BCS *Correspondence to: Craig E. L. Stark, Center for Neurobiology of Learning and Memory, Department of Neurobiology and Behavior, University of California, Irvine, 213 Qureshey Research Laboratory, Irvine, CA cestark@uci.edu Accepted for publication 7 April 2008 DOI /hipo Published online 20 May 2008 in Wiley InterScience ( wiley.com). recognize information over a delay between study and test phases (see Squire et al., 2004 for a review). More specifically, converging evidence from lesion (Meunier et al., 1993; Suzuki et al., 1993; Bachevalier et al., 2002), electrophysiological (Brown et al., 1987; Fahy et al., 1993; Li et al., 1993; Brown, 1996; Brown and Xiang, 1998; Xiang and Brown, 1998), and fmri (Cho, 2000; Henson et al., 2003) studies have suggested that the surrounding medial temporal cortices including the perirhinal cortex contribute to recognition memory. Further, restricted damage to the perirhinal cortex alone has been found sufficient to produce a severe visual recognition memory impairment (Murray, 1992; Meunier et al., 1993; Suzuki et al., 1993; Zola-Morgan et al., 1993; Eacott et al., 1994; Buffalo et al., 1999). However, the specific signals or mechanisms of recognition memory in the perirhinal cortex are still unclear. Brown and Aggleton (2001) extensively reviewed the role of the perirhinal cortex and the hippocampus in recognition memory and found evidence for several types of recognition-related signals. Electrophysiological recordings from the perirhinal cortex of monkeys during performance of recognition tasks have shown that some neurons decrease their firing rate at subsequent presentations of the same stimuli (Brown, 1996; Brown and Xiang, 1998). This change in neuronal activity has been taken as an indicator of recognition. An additional possible substrate of recognition is enhanced activity with repeated exposures, although this effect is only found in some studies and in a minority of the neurons sampled (Miller et al., 1993; Miller and Desimone, 1994; Xiang and Brown, 1998). Within the populations of neurons that decrease their activity to subsequent presentations of the same stimulus, Brown and Xiang found evidence for three distinct patterns of activity that could give rise to recognition judgments. Some neurons signal recency by decreasing activity to repetition of either new or old stimuli. Recency can be thought of as reflecting recent exposure to the stimulus, regardless of the level of pre-experimental familiarity with it. Other neurons signal familiarity by decreasing activity to old and not new stimuli, while remaining insensitive to repetition. Finally, a third population of neurons signal novelty by decreasing activity in response to repetitions of new stimuli but not to repetitions of old stimuli (ones that are highly familiar outside the experiment). Additionally, novelty neurons seem to also code for familiarity VC 2008 WILEY-LISS, INC.

2 946 YASSA AND STARK by firing shorter bursts for familiar stimuli (Brown and Aggleton, 2001). Decreases in activity with repeated exposure have also been found in human functional imaging studies of the MTL (Stern et al., 1996; Cho, 2000). Increases in activity with successful recognition have also been observed (e.g., Stark and Squire, 2000). Isolating such retrieval-related activity has often been hampered by activity linked to incidental encoding of novel foil items on par with the activity linked to successful recognition of target items (Martin, 1999; Schacter and Wagner, 1999; Henson, 2005). Stark and Okado (2003) provided a direct demonstration of this effect (see also Stark and Squire, 2000). Consistent with many other reports, they observed that activity during a study phase predicted subsequent recognition performance. In addition, during the test phase, MTL activity for the novel foil stimuli correlated with subsequent memory for these stimuli. That is, during recognition, these stimuli were being incidentally encoded. This encoding-related activity was shown to obscure recognition-related activity. These results present a major challenge to neuroimaging studies attempting to study recognition in the MTL. In this study, we hypothesized that a clearer picture of recognition memory signals might be obtained if stimuli were presented multiple times during scanning and if activity associated with new, old, and repeated stimuli were assessed. By presenting stimuli multiple times (here, in a continuous recognition memory paradigm) the novelty effects typically associated with encoding processes might be observed during their habituation. This repetition-related decrease in activity has been demonstrated reliably in the past both within and outside the MTL (Squire, 1992; Buckner et al., 1995; Desimone, 1996; Buckner et al., 1998a,b; Schacter and Buckner, 1998; Henson, 2003; Schacter et al., 2007). Further, we might observe the effect of increasing memory strength as items are successfully recognized multiple times. Recent studies have found increasing MTL signals correlated with enhanced familiarity or memory strength (Gonsalves et al., 2005; Law et al., 2005; Daselaar et al., 2006). Finally, we hypothesized that we might be able to contrast novelty, familiarity and recency signals in the MTL. We report evidence of a dynamic MTL network that changes activity with repeated exposure; we found multiple recognition-related signals that are consistent with previous electrophysiological reports. MATERIALS AND METHODS Continuous Picture Recognition Paradigm Novel pictures were displayed one at a time (2,500 ms, 500 ms interstimulus interval see Fig. 1). A total of 160 new pictures were presented 1 4 times throughout the task. An additional 40 reference pictures (well known pictures, such as the Mona Lisa or the Eiffel tower) were intermixed throughout the task, and only viewed once during the task. Pictures used for the reference trials were not only familiar in general but also FIGURE 1. Paradigm and timing parameters. Stimuli are presented for 2,500 ms, with an interstimulus interval of 500 ms. Reference, new, and repeated stimuli were intermixed throughout the task in random order. studied once before the scan session. These trials served as a common nonzero baseline for the task. Repetitions of new stimuli were spaced randomly between 10 and 25 trials apart. Stimuli were repeated a random number of times between 0 and 3 (i.e., 40 stimuli were never repeated, 40 were repeated only once, 40 were repeated twice and 40 were repeated three times). The random presentation of old, new and repeated stimuli ensured that the spacing between stimuli of any given type approximated an exponential distribution to increase the power of our event-related analyses (Dale, 1999). It also assured that participants could not predict whether or not the next trial would be a new trial, a repetition, or a reference trial. New stimuli were selected from a collection of outdoor and indoor pictures. Reference stimuli were collected using an Internet image search. All images were adjusted for size and brightness. The task was programmed using the Cogent Toolbox (Wellcome Department of Imaging Neuroscience, University College, London, UK) for MATLAB 7.0 (MathWorks, Sherborn, MA). Participants were instructed to press one button if the stimulus was seen before in the context of the experiment or the pre-scan study session (i.e., the reference items and any repetition of the new items) and press another button if the stimulus was never seen before (new items only). Participant Selection Participants were healthy volunteers, recruited from the Johns Hopkins University community by flyer advertising, word of mouth, as well as from a pool of undergraduate students participating in research for extra-credit in introductory psychology classes. We scanned 16 participants in our fmri experiment (10 females, 6 males, mean age 22 years, standard deviation (SD) 4). We obtained informed consent from all participants.

3 RECOGNITION MEMORY SIGNALS IN THE MTL 947 Behavioral Data Analysis Mean response rates were broken down by hits (HT), misses (MS), false alarms (FA) and correct rejections (CR). Mean response latencies (in milliseconds) for each of the response rates were calculated for each participant in each condition (1, 2, 3, and 4 presentations, in addition to the reference condition). Mean hit rates as well as response latencies on hit trials were compared across the second, third and fourth presentation conditions, using repeated measures analyses of variance (ANOVA). Analyses were Greenhouse-Geisser corrected for error nonsphericity. Behavioral data, as well as mean beta coefficients from the ROIs generated in the fmri data analysis were analyzed using SPSS 15.0 (SPSS, Inc., Chicago IL). Magnetic Resonance Imaging (MRI) Data Acquisition Images were acquired at the F.M. Kirby Research Center for Functional Brain Imaging at the Kennedy Krieger Institute (Baltimore, MD), using a Philips 3.0 T MRI scanner (Philips Medical Systems, Eindhoven, The Netherlands) equipped with an 8-channel sensitivity encoding (SENSE) head coil operating with a SENSE reduction factor of 2. Functional T2*-weighted images were acquired using an echo planar single-shot pulse sequence with a matrix size of , an echo time (TE) of 30 ms, flip angle of 758, a repetition time (TR) of 2,000 ms, an in-plane resolution of 3 mm 3 3 mm and a slice thickness of 3.0 mm with a 1.0 mm interslice gap. Slices were acquired as triple oblique axials, aligned parallel to the long axis of the hippocampus. In addition to the functional runs, a high-resolution T1-weighted, 3D MP-RAGE sequence was acquired for anatomical localization ( mm 3 voxels, FOV 5 256, matrix , 150 axial slices). Four dummy volumes were collected at the beginning of each scan to allow for MR signal stabilization. fmri Data Analysis Image analysis was performed using the Analysis of Functional Neuroimages (AFNI) software (Cox, 1996). Images were first coregistered to correct for within- and across-scan head motion. Acquisitions in which a significant motion event occurred (more than 38 of rotation or 2 mm of translation in any direction relative to prior acquisition), plus and minus one TR were excluded from the analyses. Structural anatomical data were registered to standard stereotaxic space (Talairach and Tournoux, 1988), and the same parameters were subsequently applied to the functional data. A general linear model (GLM) was constructed using behavioral vectors coding for hits, misses, false alarms, and correct rejections for each presentation. The GLM was constructed using a 3D deconvolution technique (Ward, 2001), based on multiple linear regression. We performed an analysis of variance (ANOVA) across task conditions, treating the reference trials as an implicit baseline (not entered into the model). The resultant fit coefficients represent activity versus baseline for a given time point and trial type in a voxel. The sum of the fit coefficients over the expected hemodynamic response (2 10 s after trial onset) was taken as the estimate of the model of the response to each trial type relative to the baseline. Another GLM was also constructed using presentation number instead of trial type, and the same tests were conducted as above. Fit coefficient maps were resampled to 2.5 mm 3 and smoothed using a full-width at half-maximum (FWHM) Gaussian filter of 4 mm to account for interindividual anatomical variance. Voxel-wise peak threshold was set at P with a minimum cluster size of 95 voxels (1,484 mm 3 ) for the whole brain analyses. The combined alpha was 0.05, corrected for multiple comparisons. The required cluster size was calculated using AFNI s AlphaSim with 2,000 iterations. AlphaSim is a program that provides a means for estimating the overall probability of false positives for a 3D functional image by Monte Carlo simulation of the process of image generation, modeling spatial correlations of voxels, intensity thresholding, masking and cluster identification (Forman et al., 1995; Xiong et al., 1995). Results were corrected for multiple comparisons throughout the entire brain-masked volume. For the regional analysis, we used a threshold of P at the voxel level and a 20 voxel (125 mm 3 ) spatial extent threshold, which yielded a combined a level of 0.045, corrected for multiple comparisons. We calculated the required threshold also based on AlphaSim s simulations with 2,000 iterations but using our MTL template to provide size and geometry information instead of the whole brain volume, effectively correcting for multiple comparisons within the MTL. We then conducted an ANOVA across conditions, and masked the resulting fit coefficient maps with anatomical segmentations of MTL regions (see the following section for segmentation methods), then extracted the per-subject per-trial-type mean of summed beta coefficients in the clusters resulting from the omnibus F-test. We then performed a trend analysis on the task conditions to investigate changes in activity with repetition relative to the baseline. Additional post hoc two-tailed one-sample and two-sample t-tests were conducted to test individual contrasts. Cross-Subject Alignment (ROI-LDDMM) For our region of interest (ROI) analysis in the MTL, further refinement of the alignment of MTL structures was performed using Region of Interest Large Deformation Diffeomorphic Metric Mapping (ROI-LDDMM; Miller, 2005). This was performed to enhance sensitivity to signal changes in the MTL, which can often be missed if only a whole brain alignment method is used (Stark and Okado, 2003). The hippocampus was first segmented by starting in the most lateral slice where it was visible in the sagittal plane and proceeding in the medial direction until it completely disappeared. The superior boundary was set by the amygdala rostrally and the choroid fissure and the lateral ventricle caudally. The white matter of the parahippocampal gyrus served as the inferior boundary (Duvernoy, 1998). In addition to the hippocampus, each participant s temporopolar, entorhinal, perirhinal, and parahippocampal cortices were defined bilaterally in the coronal plane according to the

4 948 YASSA AND STARK TABLE 1. Behavioral Data by Condition young healthy participants. The vector field transformations were then subsequently applied to the fit coefficient maps for optimum cross-subject alignment in the MTL. Condition % Correct Latency (ms) Mean Std. Dev. Mean Std. Dev. RESULTS New Old Repeat Repeat Repeat methods described by Insausti et al. (1998). The temporopolar cortex was first outlined as it appeared in anterior coronal slices using an inclusive outline of the gray matter until white matter was visible and the gyrus of Schwalbe and indentation of the inferotemporal sulcus appeared. Then, the outline was reduced to the gray matter that extended from the peak of the gyrus of Schwalbe (the most lateral aspect) and following the gyrus medially until the indentation of the inferotemporal sulcus. The perirhinal and entorhinal cortices were subsequently defined as the portions of the parahippocampal gyrus extending from the most medial aspect of the temporal lobe to include the full length of the collateral sulcus. The boundary separating those two regions depended on the depth of the collateral sulcus, which is subject to interindividual anatomical variation and is reviewed in detail in Insauti et al. (1998). The parahippocampal cortex was defined bilaterally as the portion of the parahippocampal gyrus caudal to the perirhinal cortex and rostral to the splenium of the corpus callosum. These anatomically defined ROIs were then used to calculate ROI-LDDMM vector field transformations using a modal template developed in our laboratory as the target. The template is based on coregistered gold-standard segmentations of MTL regions according to the aforementioned protocol in 20 Behavioral Data Percent correct as well as response latencies from correct trials are shown in Table 1 and Figure 2. We conducted two repeated measures analyses of variance (ANOVA). The first ANOVA compared 2nd, 3rd, and 4th presentations and revealed a significant effect of number of presentations on hit rates [F(2, 30) , P < 0.001]. 1st presentation data was not included in the ANOVA, because the nature of the responses was different (rejection vs. hit), which would induce a response bias. A post hoc trend analysis was then conducted to investigate differences among conditions. Significant linear [F(1, 15) , P < 0.001], as well as quadratic [F(1, 15) , P < 0.001] trends were found. The second ANOVA compared reaction times across 2nd, 3rd, and 4th hits and revealed a significant effect of number of presentations on mean reaction times [F(2, 30) , P < 0.001]. Trend analysis revealed a significant linear trend [F(1, 15) , P < 0.001] as well as a small but significant quadratic trend [F(1, 15) , P < 0.05]. Overall, this data suggests that performance was enhanced with repetition in this task. fmri Data Results from the whole brain analysis are shown in Figure 3 and Table 2. We found a negative correlation between BOLD signal and stimulus repetition in several cortical regions, which included the lingual gyrus, the cuneus, the precuneus, middle FIGURE 2. Behavioral data by condition. Reference trials are shown in black, while responses to the novel and repeated stimuli are shown in white. Number of correct responses increases with the number of presentations, while mean reaction time decreases with the number of presentations. Error bars are 6 1 s.e.m.

5 FIGURE 3. Whole brain results. Increases in activity with repetition are depicted in the red scale while decreases in activity with repetition are depicted in the blue scale. Numbered clusters correspond to the cluster labels in Table 2. FIGURE 4. Results of the ANOVA in the MTL. The plots show change in signal in six different medial temporal lobe clusters shown on the average structural template. Activity during the reference condition (baseline) is shown as a broken line. PHC is parahippocampal cortex, PRC is perirhinal cortex. Cluster and bar colors are used for discrete labeling and do not indicate levels of significance. Error bars are 61 s.e.m.

6 950 YASSA AND STARK TABLE 2. Whole Brain fmri Results Cluster Label a Cluster size (voxels) Mean Cluster b Region b Increases in BOLD activity with repetition Right Supramarginal Gyrus, Inferior Parietal Lobule Left Supramarginal Gyrus, Angular Gyrus, Inferior Parietal Lobule Right Superior Frontal Gyrus Bilateral Anterior Cingulate Cortex Decreases in BOLD activity with repetition Right Lingual Gyrus, Middle Occipital Gyrus, Precuneus, Cuneus Left Lingual Gyrus, Middle Occipital Gyrus, Precuneus, Cuneus Right Posterior Cingulate Right Cingulate Right Inferior Frontal Gyrus Right Middle Frontal Gyrus Left Insula Right Insula Left Putamen Left Middle Frontal Gyrus a Cluster labels correspond to Figure 3. b Labels based on the Talairach and Tournoux Atlas. occipital gyrus, cingulate, insula, putamen, and middle and inferior frontal gyri (P < 0.05, corrected). These regions showed a repetition-related decrease in their response to novel stimuli. We also found a smaller number of regions that demonstrated a positive correlation between BOLD signal and stimulus repetition. Those included the supramarginal gyrus, the inferior parietal lobule, superior frontal gyrus and anterior cingulate cortex (P < 0.05, corrected). These regions showed a repetitionrelated increase in their response to novel stimuli. We found six clusters in the medial temporal lobe that survived our threshold in the omnibus test (see Fig. 4 for the location and activation profiles of these clusters). Monotonic decreases were present bilaterally in clusters in the parahippocampal cortex (PHC) (Fig. 4a,b) and in the anterior perirhinal cortex (PRC) (Fig. 4c,d). In the PHC, we found a significant main effect of repetition on BOLD signal in the left hemisphere [F(3, 45) , P < 0.001; negative linear trend F(1, 15) , P < 0.005], and a similar effect in the right hemisphere [F(3, 45) , P < 0.001; negative linear trend F(1, 15) , P < 0.001]. A post hoc t-test comparing activity during the Hit3 condition to the familiar reference baseline found significant differences only in the right cluster (t , P < 0.05). An additional test comparing activity during the CR condition to the familiar reference baseline also found differences only in the right cluster (t , P < 0.05). In the anterior PRC, we found a significant main effect of repetition as well in the left hemisphere [F(3, 45) , P < 0.05, negative linear trend F(1, 15) , P < 0.005], and the same effect in the right hemisphere [(F(3, 45) , P < 0.005; negative linear trend (F(1, 15) , P < 0.005]. In contrast, we found monotonic increases bilaterally in clusters in the posterior PRC (Fig. 4e,f). We found a main effect of repetition in the left PRC [F(3, 45) , P < 0.01; positive linear trend F(1, 15) , P < 0.005], and the same effect in the right PRC [F(3, 45) , P < 0.05; positive linear trend F(1, 15) , P < 0.05]. In the PRC, there was an additional dissociation between the right and left hemispheres. This can be illustrated easily by comparing activity from multiple presentations of novel stimuli to the pre-experimentally familiar reference condition. Our GLM treated the activity for these reference items as the baseline. In the right perirhinal cortex (both anterior and posterior), reference items were on par with items that have become highly familiar by repetition during the course of the experiment (Hit 3). A one-sample t-test of the Hit 3 condition against the baseline reference items yielded no significant difference (anterior t , P > 0.05, posterior t , P > 0.05). However, in the left perirhinal cortex (both anterior and posterior), reference items were on par with correct rejections and were strikingly different from the repeated items (Ref vs. Hit 3, P < 0.01). A one-sample t-test of the CR condition against the baseline reference items yielded no significant difference (anterior t , P > 0.05, posterior t , P > 0.05). Thus, not only were increases and decreases observed in the MTL with repetition, but effects of general familiarity, novelty and recency were also observed. These effects are discussed in further detail in the following section. DISCUSSION Analysis of the whole-brain data showed robust changes in activity with repetition. Both increases and decreases in activity

7 RECOGNITION MEMORY SIGNALS IN THE MTL 951 relative to baseline were noted in several regions. None of these changes were particularly striking as many of these regions have been previously implicated in studies with a single repetition. We found similar effects in the MTL regions we investigated. Past studies have demonstrated that as a stimulus becomes more familiar, there are decreases (Squire, 1992; Buckner et al., 1995; Desimone, 1996; Buckner et al., 1998a,b; Wiggs and Martin, 1998; Maccotta and Buckner, 2004; Law et al., 2005; Grill-Spector et al., 2006; Schacter et al., 2007) as well as increases (Konishi et al., 2000; McDermott et al., 2000; Gonsalves et al., 2005; Law et al., 2005; Wagner et al. 2005; Daselaar et al., 2006) in activity across the MTL. While it is plausible that encoding-related processes may be associated with decreases in signal, and retrieval-related processes may be associated with increases, there is currently no definitive evidence that this is the case. For our purposes, we chose to focus not on the direction of change but instead on the types of information that could be carried by these signals. Our interpretations are guided by electrophysiological recording studies in nonhuman primates, which have previously documented several types of signals in the MTL that could support recognition. Recency, Novelty, and Familiarity Brown and Aggleton (2001) described three different types of responses in neurons in the perirhinal cortex. One type of neuron, dubbed a recency neuron, codes for recent repetition during the experiment regardless of general familiarity with the item. Thus, a recency neuron would change its firing rate to a repeated novel stimulus and would also change its firing rate in the same way to a repeated familiar stimulus. A second type, familiarity neurons, code for the level of familiarity regardless of recent repetition. Thus, a familiarity neuron would not change its firing rate to repetitions of either novel or familiar stimuli, but would generally shift its firing rate based on longterm familiarity. Finally, novelty neurons respond in a more complex way, coding for the very first encounter with a stimulus and additionally coding for long-term familiarity with the stimulus. They fire strongly in response to the first presentation of a stimulus ever and weakly in response to repetitions of stimuli only recently encountered. To highly familiar stimuli, they fire very briefly, but very rapidly, regardless of whether the stimulus was encountered recently (Brown and Xiang, 1998; Xiang and Brown, 1998). Thus, these neurons seem to discriminate whether the stimulus is truly novel and, if not, whether the exposure to the stimulus came only recently or is based on long-term familiarity with the stimulus. In our study, we found MTL signals that reflect the same types of information that Brown and Aggleton suggest. Although they have predominantly reported decreases in firing rates with repetition and familiarity, it is important to note that increases in activity may convey the same information. BOLD fmri is not a direct reflection of pyramidal cell spiking, but rather appears to be more closely related to perisynaptic activity (Logothetis et al., 2001). Additionally, inhibitory input to a region increases rather than decreases the BOLD response (Caesar et al., 2003). Thus, in order to relate fmri to physiological data, investigating the pattern of change in BOLD signal can be more useful than the considering the direction of change. We can attempt to determine whether the information conveyed in the neural codes described by Brown and Aggleton is reflected in the fmri signal. It would be foolish to assume that we would necessarily find pure reflections of recency, familiarity and novelty in our fmri signals, since there are thousands of neurons sampled in each voxel, and thus it is likely that regional activity is driven by a mixture of signals. However, we can assess if our fmri signals are in any way influenced by recency, familiarity, and novelty or a mixture of these factors. That being said, our results do show that recency is clearly reflected in BOLD fmri signals in the MTL. In the left perirhinal cortex (anterior and posterior ROIs), quite clear recency codes were observed. Here, BOLD activity during viewing a novel stimulus for the first time was similar to activity during viewing a familiar reference stimulus for the first time in the experiment (Fig. 4c,e). This suggests that these regions do not code for the relative familiarity of the stimulus. However, during repetition of the novel stimulus, these regions change activity (decrease in the anterior cluster, and increase in the posterior cluster). The combination of repetition sensitivity and lack of a familiarity effect are strong evidence for activity in these two regions being modulated by recency information. We also find evidence that suggests that novelty may be coded in the MTL. In the parahippocampal cortex (left and right ROIs), as well as the right PRC (anterior and posterior ROIs), fmri signals that resemble the novelty signals Brown and Aggleton reported were present. Here, BOLD activity during viewing a novel stimulus for the first time was different from activity during viewing a familiar reference stimulus for the first time in the experiment. Additionally, activity during viewing the novel stimulus for the first time was also different from activity during viewing a repetition of the same stimulus (Fig. 4a,b,d,f). This suggests that this region can discriminate whether a stimulus is novel or not, and if not, whether the lack of novelty is driven by recent exposure or long-term familiarity. However, it is somewhat more difficult to fully ascribe these fmri signals to novelty, given that we do not have a clear handle on precisely how rapid, brief firing (the suggested manner in which novelty neurons code for long-term familiarity) would modulate the BOLD response, relative to other responses. The most plausible hypothesis is that brief firing results in a BOLD signal that is smaller in amplitude than that due to continuous strong firing (as is the case for novel stimuli), and yet still larger than due to weak firing (as is the case for repeated novel stimuli). This fits the BOLD signals that we find in these regions remarkably well. We can also demonstrate that they do not contain pure familiarity or pure recency codes, since they are modulated by both repetition and long-term familiarity. Their response profiles are very similar to those of novelty neurons. We also find global shifts in activity relative to the pre-experimentally familiar baseline activity suggesting that several of

8 952 YASSA AND STARK our ROIs also signal familiarity. The right PHC and right anterior PRC treat the familiar reference trials differently from other trial types. In one of these regions, the right PHC (Fig. 4b), there is a significant difference between CR and reference items as well as a significant difference between Hit 3 and reference items, suggesting that this region can discriminate between familiar and novel stimuli, and can further discriminate between familiarity due to pre-experimental exposure and familiarity due to repetition. A similar effect is seen in the right anterior PRC (Fig. 4d), however it does not quite reach significance. All of our ROIs were modulated to some extent by repetition, and thus a pure familiarity signal was not observed. Additionally, some regions clearly did not signal familiarity; in the left perirhinal regions (Fig. 4c,e), activity on the first presentation of the novel stimuli was identical to activity on the familiar reference stimuli. It is somewhat difficult to interpret the parametric modulation we get with number of repetitions in light of Brown and Aggleton s results, because their electrophysiological studies only used a single repetition of the novel and familiar items. It is possible that some of the patterns we observe are only detectable when stimuli are repeated more than once. It is also possible that our ROIs contain a mixture of signals and not just one type of signal. Several recent studies in humans have looked for the presence of novelty and/or familiarity codes in the MTL. Although the results of these studies are largely consistent with our findings, all have used different definitions of novelty and familiarity. For example, a recent fmri study by Daselaar et al. (2006) found a triple dissociation in the MTL with a putative recollection signal in the posterior hippocampus, a putative familiarity signal in the posterior parahippocampal cortex, and a putative novelty signal in the anterior hippocampus and rhinal cortex. Daselaar et al. (2006) defined familiarity as a linear increase in signal with increased recognition confidence and novelty as a linear decrease in signal with increased recognition confidence. Although the authors used a different operational definition of familiarity and novelty in their experiment than the one used in our study and the electrophysiological recording studies previously discussed, they find similar changes in BOLD activity modulated by memory strength in different regions of the MTL. Another study by Gonsalves et al. (2005) used fmri and anatomically constrained MEG to study parametric variations in memory strength in the MTL. Consistent with our results, the authors found decreases in activity that correlated with increasing memory strength in the PHC and PRC. However, these decreases could be attributed to either increasing familiarity or decreasing novelty in their study. Finally, a recent study by Rutishauser et al. (2006) using single cell recordings in epilepsy patients found two distinct firing patterns for neurons in the hippocampus-amygdala complex. One class of neurons showed a selective increase in firing in response to old stimuli (hence named familiarity detector) and another showing a selective increase in firing in response to new stimuli (hence named novelty detector). In this study, a very different definition of novelty and familiarity was employed; the authors defined novelty or familiarity neurons in terms of the type of stimulus that caused an increase in firing rates. Unlike previous studies, which found mostly decreases in activity with repetition, Rutishauser et al. (2006) find only increases and not decreases in firing rates. Overall, however, these studies point to the same conclusion, which is that there are several aspects of the stimulus coded in the medial temporal lobe that support later recognition. Although these studies are all clearly divergent in methodologies and in their key operational definitions of familiarity and novelty, our findings are largely consistent with their results. However, our focus was on attempting to use the same definitions of these terms as the ones used by the electrophysiological studies that isolated these codes in nonhuman primates. To our knowledge, this is the first study to make a direct link between human fmri data in the perirhinal cortex to the electrophysiological recording data reported by Brown and Aggleton and others. A notable absence here is the lack of change in activation in the hippocampus. There are many reasons why this could be the case, some theoretical and some methodological. It is possible that there were changes in activity in the hippocampus that did not survive our threshold, or perhaps were not modulated by repetition in the task. It is also possible that our task did not place high demands on the putative function(s) of the hippocampus. If we think of the hippocampus as a pattern separation device that is engaged heavily when similar overlapping representations must be pulled apart to avoid interference (Rolls and Treves, 1998; O Reilly and Rudy, 2001; Norman and O Reilly, 2003; Kirwan and Stark, 2007), then the likelihood of finding changes in this region in this specific task is slim, considering that all of the stimuli were unique and nonoverlapping. It is also possible that the hippocampus is more involved in recollection and not so much in simple familiarity-based visual object recognition (Brown and Aggleton, 2001), a theory that is supported by several electrophysiological studies that found little or no activity in the monkey hippocampus during simple recognition tasks (Brown et al., 1987; Rolls et al., 1993; Xiang and Brown, 1998). It has also been suggested that in comparison to the adjacent structures of the parahippocampal gyrus, the hippocampus is not particularly required for the tuning of encoding by enhanced or diminished responsiveness to repeated exposure; the hippocampus instead codes the relational organization of distinct cortical representations (Eichenbaum et al., 1996). Recent fmri evidence, consistent with electrophysiological recordings, suggests that the hippocampus is not particularly sensitive to stimulus repetition, while the perirhinal and parahippocampal cortices show linear decreases with repeated exposure (Preston and Gabrieli, 2008). Thus, the lack of change with repetition is not altogether surprising given the nature of the experiment, and our current understanding of hippocampal function. However, one must be careful not to interpret this null finding too strongly. For example, a pair of studies reported hippocampal activity during simple recognition memory testing and highlighted one reason why null results are often obtained (Stark and Squire, 2000, 2001). In these studies, the initial presentation of a recognition test item failed to elicit hippocampal activity while a second presentation of the same test item dem-

9 RECOGNITION MEMORY SIGNALS IN THE MTL 953 onstrated activity. Activity for targets (or hits) remained relatively constant in the hippocampus, while activity for foils (or correct rejections) decreased on the second presentation. The authors interpreted the change in activity associated with foils as resulting from an artificially inflated activity for these stimuli during the first test, owing to their novelty. In the present study, we have data corresponding to the initial presentation of the foils and to the multiple presentations of the targets, but we do not have data corresponding to a second presentation of the foil. Thus, in the contrasts corresponding to the ones we have available here, Stark and Squire (2000, 2001) reported no hippocampal activity as well. Finally, we should note that in contrast to the vast majority of neuroimaging studies of recognition memory, we used a continuous recognition memory task that may differ in fundamental ways from more traditional designs that separate encoding and retrieval phases. We suggest that while such designs separate the two phases, they do not isolate process-pure encoding and retrieval signals. As noted above, encoding and retrieval are best thought of as automatic processes that are continuously engaged as we experience events; this is more closely paralleled by a continuous recognition memory task. Encoding happens during putative retrieval tasks and retrieval happens during putative encoding tasks. The approach participants take and the amount of attention to encoding and retrieval components clearly vary across the two task designs, as attention must be paid to encoding each trial in the continuous recognition framework. Given the number of uncertainties here we feel it is unwise to interpret the null results in the hippocampus too strongly. We instead choose to focus on the positive results in the parahippocampal gyrus. In conclusion, we found that recognition is a dynamic process that involves multiple signals in the medial temporal lobe. Our results demonstrate that novelty, recency, and familiarity information are being coded in the human MTL during continuous recognition, and provide a critical link to electrophysiological recording studies in nonhuman primates. The results presented herein contribute to our understanding of the division of labor in the MTL memory system, and provide experimental support for the putative role of structures in the parahippocampal gyrus in supporting recognition memory. Acknowledgments The authors would like to thank the staff of the F.M. Kirby Center for Functional Brain Imaging for their assistance in data collection and acknowledge the use of the resources of the Center for Imaging Science at the Johns Hopkins University in the LDDMM alignment analysis. REFERENCES Bachevalier J, Nemanic S, Alvarado MC The medial temporal lobe structures and object recognition memory in nonhuman primates. In: Squire LR, Schacter DL, editors. Neuropsychology of Memory, 3rd ed. New York: Guilford Press. pp Brown MW Neuronal responses and recognition memory. Semin Neurosci 8: Brown MW, Aggleton JP Recognition memory: What are the roles of the perirhinal cortex and hippocampus? Nat Rev Neurosci 2: Brown MW, Wilson FAW, Riches IP Neuronal evidence that inferomedial temporal cortex is more important than hippocampus in certain processes underlying recognition memory. Brain Res 409: Brown MW, Xiang JZ Recognition memory: Neuronal substrates of the judgement of prior occurrence. Prog Neurobiol 55: Buckner RL, Koutstaal W, Schacter DL, Dale AM, Rotte M, Rosen BR. 1998a. Functional-anatomic study of episodic retrieval. Ii. Selective averaging of event-related fmri trials to test the retrieval success hypothesis. Neuroimage 7: Buckner RL, Koutstaal W, Schacter DL, Wagner AD, Rosen BR. 1998b. Functional-anatomic study of episodic retrieval using fmri I: Retrieval effort versus retrieval success. Neuroimage 7: Buckner RL, Petersen SE, Ojemann JG, Miezin FM, Squire LR, Raichle ME Functional anatomical studies of explicit and implicit memory tasks. J Neurosci 15: Buffalo EA, Ramus SJ, Clark RE, Teng E, Squire LR, Zola SM Dissociation between the effects of damage to perirhinal cortex and area TE. Learn Mem 6: Caesar K, Gold L, Lauritzen M Context sensitivity of activitydependent increases in cerebral blood flow. Proc Natl Acad Sci USA 100: Cho K An fmri study of differential perirhinal and prefrontal activation during familiarity/recency discrimination. Eur J Neurosci 12:13. Cox RW AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages. Comp and Biomed Res 29: Dale AM Optimal experimental design of event-related fmri. Hum Brain Mapp 8: Daselaar SM, Fleck MS, Cabeza R Triple dissociation in the medial temporal lobes: Recollection, familiarity, and novelty. J Neurophysiol 96: Desimone R Neural mechanisms for visual memory and their role in attention. Proc Natl Acad Sci USA 93: Duvernoy H The Human. 2nd ed. New York: Springer. Eacott MJ, Gaffan D, Murray EA Preserved recognition memory for small sets, and impairment stimulus identification for large sets, following rhinal cortex ablations in monkeys. Eur J Neurosci 6: Eichenbaum H, Schoenbaum G, Young B, Bunsey M Functional organization of the hippocampal memory system. Proc Natl Acad Sci USA 93: Fahy FL, Riches IP, Brown MW Neuronal activity related to visual recognition memory: Long-term memory and the encoding or recency and familiarity information in the primate anterior and medial inferior temporal and rhinal cortex. Exp Brain Res 96: Forman S, Cohen J, Fitzgerald M, Eddy W, Mintun D, Noll D Improved assessment of significant activation in functional magnetic resonance imaging (fmri): Use of a cluster-size threshold. Magn Reson Med 33: Gonsalves B, Kahn I, Curran T, Norman K, Wagner A Memory strength and repetition suppression: Multimodal imaging of medial temporal cortical contributions to recognition. Neuron 47: Grill-Spector K, Henson R, Martin A Repetition and the brain: Neural models of stimulus-specific effects. Trends Cogn Sci 10: Henson R Neuroimaging studies of priming. Prog Neurobiol 70:53 81.

10 954 YASSA AND STARK Henson R A mini-review of fmri studies of human medial temporal lobe activity associated with recognition memory. Q J Exp Psychol 58: Henson R, Cansino S, Herron JE, Robb W, Rugg M A familiarity signal in the human anterior medial temporal cortex? 13: Insausti R, Juottonen K, Soininen H, Insausti A, Partanen K, Vainio P, Laakso P, Pitkanen A MR volumetric analysis of the human entorhinal, perirhinal, and temporopolar cortices. Am J Neuroradiol 19: Kirwan C, Stark C Pattern separation processes in the human medial temporal lobe: An fmri investigation. Learn Mem 14: Konishi S, Wheeler M, Donaldson D, Buckner R Neural correlates of episodic retrieval success. Neuroimage 12: Law J, Flanery M, Wirth S, Yanike M, Smith A, Frank L, Suzuki W, Brown E, Stark C fmri activity during the gradual acquisition and expression of paired associate memory. J Neurosci 25: Li L, Miller EK, Desimone R Activity of neurons in anterior inferior temporal cortex during a short-term memory task. J Neurophysiol 69: Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A Neurophysiological investigation of the basis of the fmri signal. Nature 412: Maccotta L, Buckner RL Evidence for neural effects of repetition that directly correlate with behavioral priming. J Cog Neurosci 16: Martin A Automatic activation of the medial temporal lobe during encoding: Lateralized influences of meaning and novelty. 9: McDermott K, Jones T, Petersen S, Lageman K, Roediger H Retrieval success is accompanied by enhanced activation in anterior prefrontal cortex during recognition memory: An event-related fmri study. J Cogn Neurosci 12: Melcher D Persistence of visual memory for scenes. Nature 412:401. Meunier M, Bechavalier J, Mishkin M, Murray EA Effects on visual recognition of combined and separate ablations of the entorhinal and perirhinal cortex in rhesus monkeys. J Neurosci 13: Miller EK, Li L, Desimone R Activity of neurons in anterior inferior temporal cortex during a short-term memory task. J Neurosci 13: Miller EK, Desimone R Parallel neuronal mechanisms for short-term memory. Science 263: Miller MI, Beg MF, Ceritoglu C, Stark C Increasing the power of functional maps of the medial temporal lobes by using large deformation diffeomorphic metric mapping. Proc Natl Acad Sci USA 102: Milner B, Squire LR, Kandel ER Cognitive neuroscience and the study of memory. Neuron 20: Murray EA Medial temporal lobe structures contributing to recognition memory: The amuygdaloid cortex versus the rhinal cortex. In Aggleton JP, editor. The Amygdala: Neurobiolgical Aspects of Emotion, Memory and Mental Dysfunction. New York: Wiley. pp Norman KA, O Reilly RC Modeling hippocampal and neocortical contributions to recognition memory: A complementary-learning-systems approach. Psychol Rev 110: O Reilly RC, Rudy JW Conjunctive representations in learning and memory: Principles of cortical and hippocampal function. Psych Rev 108: Preston AR, Gabrieli JD Dissociations between explicit memory and configural memory in the human medial temporal lobe. Cereb Cortex (in press). Rolls ET, Cahusac PMB, Feigenbaum JD, Miyashita Y Responses of single neurons in the hippocampus of the macaque related to recognition memory. Exp Brain Res 93: Rolls ET, Treves A Neural Networks and Brain Function. Oxford: Oxford University Press. Rutishauser U, Mamelak A, Shuman E Single-trial learning of novel stimuli by individual neurons of the human hippocampusamygdala complex. Neuron 49: Ryan JD, Cohen NJ The nature of change detection and online representations of scenes. J Exp Psychol Hum Percept Perform 30: Schacter DL, Buckner RL Priming and the brain. Neuron 20: Schacter DL, Wagner AD Medial temporal lobe activations in fmri and PET studies of episodic encoding and retrieval. 9:7 24. Schacter DL, Wig GS, Stevens WD Reductions in cortical activity during priming. Curr Opin Neurobiol 17: Squire LR Memory and the hippocampus: A synthesis from findings with rats monkeys and humans. Psychol Rev 99: Squire LR, Stark CE, Clark RE The medial temporal lobe. Annu Rev Neurosci 27: Stark CE, Okado Y Making memories without trying: Medial temporal lobe activity associated with incidental memory formation during recognition. J Neurosci 23: Stark CE, Squire LR Functional magnetic resonance imaging (fmri) activity in the hippocampal region during recognition memory. J Neurosci 20: Stark CE, Squire LR Simple and associative recognition memory in the hippocampal region. Learn Mem 8: Stern CE, Corkin S, González RG, Guimaraes AR, Baker JR, Jennings PJ, Carr CA, Sugiura RM, Vedantham V, Rosen BR The hippocampal formation participates in novel picture encoding: Evidence from functional magnetic resonance imaging. Proc Natl Acad Sci USA 93: Suzuki WA, Zola-Morgan S, Squire LR, Amaral DG Lesions of the perirhinal and parahippocampal cortices in the monkey produce long-lasting memory impairment in the visual and tactual modalities. J Neurosci 13: Talairach J, Tournoux P A Co-Planar Steres Jotactic Atlas of the Human Brain. Stuttgard, Germany: Thieme. Wagner A, Shannon B, Kahn I, Buckner R Parietal lobe contributions to episodic memory retrieval. Trends in Cog Sci 9: Ward BD Deconvolution analysis of fmri time series data Wig GS, Grafton ST, Demos KE, Kelley WM Reductions in neural activity underlie behavioral components of repetition priming. Nat Neurosci 8: Wiggs CL, Martin A Properties and mechanisms of perceptual priming. Curr Opin Neurobiol 8: Xiang JZ, Brown MW Differential neuronal encoding of novelty, familiarity, and recency in regions of the anterior temporal lobe. Neuropharmacol 37: Xiong J, Gao J, Lancaster L, Fox P Clustered pixels analysis for funcgtional MRi activation studies of the human brain. Hum Brain Map 3: Zola-Morgan S, Squire LR, Clower RP, Rempel NL Damage to the perirhinal cortex exacerbates memory impairment following lesions to the hippocampal formation. J Neurosci 13: