Brain Imaging Techniques

Transcription

1 2 Brain Imaging Techniques A D I N A L. R O S K I E S 1. I N T R O D U C T I O N Several recently developed techniques allow researchers to investigate the neural basis of cognitive function in the living human brain, where cognitive function is construed broadly to include thinking, feeling, and acting. Currently, the most commonly used technique to explore the neural basis of cognition is magnetic resonance imaging (MRI). MRI can provide both structural and functional information about the brain. This chapter describes the potential of MRI to illuminate brain function, and discusses issues that judges may face in evaluating studies and expert testimony. Structural brain imaging techniques (which show brain anatomy, not function) have been used for some time to document abnormalities such as cancer, stroke, degeneration, and traumatic brain injury in legal cases. However, novel methods for probing brain function rather than anatomy raise new possibilities for the introduction of brain imaging data in the law. For example, functional neuroimaging may play a role in deciding what expert testimony to allow, in determinations of competency, in adjudication of criminal responsibility, in lie-detection, and in determinations of penalty. The goal of this chapter is to familiarize the reader with the main techniques for neuroimaging, their capabilities and limitations, and the sorts of questions to consider when confronted with brain imaging evidence and testimony in the courtroom. To fully understand and evaluate brain imaging requires considering aspects of experimental design and analysis. Common misconceptions are also discussed. Questions to bear in mind are outlined in the final section. 02_Morse_Ch02.indd 39 8/29/2012 2:26:27 AM

2 40 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE Th is chapter focuses on functional magnetic resonance imaging (fmri). fmri yields whole-brain images showing what parts of the brain are active when the subject performs specific tasks. Although fmri is only one of a number of techniques developed to visualize brain function, it remains the most prevalent, promising, and persuasive. However, a description of other important neuroscientific techniques is included in Chapter 3. Au: please verify cross reference. 2. STRUCTURAL AND FUNCTIONAL INFORMATION Most classical neurophysiological techniques, such as those discussed briefly in Chapter 1, reveal information about nervous system function at a cellular or subcellular level. All could be considered neuroimaging techniques, in the sense that the information they provide can be represented as visual images. However, in this primer we use the term neuroimaging more strictly to refer to methods that yield larger-scale, systems-level information about nervous system function. As any complex behavior involves large networks of brain regions acting in concert, this sort of information is crucial to understanding the link between brains and legally relevant behaviors. Figure 2.1 depicts the spatial and temporal scales to which various neuroimaging techniques are sensitive in relationship to other standard neurophysiological methods for investigating brain function. Each technique has distinct advantages and disadvantages, and each can provide information about different aspects of the biology of the brain. Consequently, each of the techniques represented above fills an important niche in terms of its ability to provide information about the spatial and temporal characteristics of neural structure and activity. No single technique can provide all the information we want, and often interpretation of neuroimaging data will depend upon findings from other neuroscientific techniques. Indeed, one indication of the accuracy or reliability of an interpretation is its consistency with findings from other techniques. 1 Techniques for imaging brain structure are often used for assessing the extent and location of injury or presence of structural abnormalities. Structural imaging is also essential for providing baseline information about normal brain anatomy for interpreting functional studies. Structural imaging techniques include computed axial tomography (CAT or CT), MRI, and recent advanced MRI techniques, such as diffusion weighted techniques. Some uses of positron emission tomography (PET) and single photon emission tomography (SPECT) also provide anatomical information, but of only modest precision (see Chapter 3 for discussion of non-fmri imaging methods). Different functional neuroimaging techniques measure different aspects of brain function. When neurons fire, they create electrical currents that can Au: please verify cross reference. 02_Morse_Ch02.indd 40 8/29/2012 2:26:27 AM

3 Brain Imaging Techniques 41 AQ: Please check figures numbering in the chapter. AQ: Please provide complete details of Churchland and Sejnowski, 1988 in the reference list. Brain Lobe Map Nucleus Layer Neuron Dendrite Synapse Size (mm) EEG and MEG Optical fmri imaging g imaging Field potentials Single unit Patch clamp Light microscopy Cellular l Electron microscopy imaging Time (s) Millisecond Second Minute PET imaging 2-DG imaging Hour Day Brain lesions Month Figure 2.1 Graph of spatial and temporal resolution of neuroscientific techniques. Each box represents the spatial scales and temporal discrimination possible with its associated neuroscientific technique. The temporal and spatial scales are graphed logarithmically, ranging from microns to the scale of the entire brain, and from milliseconds to years. Each technique only covers a limited part of the information range. To fully understand brain function, we need to integrate information from a variety of neuroscientific techniques, none of which alone can tell us more than a part of the story. The techniques that can be used on living human beings are highlighted in yellow, and cover only a limited part of the relevant territory. (Adapted from Churchland and Sejnowski, 1988)(also in Gazz p.160) be detected by some imaging techniques. Electroencephalography (EEG) and magnetoencephalography (MEG), as well as some types of optical imaging, pick up changing electrical potentials in brain tissue due to neural activity, or magnetic changes caused by these electrical currents. Other techniques measure brain function more indirectly. When neurons fire, they use energy, and some imaging techniques measure changes in the brain that result from changes in energy usage. Just as blood flow increases to supply energy to muscles when they are used, blood flow increases to supply necessary glucose and oxygen to localized regions of the brain when they are involved in processing (Raichle, 1983). These changes in blood flow are referred to as hemodynamic changes, and they are correlated with neural activity (Raichle, 1987). PET and fmri, as well as some optical imaging techniques such as near infrared spectroscopy (NIRS), directly or indirectly measure changes in blood flow that are 02_Morse_Ch02.indd 41 8/29/2012 2:26:27 AM

4 42 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE associated with neural activity. Although PET retains the advantage for some of these physiological measurements such as glucose or oxygen consumption by the brain, this chapter will focus on fmri, which has largely supplanted PET as a technique for acquiring whole-brain data of regional neural activity. 3. MAGNETIC RESONANCE IMAGING: AN INTRODUCTION A. Anatomical (or Structural) MRI The development of MRI has made it possible to visualize noninvasively, without radiation exposure, and in three dimensions the internal structure of objects composed of materials with differing chemical structures. MRI can resolve information impossible to resolve with X-ray based techniques such as CT. The basic magnetic resonance (MR) signal is generated when an object (for our purposes, a person s head) is placed in changing magnetic fields and the molecules constituting it are excited by radiofrequency pulses; the resulting behavior of the molecules creates a signal that can be measured (Buxton, 2002). Molecular differences in material being imaged produce differences in the signal (e.g., bone vs. soft tissue). Specifically, the MR signal deteriorates at different rates in different material, and this difference in signal change allows an investigator to distinguish among different tissues. Using rapidly changing magnetic field gradients and sequences of radiofrequency pulses (called pulse sequences ), MR imaging can map material in sets of two-dimensional slices or even three dimensions acquired simultaneously, so that differences in the three-dimensional molecular composition of an object can be represented. Typically, this information is represented visually as images. Until recently, anatomical magnetic resonance was primarily used in clinical care to visualize brain structure and pathologies such as tumors and brain injury. Anatomical MR scans are often introduced in the courtroom as evidence for claims of brain damage. Although acquired in three dimensions, data from anatomical MRI are typically depicted by high-resolution grayscale (black-through-white) twodimensional images of brain slices. These slices are cuts, usually in standard orientations, through the three-dimensional brain (Fig. 2.2). These images show gray matter and white matter, areas of fluid, tumors, etc. The resolution, contrast, and quality of anatomical MR scans depend upon many factors. Important elements include magnet strength, cooperation of the subject in holding still, the size of whatever is being scanned, and the type of pulse sequence used to identify particular tissues. Under good conditions, typical anatomical MRI scans can yield 1mm 3 resolution, resolving differences in structure about the size of a grain of (cooked) rice, but higher resolution scans are possible. A general principle is that higher resolution images require longer 02_Morse_Ch02.indd 42 8/29/2012 2:26:28 AM

5 Brain Imaging Techniques 43 A C B C B Figure 2.2 Example of anatomical MRI scan (maybe one with a lesion?) AQ: Please determine maybe one with a lesion? scan times. This often sets limits on what can be done with living people for practical reasons: for example, even small movements can ruin a scan. For the remainder of the chapter, we will assume that the functional MRI scans we discuss are from people whose structural MRI scans look normal. Thus any differences of function of legal relevance are not due to macroscopic structural factors such as brain deformation or lesion. B. F u n c t i o n a l M R I Only in the 1990s did it become apparent that, with appropriate pulse sequences, the magnetic resonance signal can reflect hemodynamic (blood flow) changes caused by neural activity (Kwong et al., 1992). Since the first demonstrations of MR s potential as a technique for measuring brain activity, several types of pulse sequences have been developed that are sensitive to parameters related to brain function, such as perfusion, blood volume, and blood oxygenation. The most frequently used BOLD (blood oxygenation level dependent) technique capitalizes on the fact that the relative proportions of oxygenated and deoxygenated hemoglobin in the blood change as a result 02_Morse_Ch02.indd 43 8/29/2012 2:26:28 AM

6 44 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE of increased neural activity (Ogawa et al., 1993). During neural activation, such as increased firing of neurons in the motor cortex when a participant taps his fingers, there is an associated increase in cerebral blood flow that can be detected indirectly with fmri as a relative increase in oxygenated hemoglobin (or oxyhemoglobin) compared to deoxyhemoglobin. Oxyhemoglobin and deoxyhemoglobin have different magnetic properties; the net decrease in the relative concentration of deoxyhemoglobin results in an increase in the BOLD signal in approximately the same area that shows greater activity of local neuron activity and their inputs. In our finger-tapping example, real blood flow can increase up to 40 percent in motor areas compared to a measurement obtained at rest. The corresponding change in the fmri BOLD signal is smaller, on the order of 3 to 8 percent It is important to note that these smaller differences cannot be detected by the naked eye, as a radiologist might use to read an X-ray or CAT scan. Nevertheless, digital subtraction of activation scans comparing levels of BOLD signal during two tasks can reveal regions in which the physiological demands of neural tissue are altered, and thus areas involved in the differential processing of these tasks (Kwong et al., 1992; Menon et al., 1992). Data from fmri are usually displayed as multiple anatomical images of the brain, shown in the background in gray scale, with areas representing increased (and/or decreased) neural activity shown as colorized pixels superimposed on the anatomical scans (Fig. 2.3). It is important to note that each pixel in an image slice actually represents a three-dimensional region, called a voxel (volumetric pixel), in space. Data from fmri are also sometimes represented as colored areas painted on the surface of a three-dimensionally rendered brain. A number of issues attend the representation of results from fmri; these will be discussed later in the chapter. Figure 2.3 Example of functional MRI scan slices and three-dimensional brain 02_Morse_Ch02.indd 44 8/29/2012 2:26:28 AM

7 Brain Imaging Techniques 45 C. Basic Principles of MRI and fmri The following section is technical, and may not be essential to understand later sections. We include it for completeness. The magnetic resonance (formerly nuclear magnetic resonance, or NMR) signal derives from the intrinsic characteristics of atomic nuclei with magnetic properties in an external magnetic field. Molecules with magnetic properties act like tiny bar magnets, or dipoles, and their behavior can be affected by the application of magnetic fields. Like a top, they can be made to spin, or precess, around an axis determined by the direction of the magnetic field. Precession or oscillation of magnetic dipoles in a magnetic field constitutes the basic MR signal. The MR signal in the brain arises primarily from the hydrogen nuclei of water molecules, which act as tiny dipoles. The images from MR depend on the rate of decay of the MR signal. The decay time of the MR signal depends upon the local environment of the molecules. For example, the signal decays more slowly in gray matter than in white matter, and more slowly still in many lesions such as tumors. The contrast achieved between one tissue and another in an MR image is primarily due to ways of harnessing these differences in the decay time of the MR signal. The MR scanner is a machine dominated by a large magnet that produces a strong magnetic field (Fig. 2.4). Both the MR signal and the means of image Figure 2.4 Th e magnetic resonance scanner 02_Morse_Ch02.indd 45 8/29/2012 2:26:29 AM

8 46 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE formation require a complex series of applications of magnetic field gradients and radiofrequency pulses, which effectively tags the signal from each location with a unique signature. The resulting complex time-varying signal measured by the receiver coils (a temporal frequency signal) directly reflects the spatial frequency content of the object being imaged, and with the application of some mathematics (i.e., Fourier transforms), an image is produced. Variation in the application of magnetic gradients and radiofrequency pulses can change the parameters an MR scan is sensitive to, and different pulse sequences distinguish various anatomical and functional scanning techniques, including the ones mentioned above. Details of pulse sequence design critically affect the MR signal and are essential to any thorough account of MR, but they are too complex to be treated here. 2 The large number of parameters involved in any MR imaging experiment and the wide spectrum of values that can be chosen for each make the technique wonderfully versatile, but also make the results challenging to interpret and difficult to replicate. In interpreting any scan, the first thing one should ascertain is to what, given the scan parameters, this scan is sensitive to: e.g., the difference between white and gray matter, blood oxygenation level, blood flow, diffusion, etc. Many other factors of design and analysis also have to be considered, and will be treated further below. (Buxton, 2002; Cabeza & Kingston, (2006). For more detailed and comprehensive discussions of MR physics and imaging methods, see Huettel et al., (2004).) D. F u n c t i o n a l I m a g i n g A systems perspective on the brain reveals that changes in neural activity during the performance of cognitive tasks are found in networks of cortical regions rather than single locations, and these regions may be widely distributed throughout the brain (see Chapter 1). By determining which sets of regions are active during which tasks, we are beginning to understand how the brain breaks complex cognitive tasks into simpler information-processing components. Even the most basic cognitive tasks have multiple elements and involve activity in many different parts of the brain. The fact that neuroimaging experiments have yielded reliable, largely replicable information about some of the critical nodes of neural activity in these networks during tasks ranging from vision, to language, to memory, to skill learning, is a testament to the fact that the brain is a highly organized structure, with functionally specialized units. The degree to which functional components are localized, and the scale at which localization occurs, both have significant implications 02_Morse_Ch02.indd 46 8/29/2012 2:26:29 AM

9 Brain Imaging Techniques 47 for the capacities and limitations of neuroimaging. Our understanding of the degree and significance of localization is still evolving. Functional imaging studies, often referred to as activation studies, are neuroimaging studies designed to attribute functional roles to specific brain regions in which electrical or corresponding hemodynamic changes have been identified. By delineating the neural correlates of cognitive processes, we improve our grasp of the components constitutive of cognitive processes, as well as enhance our understanding of brain organization. In some cases activation studies delineate the functional anatomy of a cognitive task (i.e., they identify the regions involved in performing that task). In the example of finger tapping, we can establish where the motor areas of the brain are. In others they attempt to determine in more detail the type of computation a region performs. In the case of finger tapping, we might identify motor areas that vary as a function of how fast or forcefully finger movements are made. Studies such as these often form the background knowledge that may be invoked in legal cases. Because functional MRI can be done on almost all clinical MR scanners without special equipment, it is widely available and easy to generate data: practically anyone can, and any study, no matter how ill-conceived, holds promise for generating images of brain function. This poses a problem both to the field and to the law, because it is not easy to generate good data, or data that are both interpretable and probative of specific questions. Of great concern to the law and to science alike is distinguishing between well-executed and poorly executed experiments, and valid and invalid analysis and interpretation of the results. Most fmri experiments to date have focused on functional anatomy of groups of people. This raises a major concern for law about the degree to which functional data about individuals can be interpreted, and whether and how scientific generalizations about brain function can be rendered applicable to individual cases. Au: please verify cross reference E. Imaging of the Resting State A novel approach with fmri images the brains of subjects at rest (not engaging in a defined cognitive task) (Raichle et al., 2001). This imaging of the resting state or default mode of brain function shows promise for revealing potentially important features of global structural/functional neural organization. Slow oscillations in blood flow occur throughout the brain at rest, and cross-correlation of the resting (sometimes called baseline ) dynamics across different areas can reveal interesting aspects of neural organization. Certain patterns are typical of normal subjects. Predictable changes in these patterns are found over the course of development (Fair et al., 2008; Fair et al., 2009), 02_Morse_Ch02.indd 47 8/29/2012 2:26:29 AM

10 48 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE and it is possible that certain patterns deviating from normal may be indicative of psychiatric and neurological conditions (Bluhm et al., 2007; Broyd et al., 2009). One important feature of these studies is that, since they do not require the subject to perform specific tasks, it may be feasible to use this kind of paradigm in populations that are cognitively impaired, unable to follow directions, or noncompliant (though the subjects still need to be compliant enough to stay still). Thus, it may be of particular interest to populations relevant to the law. On the other hand, it is important to note that this work is still relatively new, and the interpretation of results is currently an issue of debate in the neuroimaging community (Morcom & Fletcher, 2007). We mention it here because it is possible that this technique will be increasingly important, and more widely accepted, in the future. F. Image Acquisition Is a Small Part of a Neuroimaging Experiment Regardless of how neuroimages are gathered, it is essential to bear in mind that image acquisition itself is merely a small piece in a much larger enterprise of functional neuroimaging. Image acquisition is sandwiched between two essential components of good neuroimaging studies: experimental design and data analysis. These two components are complicated and dramatically affect the results and quality of a neuroimaging study. Understanding these elements of neuroimaging is also critical to assessing neuroimaging data. AQ: Please provide in text citations for Figures 2.5, 2.7, 2.8, and 2.9. Figure 2.5 I m a g i n g re s t i n g- s t at e c on ne c t i v it y 02_Morse_Ch02.indd 48 8/29/2012 2:26:29 AM

11 Brain Imaging Techniques 49 Indeed, experimental design and analysis pose the greatest challenges in this enterprise. In what follows, we first discuss the capabilities and limitations of fmri, and then turn to several issues critical to designing solid neuroimaging experiments and to properly interpreting results. 4. CAPABILITIES AND LIMITATIONS OF FMRI A. S p a t i a l R e s o l u t i o n A number of factors affect the spatial and temporal resolution of fmri, but a primary factor is the strength of the magnetic field produced by the scanner. Various devices, such as head coils and surface coils, have also been developed to increase the measured signal strength for brain imaging. In theory, there is no firm lower limit to the abilities of MR to resolve spatial information, for with increasing gradient strength, improved head coils, and increasing imaging time, smaller and smaller structures can be distinguished. However, most magnets suitable for studying human brains are currently between 1.5 and 4 tesla in field strength (although recently 7 tesla magnets have been employed). Structural scans routinely are done at the millimeter level, and only lengthy scan time prevents higher-resolution structural scans. Spatial resolution in functional scans is further compromised by the need to scan larger volumes of tissue to reliably detect a change in the BOLD signal (see below), combined with the desire to scan as fast as possible. Functional images of human brains are rarely pushed below the level of several millimeters, about 10 times coarser than the resolution of typical anatomical scans, with voxels about the size of a large pea. A voxel this size may contain several million neurons. As with all imaging modalities, signal-to-noise ratio (SNR) is a principal limiting factor. The SNR is a measure of how much the information of interest (signal) is corrupted by junk information (noise), and thus is related to how confident one can be about the results of an experiment. For example, imagine trying to hear a person speaking in a noisy room. If the person speaks just a little louder than the noise, you may not be able to hear them, or you may not be sure about what you hear. If the person speaks much louder than the noise, you hear them more easily. The louder the person speaking (signal) compared to the noise, the better the SNR, and the more certain you can be that what you think to be the person s speech is in fact that, and not just noise. A similar principle applies to imaging. There are many sources of noise in an image, including slight variations in the applied magnetic field; thermal noise from the magnet, which can vary from session to session; noise from physiological factors, such as heartbeat or respiration; and bodily movement. SNR for fmri is proportional to the 02_Morse_Ch02.indd 49 8/29/2012 2:26:30 AM

12 50 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE intrinsic signal (including field strength), to voxel volume, and to the square root of imaging time. Reducing voxel size by half (i.e., voxel volume by a factor of 8) requires a 64-fold increase in imaging time in order to maintain the same SNR. Because the modulation of the BOLD signal magnitude during functional activation is relatively small, usually 2 to 5 percent over resting signal intensity, and often even less, the art of MR lies in maximizing signal to noise. Voxel sizes chosen in fmri experiments reflect this tradeoff, and not the spatial resolution of the technique per se. Efforts to improve SNR are at the heart of many other aspects of fmri experimentation. In addition to the sources of noise enumerated above, variations in scanner parameters; subtle differences in the age, gender, medical, or physiological state of the subject, including factors such as whether or not he or she has had caffeine, where he/she is in the circadian rhythm, and even small differences in task performance or strategy; all can have effects on the scan outcome. These factors can also affect reproducibility of the results and indicate the importance of controlling for unwanted variables. They also highlight the exquisite sensitivity of the technique. B. Te m p o r a l R e s o l u t i o n fmri became possible with the development of instrumentation enabling very rapid switching of magnetic field gradients and the invention of pulse sequences exploiting this technology (Stehling et al., 1991). Currently, a single MR image may be acquired in under 100 milliseconds and whole-brain data may be acquired in one to two seconds. However, the signal-to-noise quality of a single image is far too low to make single functional scans useful in and of themselves. Images from fmri are typically averaged over many like trials in order to yield fmri data with adequate SNR. C. The Hemodynamic Signal and Its Relation to Brain Activity Recall that the BOLD signal is due to a complex constellation of factors involving blood flow, blood volume, and oxygen consumption (see Fig. 2.6). Thus, fmri measures hemodynamic changes, not neural changes. Although there is a close correlation between aspects of neural activity and BOLD signal, the relationship is neither entirely understood nor straightforward. BOLD signal correlates more closely to changes due to local synaptic activity than to action potentials (Logothetis, 2008; Logothetis & Wandell, 2004), and other factors besides neural activity are associated with hemodynamic responses (Sirotin & Das, 2009). Researchers continue to explore the nature o f t h i s r e l a t i on s h ip. 02_Morse_Ch02.indd 50 8/29/2012 2:26:30 AM

13 Brain Imaging Techniques 51 T2* weighted image intensity + MR properties Physical effects Decay time (T2*) + Magnetic field uniformity (microscopic) Other Factors Vessel diameter Vessel orientation Hematocrit Blood volume fraction + Physiological effects Cerebral blood volume (CBV) + Blood oxygenation + + Cerebral blood flow (CBF) + Metabolic rates Brain function Glucose amd oxygen metabolism + Neuronal activity Figure 2.6 Schematic of changes resulting in BOLD signal Th ere are considerable limitations to the types of functional inferences that can be made from fmri data. For example, both excitatory and inhibitory neural transmissions use energy, and both therefore lead to increased blood flow and metabolic demands. Consequently, the fmri signal cannot distinguish between excitatory and inhibitory neural activity, despite the fact that the two types of transmission have radically different functional consequences. Combining activation data with other information, such as the location or connectivity of the region, can help constrain these interpretations. The BOLD response is not straightforwardly spatially or temporally correlated with precise regions of neural activity. For example, some fmri pulse sequences are thought to highlight draining veins, which may be several millimeters from the site of neuronal activity causing the oxygenation changes, while other sequences are thought to target capillary beds of the active tissue more specifically. As research in fmri has progressed, pulse sequences have 02_Morse_Ch02.indd 51 8/29/2012 2:26:30 AM

14 52 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE been optimized to reflect physiological variables of interest and to minimize artifacts. However, there are always tradeoffs. For example, scanning with small voxel size biases the technique toward identifying small regions of large oxygenation changes, which again may correspond to the locations of draining veins, not neural tissue. These measurements could provide a somewhat misleading picture about the location and magnitude of underlying neural activity. In addition, the blood flow response is not temporally coincident with neural activity. Changes in blood flow lag several seconds behind the neural activity that engenders them, and it takes a few seconds for the hemodynamic signal to build to its maximum and then to decay, even though the neural activity could take place in tens of milliseconds (Buxton, 2002). So, although fmri can collect measurements of blood flow changes in close to real-time, what it measures corresponds to neural activity that has happened several seconds previously. Nonetheless, current data suggest that the onset and time course of the measured hemodynamic response in a given area to a particular stimulus is quite reliable, which may allow certain inferences to be drawn from data about the temporal properties of the measured responses (Kiehl & Liddle, 2003; Miezin et al., 2000). In essence, with fmri our view of brain activity is smeared in time and space by the hemodynamic response; despite these limitations, the response gives us important information about brain function. 5. I S S U E S I N E X P E R I M E N T A L D E S I G N The brain is constantly active, blood is always flowing, and most of the modulation of blood flow is not task-related. BOLD signal in a brain area, by itself, does not tell us much about function. A great deal of design and processing goes into constructing those compelling brain images with colored hot spots that one often sees. They are not photographs of the brain. Brain images differ from photographs in a myriad of ways: one being that the images one sees are nearly always comparisons or contrasts between states. 3 In activation paradigms, responses are typically measured and compared between two or more task conditions during a series of scans conducted in a single experimental session. Regional differences in the measured signal between various tasks are taken to reflect differences in the amount of local neuronal activity associated with performance of those tasks. In order to understand a functional imaging study, one must first establish what tasks the experiment involves and what comparisons are made. Are they the appropriate ones? 02_Morse_Ch02.indd 52 8/29/2012 2:26:30 AM

15 Brain Imaging Techniques 53 A. Functional Decomposition and Experimental Design Complex cognitive tasks are almost always composed of numerous suboperations: for instance, for most activation experiments, a subject receives instructions, perceives stimuli, performs certain cognitive operations, and responds overtly in some prescribed way. Functional decomposition refers to the conceptual breakdown of a complete task into its component parts, a carving of the entirety of a task along its functional joints. Although it rarely gets much attention, careful and appropriate functional decomposition lies at the heart of successful neuroimaging experiments. The most straightforward approach to experimental design is to characterize a target task with respect to its input, processing demands, and output. The ideal experiment holds as many of these components fixed across comparison tasks as possible, and manipulates the fewest variables possible at a time. If relevant psychological or neuropsychological evidence regarding the nature of the task is available, it can be used to inform the functional decomposition for a neuroimaging experiment. Comparison tasks must be closely matched with the target task on a number of fronts in order to try to manipulate specific task demands, and to hold constant, as much as possible, task features irrelevant to the issue at hand. Because so many cognitive operations of interest are not well understood, a simple, Task or behavior of interest Functional decomposition Comparison task 1 Process 1 Comparison task 2 Functional decomposition Process 2 Functional decomposition Process 1 Process 3 Process 3 Process 2 Process 4 Process 5 Figure 2.7 Comparisons AQ: Please provide Figure caption 02_Morse_Ch02.indd 53 8/29/2012 2:26:30 AM

16 54 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE streamlined experimental design, varying the fewest possible components not directly relevant to the task of interest, will be the most interpretable. In this way, differences in activation between scan conditions can be clearly attributed to the manipulated variable(s). Analogously, parameters held constant between tasks may be used to infer the functional roles of regions with consistent differences in activity between active states and baseline. B. P i t f a l l s o f F u n c t i o n a l A t t r i b u t i o n Given that neuroimaging strives to attribute functional roles to specific brain regions, or sets of regions, it is not surprising, then, that papers in the neuroimaging literature frequently assert, implicitly or explicitly, that this study proves that area X subserves function Y. This is seldom accurate. It is virtually impossible to interpret the functional relevance of any particular activated region on the basis of a single, or even a few, scans. What can be said with assurance is that Complex Task A performed with the parameters used in the experiment leads to activity in Region X, and Task A can be provisionally functionally decomposed into Subtasks W, Y, Z, etc. At its best, a neuroimaging experiment can provide strong evidence implicating that region in a particular process; no single study can prove the function of a particular brain region. Strong claims of functional attribution require an extensive and iterative process we refer to as functional triangulation. Much like triangulation in surveying, the getting the lay of the functional anatomical landscape requires multiple measurements during a variety of tasks and a continual honing of the interpretation of the role a particular area plays in cognition. Consistent activation in a particular brain region seen across a wide range of tasks that share a functional subcomponent constitutes good evidence for functional attribution; absence of that region in similar tasks lacking that functional component also strengthens the argument (However, because the analysis is meant to minimize false positives, care must be taken when making claims about lack of activation). The challenge of functional triangulation lies in devising appropriate tasks to test functional hypotheses. Since the space of cognitive function is high-dimensional, many measurements may be needed in order to accumulate sufficient evidence to triangulate function to an area in any sense, with the proviso that any functional attribution is open to revision when inconsistent data are presented. Our intuitive functional decompositions are provisional; they may not accurately reflect the way brain processes are organized. Recognition of the importance of functional triangulation and the provisional nature of our theories of functional organization is crucial for the law to recognize, for inferences made on the basis of these functional decompositions 02_Morse_Ch02.indd 54 8/29/2012 2:26:30 AM

17 Brain Imaging Techniques 55 are subject to error. Both claims that Area X subserves Function Y (forward inferences) and that Area X was active, thus Function Y was involved (reverse inferences) are contestable, and the probability of error is difficult to estimate, though techniques for better doing so are being developed (see, for example, Friston et al., 2008; Poldrack, 2006). Grounding legal decisions on the basis of such claims may not be warranted, both because of their provisional nature and because error rates are unknown. C. Ta s k Ty p e s Th ree types of tasks are used in activation studies: target tasks involve a task or subtask of interest; comparison tasks are tasks similar to the target task, which manipulate or hold constant a component of interest in the target task; and baseline tasks are low-level tasks, used in effect to represent an inactive state. For example, the task of interest may be a visual memory task; a comparison task may involve seeing but not being asked to remember the same stimuli, and a baseline task may be passively viewing a blank screen. Differences in regional activation between active task scans, or between active and baseline task scans, reflect differences in the associated task demands. Different target tasks can also be compared to each other with respect to a common baseline task, which reveals both the commonalities and differences between the target tasks. Selection of the tasks involved in a study is critically important, strongly affecting its interpretability and outcome. The choice of comparison and baseline conditions is in part governed by the functional decomposition(s) of the primary task of interest. Many activation studies are interested in understanding the neural processing underlying a particular task, the target task. Ideally, we could compare levels of activity during the target task to activity levels when the brain is doing nothing or at rest. While this is an impossibility, for the brain is never really at rest (it governs homeostatic functions, sets the level of arousal or alertness, etc.), it is possible to devise conditions that involve minimal amounts of cognitive processing (Gusnard & Raichle, 2001; Raichle et al., 2001). Whether this is indeed the case is disputed in the field (see, for example, Morcom & Fletcher, 2007). Baseline tasks are typically chosen to be simple tasks that do not share important features or task demands with the tasks of interest. Many researchers use standard baseline tasks in all their studies, regardless of the study s goal. This makes it easier to begin to find common patterns of activation across numerous studies, and allows the possibility of future meta-analyses. Regardless of the choice of baseline task, it is of primary importance to be aware of the cognitive operations required by that task, as well as others in which the subject might engage. Comparison tasks are more complex tasks, often high-level tasks chosen either because they differ in some specific way from the target task or because 02_Morse_Ch02.indd 55 8/29/2012 2:26:30 AM

18 56 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE they share important features with the primary task of interest, or both. Functional decomposition of the primary task of interest can identify functional characteristics that should be shared or absent in potential comparison tasks in order to test a specific hypothesis concerning the functional role of an area or areas or a particular functional decomposition. The choice of comparison or contrast tasks is a major determinant of the ultimate perspicuity and value of a study: careful choice of tasks can go far in disclosing the functional roles of one or a few brain regions, while inappropriate choice of tasks can result in a study which fails to target any specific cognitive question. D. D e s i g n Ty p e s AQ: Please provide complete details of Buckner et al., 1996 in the reference list. The ability of fmri to acquire data in brief time slices confers important advantages over other imaging methods, such as PET. Blocked designs are necessary in PET, and used to be common in fmri, but are becoming less so. In these, subjects perform one type of task for a period of time (in fmri, usually between seconds, and in PET for several minutes) and then they switch to another type of task. Blocked designs are relatively easy to analyze, for the data are averaged across blocks of the same task type, and subtracted from data averaged across blocks of a different task type. The information one gains from such studies reflects aggregate differences in brain activity in different regions between the two task periods. Blocked designs have some serious limitations, however. Some tasks are performed differently when one knows exactly what sort of task one has to perform, and the ability to distinguish between different trials is lost. Thus, certain kinds of information regarding differences in task or performance cannot be investigated by this method, such as differences between correct and error trials. Newer experimental designs, called single-trial or event-related designs, take advantage of the higher temporal resolution of fmri. This kind of design is more flexible, making possible a variety of sophisticated experiments. Different trial types can be randomly interleaved, and data can be analyzed based on aspects of the stimulus or the behavioral response (Buckner et al., 1996). These methods allow brain data associated with individual trials such as type of stimulus, whether or not the subject performed correctly, and reaction time to be probed. Single-trial designs are often more powerful than blocked designs for exploring cognitive function. In addition, these methods reveal hemodynamic changes related to the criteria for sorting, capturing subtler differences in task performance than blocked methods. Event-related analyses can achieve better time resolution of regional activity than can blocked subtractive designs. These methods can also provide access to the timecourse of the hemodynamic response, information that can be relevant to understanding cognitive processes. 02_Morse_Ch02.indd 56 8/29/2012 2:26:30 AM

19 Brain Imaging Techniques 57 Event-related designs are typically analyzed by correlating activation patterns with mathematical models of task performance that incorporate various task-related aspects of the experiment, such as stimulus onset, response timing, response type, etc. The models are based on functional decompositions; the validity of the results will depend on the accuracy of the models. These event-related paradigms are now widely used (Buckner, 1998; Burock et al., 1998; Dale, 1999). E. The Importance of Behavior Many people fail to understand the importance of behavioral science in understanding cognition in general, and in functional neuroimaging. However, behavioral measures establish important checks in imaging studies. A minimal requirement for a well-designed study is that subjects actually perform the task the experimenter believes they are performing. Thus, it is desirable, if at all possible, to establish that the subject is in fact performing the task. To do this, behavioral measures are usually acquired during the scanning session. Data such as reaction times, accuracy measures, frequency of action, etc., can be used to assess task performance and the strategies being used, both of which may have significant impact on the interpretation of neuroimaging results. Sometimes behavioral measures can be acquired after the scan session; AQ: Please provide better quality figure. Figure 2.8 Schematic of blocked designs and event-related designs; figure of time course from event-related experiment. Schematic of boxcar function and subtraction, and of model and event-related data. 02_Morse_Ch02.indd 57 8/29/2012 2:26:30 AM

20 58 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE for example, to establish that memory encoding was indeed occurring when stimuli were presented during the experiment. However, there are limits to what behaviors can be performed in a scanner, both because motion compromises scan quality, and because of circumstances attending experimental design. For example, it is extremely difficult to construct a study to image lying or deception, since following the experimenter s orders to not tell the truth, even sporadically, is not the same as lying. There may be some behaviors of interest to the law that cannot be adequately investigated by these experimental techniques. 6. A N A L Y S I S O F N E U R O I M A G I N G R E S U L T S Once subjects have been scanned while performing tasks, a great deal of data processing and statistical analysis ensues in order to produce the images that we have all become familiar with. As was mentioned earlier, the greatest technical challenge in neuroimaging, and one of its prime limiting factors, lies in the noisiness of the technique. A single fmri scan has too low a signal-tonoise ratio to warrant any conclusions. Various methods of experimental design and analysis have been developed to increase SNR. However, many of the practical limitations of fmri are also a consequence of these attempts to improve SNR. A. Av e r a g i n g D a t a Perhaps the most effective way to deal with low SNR is to pool data over multiple trials. Because fmri is noninvasive and does not involve radioactive substances, there is in theory no limit to the number of times a single subject can be scanned. This provides a substantial advantage over PET in gaining signal over noise, for it allows, in theory, unlimited opportunities to average data. However, data averaging is a two-edged sword, and one must remain sensitive to both the benefits and costs imposed by methods of data averaging. All neuroimaging methods average data over multiple trials, but they can do it in different ways, each with different consequences. The issue of averaging is of prime importance to the law, for science tends to focus on generalizations that are true of the population at large, whereas the law is interested in individual cases. This difference in focus often makes it difficult to apply neuroscientific findings to legal questions. a. Wi t h i n - s u bj e c t s t u di e s Some neuroimaging studies are done with single subjects. A subject might remain in the scanner performing a task hundreds of times, and sometimes 02_Morse_Ch02.indd 58 8/29/2012 2:26:31 AM

21 Brain Imaging Techniques 59 may return for multiple scan sessions. These within-subject studies average data across many trials in one subject. Some tasks are well-suited to this, and others are not. For example, many visual and motor tasks can be performed numerous times and seem to be performed in the same way; other tasks change with practice, rely on novelty, or are too demanding to realistically perform enough times to get adequate signal to noise. Within-subject studies can reveal important information about how an individual performs different tasks. Comparing results from different individuals in the domain of memory, for example, reveals that there is variability among individuals (Miller et al., 2002). However, evidence suggests that individual patterns of activation are reliable (Aron et al., 2006; Miller et al., 2002). Th e ability to perform studies on individuals is important for legal purposes, since neuroscientific evidence is usually brought to bear only in cases concerning individuals. For this, within-subject studies are essential. However, often the law is interested in individual responses in tasks that have only one or a few instances, such as the veridicality of responses to questions about a particular issue. This sort of subject will be difficult to ascertain with fmri. For many other legal purposes, the relevant issue is how the individual compares to others, and for this between-subject studies are more relevant. b. Between-subject or across-subject studies To say something about human cognition, most neuroimaging studies use multiple subjects performing the same task, and they average data across subjects. While this may achieve greater signal to noise because of larger sample size, it may also introduce more variability and noise into the results, because there are individual differences in brain size, shape, and even functional anatomy. Scientists have developed several methods for warping the data from individual brains into a standard anatomical template that permits averaging, but the warping algorithms are not perfect, and they introduce some error into the data. Averaging data across multiple subjects can help increase signal to noise when functional anatomy is consistent across subjects; when inconsistent it can average out real effects that have variability in location. It is also possible that different subjects can use different strategies when performing complex tasks, further introducing variability into the data set (Miller et al., 2002; Seghier et al., 2008b). The degree of individual functional-anatomical variability is a critical empirical question for which we do not yet have answers. Comparisons between data from single subjects performing the same task shows that some tasks demonstrate considerable variability. It is unclear whether this is due to anatomical 02_Morse_Ch02.indd 59 8/29/2012 2:26:31 AM

22 60 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE AQ: Please provide better quality figure. Figure 2.9 Individual data and group averages differences, functional differences, or differences in strategy. It is clear, however, that the average of data from multiple subjects may not look much like the average of data from any individual subject (Miller et al., 2002). Moreover, it demonstrates that activation patterns in subjects considered to be normal can vary considerably. In addition, merely computing the average image across subjects, which is what most scientific studies currently do, does not provide any information regarding the range and degree of variability across the population (Mikl et al., 2008; Seghier et al., 2008a). These issues pose severe challenges for the use of neuroimaging in law. The ability of neuroimaging to be relevant to the law will depend critically upon an understanding of individual variability, especially with regard to arguments that an individual falls outside the normal range with respect to brain structure or function. Without adequate baseline studies and measures of variability among the general population, such claims cannot be evaluated. B. Movement, Smoothing, and Normalizing Several other methods are commonly used in analyzing fmri data. Most analyses eliminate scans that show evidence of head motion, which creates artifacts, 02_Morse_Ch02.indd 60 8/29/2012 2:26:31 AM

23 Brain Imaging Techniques 61 AQ: Please provide better quality figure. Fi g u r e 2.10 Example of functional/anatomical variability or visible corruption of the data. Some motion may be corrected by automated algorithms. These steps are important to eliminate artifacts in the data, but the algorithms are not perfect and some artifacts may remain. Many imaging studies smooth their raw data in order to increase signal to noise. The degree of smoothing can change the resulting image (Mikl et al., 2008). There are no clear-cut guidelines on whether or not to smooth. Normalization involves standardizing mean signal levels across different subjects, for absolute signal strength can vary across scan sessions, across scanners, etc. Normalization is important to ensure that individual subject data sets contribute equally to multi-subject analyses. 02_Morse_Ch02.indd 61 8/29/2012 2:26:31 AM

24 62 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE C. C o m p a r i s o n s Almost all neuroimaging experiments are interested in comparing data between different task conditions. Various methods are commonly used to determine regional differences in brain activity. The methods used depend upon the experimental design and interests of the experimenter. a. Subtractive and correlation analyses fmri data can be analyzed either with subtractive techniques (for example, in blocked designs) or by correlations of the activation time courses with a time-varying model signal (such as in event-related designs). 4 Su bt r a c t ion paradigms directly compare two tasks by subtracting the data acquired during the performance of one task from the data acquired during the performance of another, revealing regions differentially activated between the two tasks. In subtraction, averaged and normalized images of one comparison or baseline condition are subtracted from averaged and normalized images of a target task. Regions of positive intensity in the resulting image reflect brain regions more active during the target task; regions of negative intensity in the resulting image reflect regions more active during the comparison task. Differences in regional activation between scan conditions can be attributed to differences between the tasks performed (Petersen et al., 1988; Posner et al., 1988). Despite some misunderstanding about the assumptions inherent in subtractive techniques (Uttal, 2001), the implications of image subtraction go no further subtraction is merely a means of revealing differences in activity between two conditions. Correlation analysis, used customarily in event-related designs, involves more assumptions about task performance, since the data are correlated with a model that incorporates elements of the functional decomposition of the task, as well as information about temporal delays of hemodynamic changes. Mathematical algorithms are used that try to find the best fit between model and data (Friston et al., 1995a; Friston et al., 1995b; Worsley & Friston, 1995). Both subtraction and correlation analyses of fmri data can employ a phase-shift to take into account the delay of the hemodynamic signal, with noticeable improvements in signal. The advantage of subtraction over the correlation method is that data indicating the magnitude of responses are not discarded. The magnitude of a response is a significant aspect of brain activation, and an important complement to the location of the response. Correlation s advantage over subtraction is that the reference time course can be constructed to include signal rise- and fall-times that more closely approximate biologically relevant variables, leading, in some cases, to 02_Morse_Ch02.indd 62 8/29/2012 2:26:32 AM

25 Brain Imaging Techniques 63 higher signal to noise. The best of both methods can be used by doing an initial correlation analysis and then reporting magnitude changes in regions significantly active. b. Mu lt i va r i at e a na lys e s Subtractive and correlation analyses use linear methods to identify brain regions showing change in hemodynamic signal correlated with task. Multivariate analyses, which are increasingly popular, look for distributed patterns in the data and correlate multi-voxel patterns with task components (Haxby et al., 2001). The idea behind these analyses is that subtle patterns within regions of increased task-related activity carry information about the task or task processing. Levels of signal change across many voxels are classified by machinelearning algorithms that correlate these patterns with stimulus type, response type, or other task-related aspect (Eger et al., 2008; Haynes 2008; Norman et al., 2006). These classifiers have demonstrated remarkable predictive abilities in some studies, showing that the patterns discerned are generalizable within constrained domains. For example, some of the studies that purport to be able to read off mental states from fmri data use these techniques, and they train the classifier on data from the individual subject (Mitchell et al., 2008; Soon et al., 2008). However, the limits of the generalizability of the findings remains to be determined; it is likely that the limitations will be significant with respect to predictive abilities relevant to the law. As an example, a classifier was trained on brain data from a person when looking at one of 10 tools and dwellings that could predict with between 60 to 90 percent accuracy which of the stimuli that person was looking at. When trained on data from 11 people, and used to predict the object shown to a different person, the accuracy generally fell, but remained higher than chance, indicating some commonality to neural representations across subjects (Shinkareva et al., 2008). To what extent this technique would be reliable in people without prior training on constrained stimulus sets is an open question. The question has relevance for the possibilities of lie-detection, for example, or other cases in which prior training to a particular individual s patterns of brain activity is likely to be impossible (Haynes, 2008). D. A n a l y s i s o f S i g n i f i c a n c e All experimental data are subject to confounds, error, and noise. Statistical analyses are used to evaluate the reliability of results and to establish levels of confidence in scientific claims. Common across many sciences is a statistical measure of significance of p < 0.05, which indicates less than a 5 percent chance of error in the claim that a finding is significant. However, in fmri, each voxel 02_Morse_Ch02.indd 63 8/29/2012 2:26:32 AM

26 64 A PRIMER ON CRIMINAL LAW AND NEUROSCIENCE is a non-independent measure. Because fmri analyses are performed across thousands of non-independent data-points at once, using a statistical measure of p < 0.05 would result in hundreds of false positives throughout the brain: for example, 5 percent of 10,000 is 500. Statistical correction for multiple comparisons must be made when analyzing brain imaging data, but precisely how to do this is a topic of continuing debate in the field. Some researchers suspect that a large percentage of findings reported in the neuroimaging literature as statistically significant are in fact not. Many researchers use automated software packages, such as SPM (Statistical Parametric Mapping) that process and analyze the data (Friston et al., 1995a; Friston et al., 1995b; Friston et al., 1991; Worsley & Friston, 1995). In employing such programs, researchers may not know exactly what statistical and other manipulations are being done. While these packages are immensely helpful to the scientific community, in some situations the canned statistics may not be appropriate. The opacity of the analysis also makes outside evaluation of the methodology difficult. How best to apply statistical techniques is among the most vexing questions in neuroimaging. E. D i s p l a y i n g N e u r o i m a g i n g D a t a Data from neuroimaging may also be visually presented in a variety of ways. Neuroimaging data are most often presented as colored areas painted on the grayscale image of a brain slice or a three-dimensionally rendered brain surface, but sometimes they are presented on an inflated or flattened brain. Often the areas of significant activation are overlaid upon high-resolution grayscale anatomical images, and the edges are smoothed to make the image appear more detailed and the areas of activation more organic. In addition, there are multiple meanings such colorized pixels could have. Sometimes the colors represent percent signal change. Sometimes they represent areas of statistically significant activation or deactivation. They may show all regions that exceed a particular value for signal change or significance, or they may use a graded color scale to represent different values, or both. While it is common for researchers to use hot colors to represent activation and cool colors to represent deactivation, all these choices are merely conventional. Areas of the brain do not light up, they do not change color, and the colorized images are not photographs of the brain. Nonetheless, these conventional choices probably have compelling psychological consequences. Many people take neuroimages to be pictures of the brain at work and think they are seeing the brain in action (Roskies, 2007). The pictorial aspects of brain 02_Morse_Ch02.indd 64 8/29/2012 2:26:32 AM

27 (A) (B) P<0.005 P<0.05 P<0.005 P<0.05 (C) (D) (E) (F) (G) Caret SurfRend Figure 2.11 of the same data represented in a number of ways Au: words missing? 02_Morse_Ch02.indd 65 8/29/2012 2:26:32 AM