Sensory Processes, Npsy 12a rev.c page 1 Visual Memory Laboratory Experiment designed and programmed by Yuko Yotsumto Explanatory handout prepared by Robert Sekuler Background for Experiment. We used S. Sternberg s visual recognition paradigm in order to examine episodic recognition memory for synthetic human faces. Timing and operations on each trial. On each self-initiated trial, a subject saw a trio of sequentiallypresented faces, each 250 msec in duration. These three faces, which were separated by interstimulus intervals of 200 msec each, comprised the trial s Study items. Then a warning tone was followed by a final, Probe face (250 msec). (The interval between the end of the third Study face and the onset of the Probe face was 1200 msec.) The subject judged whether the Probe face had or had not been among the Study items just seen. Immediate knowledge of results was provided by distinctive tones for correct and incorrect responses. As is customary in Sternberg paradigm, on half the trials, the Probe item had been in the Study set, and on half the trials it had not. Some needed terminology: when a Probe had been in the Study set, it is called a Target; when a Probe had not been in the Study set, it is called a Lure. Each face subtended ~4.5 by 7 degrees visual angle; mean luminance varied slightly between computer displays, averaging ~40 cd/m 2. Principal hypotheses. 1. The accuracy of recognition memory depends upon the inter-item similarities among the Study items. 2. When the Probe is a Lure, accuracy of recognition depends upon the Lure s similarity to the Study items. 3. When the Probe is a Target, accuracy of recognition depends upon the matching Study item s position, first, second, or third, in the Study series. Stimulus materials: General information. The faces used in the experiment were generated by Hugh R. Wilson, Günther Loeffler, and Fran Wilkinson (York University, Ontario). The research paper that describes the stimuli and their properties is quite technical, and will not be published for another month or so. Although the Wilson et al. paper is an elegant piece of work, reading it is not necessary for this lab report, or for understanding the experiment we did. The 20 faces used in our experiment are shown in the face matrix at the end of this handout; these faces were selected from a much larger population. In general, Wilson et al. s synthetic faces afford several demonstrable advantages as research tools: 1. They embody basic facial characteristics sufficient to permit the face to be identified as belonging to a particular individual. (This should remind you of the Greeble studies described in Sekuler and Blake s textbook). 2. They omit irrelevant or accidental features such as moles, shaving cuts, skin texture, or pimples. 3. They omit features that convey emotion, such as the upturn or downturn of the mouth, and the orientation of the eyebrows. The faces emotional neutrality eliminates the possibility of basing recognition on such cues, which can change over time. 4. Faces are equated in terms of average contrast, size, luminance, and spatial frequency content, eliminating these irrelevant features as potential clues to identity.
Sensory Processes, Npsy 12a rev.c page 2 5. The synthetic faces are treated by the nervous system as faces. For example, when someone tries to distinguish between two of the synthetic faces, or tries to recognize them, fmri studies show activation in the infero-temporal cortex s face area (see Sekuler and Blake s Chapter Six). Moreover, these synthetic faces produce face-inversion effects that are as strong as those produced by real, non- synthetic faces. 6. Most importantly, the synthetic faces can be represented simply, in mathematical form, in equation form. The parameters in a face s equation represent qualities such as the overall shape of the face, the height and separation of the eyes, the length and width of the nose, the shape of the hairline, and so on, for a total of 37 parameters. These parameters vary from face to face, and the computer uses those parameters to generate each face that we see on the computer display. This mathematical description makes it possible to create new faces that are known mixtures of particular faces. For example, you could create a face that is 5% Ben Affleck and 95% Adam Sandler, or one that is 90% Ben Affleck and 10% Adam Sandler, or any other combination that you might be able to stomach, including combinations of more than two faces. This graded variation is accomplished by adjusting the weights of some or all of the 37 parameters in the equation that describes any face. This enables us to generate faces that differ from one another by any amount, small or large, that we want. This metric variation controls the physical differences (and perceptual dissimilarities) among faces. And we know that inter-item similarity strongly influences recognition memory. (For details of such effects, you might consult Kahana and Sekuler (2002) and/or Zhou, Kahana and Sekuler (in press). These articles are available for downloading from the Publications link at www.brandeis.edu/~sekuler. The first of these articles is quite technical; neither article deals with face stimuli per se, but both discuss the role of inter-item similarity in recognition memory. As a result, they may be worth at least a glance or two, particularly figures showing results of data analysis. Stimulus materials: Brief information on the 20 face stimuli used in Npsy12 laboratory. During the experiment, faces were drawn from a set of 20 different synthetic faces. All were mixtures of (a) an average female face (a face in which 40 different female faces were represented equally), and (b) one of three different particular female faces. The average female face is shown in Figure 1. Then three of the female faces that went into the average were selected; we ll call the three A, B, and C. Photographs of the actual people on whom these three synthesized were based are shown in Fig 2. They should resemble synthesized faces 6, 13 and 20 in the face matrix at end of this handout. Fig.1.Average female face, based on 40 faces. Figure 2. Photographs of actual people on whom synthesized faces A, B, and C were based. The faces in rows A, B, and C were created by mixing a synthesized version of each real face with the synthesized average face. The proportions contributed by the real faces vary across each row of the matrix; those proportions are shown at the bottom of the face matrix. For example, the faces in the left column of the face matrix comprise 4% of a real face and 96% of the average face.
Sensory Processes, Npsy 12a rev.c page 3 Now what about row D? The faces in row D are equal mixtures of faces from the other rows. For example, face 24 is an equal mixture of faces 3, 10, and 17. This means that each face in row D has some resemblance to all the faces with which it shares a column. As a result, face 4 is more different from face 11 or from face 18, than any of those faces is from face 25. We can restate the point this way: suppose you put all the faces into a 3D space. The average face (Fig. 1) would be at the origin (0, 0, 0) of the 3D space. The faces in row A would lie along one axis in the space; the faces in row B would lie along a second axis; the faces in row C would lie along the third axis. The faces in row D would lie along one diagonal in that 3D space. So far, I ve described the stimuli in terms of physical characteristics and physical differences, that is characteristics represented in the equations used to generate them. But, to understand how perceptual similarity might affect visual memory, we have to take account not of the physical differences among the faces, but of their perceptual differences. These two things physical and perceptual differences-- turn out to be related, but not identical. Here s how we measured perceptual differences among the faces (this experiment was carried out by Yuko Yotsumoto). Perceived dissimilarities among Dimension 2 2 1 0-1 -2 c4 c5c3 c2 c1 d1 d2 b1 a1 mean a2 d3 d5 b2 d4 a3 b3 a5 a4 b4 b5 2 1 1.01.5 0 0.0.5-1 -.5-2 -1.0-2.0-1.5 Dimension 1 Dimension 3 faces were measured using the method of triads (discussed in class). Fifteen subjects dissimilarity judgments were analyzed by multidimensional scaling (MDS). As you have heard in class, and read in the Appendix to the textbook, MDS transforms perceived dissimilarities (or the inverse, similarities) between stimuli into distances within a Euclidean space. The more dissimilar two objects are perceived to be, the larger the distance between the objects representations in the Euclidean space. An MDS solution in 3-dimensions gave a very good account of dissimilarity judgments with faces. The figure above shows the MDS solution. Each data point in the plot represents a single face, and distance between dots represents the similarity between faces. You do not need to try to read the values of points in the MDS plot. All necessary values are provided in the data sets. In analyzing the data from the Npsy12 laboratory, you will use the MDS-derived, inter-face distances as the metric of inter-face similarity (remembering, of course, that distance and similarity are inversely related: small distance means large similarity). For our purposes, you need not worry about what the dimensions mean that s a whole, other story, for some other time. (Chapter 13 of the texbook presents an MDS solution, one of whose axes was difficult to describe in words.) The important point to note is as we generate trios of Study items from the face matrix, we can create study lists whose items are perceptually highly similar to one another, or lists whose items are quite different from one another, or lists that lie somewhere between. Stop here, and, using the face matrix at the end of the handout, make sure you can see how this variation in interitem similarity is produced. This is important. Design of experiment. In our experiment, subjects memory was assessed in two different conditions, which we designate A and B. (You may recall that your subject ID number contained
Sensory Processes, Npsy 12a rev.c page 4 either an A or a B.) The Study sets were constructed differently for the two conditions. In condition A, the study items were forced to come from three different rows in the face matrix below. For example, one face might be a random choice from row A, another from row D, and the third from row B. Or, the Study faces might come from row C, row D, and row A. In condition B, the three Study faces were selected at random, with only the constraint that no face could appear twice in the Study set. At this point, stop to consider how the two different rules for generating Study sets, relates to hypothesis 1 (above). Which rule will tend, on average, to produce Study sets whose members are likely to be more similar to one another? Half the Probes were Targets and half were Lures, which brings us to hypotheses 2 and 3 (above). In either condition, a Lure was allowed to come from any of the four sets of faces, A-D. There was no constraint on the choice, other than the obvious one: that the chosen face cannot have been in the Study set, otherwise it could not qualify as a Lure. Data. You will download two sets of data, your own and a set generated by someone tested in the condition that you were not tested with. If you were in Group A, your data file is at http://people.brandeis.edu/~yotumoto/data_groupa/. If you were in Group B, your data file is at http://people.brandeis.edu/~yotumoto/data_groupb/ If you don t recall which group you were in, or what your subject identification number was, a list of subject identifications is available, as an Excel spreadsheet, subjectid, at the top the pages from which data files are downloaded. You should download your own set of data, and the set of data for the other group s subject who has the same identification number as yours. For example, if you were subject A05, you should download data file A05, and then data file B05. Subject B05 would work with the same two files. You should use these two sets of data to evaluate the three hypotheses stated above. Each set will be in a tab-delimited, text file. These can be read into Microsoft Excel, Corel Quattro Pro, Sun StarOffice, ThinkFree Office, Appleworks, or any comparable program or office suite. You will analyze, plot, and discuss your data, in order to evaluate the three hypotheses stated above. You will not do any inferential statistics, e.g., significance tests, but you should do needed descriptive statistics. Your analysis should focus on response accuracy, ignoring response times, which are given in the data sheets. The principal statistics that you should calculate are ones that describe recognition accuracy: Proportions of correct responses for various conditions. As shown in Sekuler and Kahana (2002), it is important to assess recognition accuracy separately for Lure and Probe trials. Also, it can be important to make separate assessments for each of three possible types of Probe trials: trials on which the Probe matched the first Study item, the second Study item, or the third Study item. (These three types, incidentally, were equally probable.) If you are familiar with Excel s PivotTable function, you may find that this function could expedite your data analysis, But, if you do not know how to use this function already, be very careful if you try to learn on the job, with these sets of data. Some good advice: once you ve read the two sets of data into spreadsheets, make copies of those sheets, and do your data analysis on the copies only. The first 13 rows in the data file give background information for the experimental run, time of day, subject ID, etc. In analyzing the data, ignore this background information (except for the subject ID). Below this, data from each of the 432 trials is given in a separate row. Here are explanations of the numbers found in each column: ntrial: number of trials, 1 to 432 Rtime: Reaction Time in msec (the time of button press onset of the Probe). Ignore this in analysis. Corr(1)/Incorr(0): correct or incorrect. Correct answer = 1, Incorrect answer = 0. Response: your response. Yes = 1, No=0. Ignore this in analysis.
Sensory Processes, Npsy 12a rev.c page 5 LL: List length. The number of Study items on the trial. For this experiment, LL always =3. Ignore. PP: Probe Position. On Target trials, which Study item matched the Probe: 1, 2 or 3. PP=0 means the trial was a Lure trial. S1_ID: ID for the first study face, values are keyed to the face matrix below. S2_ID: ID for the second study face, values are keyed to the face matrix below. S3_ID: ID of the third study face, values are keyed to the face matrix below. ProbeID: ID of the probe face, values are keyed to the face matrix below. S1mod: Face type for first Study item (1=A, 2=B, 3=C, 4=D). Letter is the row in face matrix. S2mod: Face type for second Study item (1=A, 2=B, 3=C, 4=D). Letter is the row in face matrix. S3mod: Face type for third study item (1=A, 2=B, 3=C, 4=D). Letter is the row in face matrix. Pmod: Face type for Probe (1=A, 2=B, 3=C, 4=D). Letter is the row in face matrix. p-s1 distance: perceptual distance between the Probe and the first Study item p-s2 distance: perceptual distance between the Probe and the second Study item p-s3 distance: perceptual distance between the Probe and the third Study item nearest non-matching Study item (binned): perceptual distance between the Probe and the nonmatching Study item that is most similar to the Probe. bin 1 represents small values, bin 2, large values mean distance between Probe and Study items (binned): Mean perceptual distance between the Probe and all Study items. Bin 1 represents the smallest values; bin 4 represents the largest values mean distance between Study items (binned): Mean perceptual distance between Study Items; bin 1 (small values) - bin2 (large values) P-nearest Lure distance: perceptual distance between the Probe and the one non-matching Study item that is most similar to the Probe. mean P-lures distance: Mean perceptual distance between the Probe and all Study items. For Lure trials, this is calculated as [(distance between P and S1)+(distance between P and S2)+(distance between P and S3)]/3; for Target trials, this is calculated as [(distance between P and non target Study item)+(distance between P and the other non target S)]/2. mean distance between Study items: Mean pairwise, perceptual distance for Study Items. Calculated as [(distance between S1 and S2)+(distance between S2 and S3)+(distance between S1 and S3)]/3 Note: binned signifies that continuous values of some variable have been sorted into bins (discrete varirables) to make your analysis easier. For example, consider the variable described as mean P-Study items distance (binned): For that variable, we calculated the values of the actual, continuous variable, and then sorted the values into four bins, or categories, ranging from the smallest values (bin 1) to the largest values (bin 4). Note moreover, that the variable nearest non-matching Study item (binned) is a binned, discrete version of the continuous variable Pnearest Lure distance. Assignment. Your lab report, 2000-2500 words plus graph(s), references, and table(s), is due no later than 5 pm on Friday, December 6. The report, which should submitted in electronic form via the course WebCT site, can be in APA style, or in the style of an article in Nature or Science. Please indicate at the top of the submission which format you have chosen. Feel free to do additional analyses, above and beyond those required to address the three main hypotheses. There are several other interesting analyses that can be carried out with these data. To take just one example, you might see whether false positives ( yes responses on Lure trials) were governed by all the items present in the Study set, or only by that one Study item that was most similar to the Lure. This kind of comparison, and others like it, helps to define the kind information on which recognition
Sensory Processes, Npsy 12a Robert Sekuler rev.c page 6 November 2002 judgments are based. In preparing your lab report, consult any references that you think helpful. Searches on PubMed and/or PsycInfo databases may pay off handsomely, steering you to appropriate sources. And, of course, cite all sources. Performance on the lab report counts for 15% of the course grade. Matrix of face stimuli in this experiment