Exploring the Sequential Lineup Advantage Using WITNESS

Transcription

1 DOI /s ORIGINAL ARTICLE Exploring the Lineup Advantage Using WITNESS Charles A. Goodsell Scott D. Gronlund Curt A. Carlson Ó American Psychology-Law Society/Division 41 of the American Psychological Association 2010 Abstract Advocates claim that the sequential lineup is an improvement over simultaneous lineup procedures, but no formal (quantitatively specified) explanation exists for why it is better. The computational model WITNESS (Clark, Appl Cogn Psychol 17: , 2003) was used to develop theoretical explanations for the sequential lineup advantage. In its current form, WITNESS produced a sequential advantage only by pairing conservative sequential choosing with liberal simultaneous choosing. However, this combination failed to approximate four extant experiments that exhibited large sequential advantages. Two of these experiments became the focus of our efforts because the data were uncontaminated by likely suspect position effects. Decision-based and memory-based modifications to WITNESS approximated the data and produced a sequential advantage. The next step is to evaluate the proposed explanations and modify public policy recommendations accordingly. Keywords Eyewitness identification Lineup procedures lineup advantage Computational models A great deal of research has focused on the factors that enhance the accuracy of eyewitness identification (e.g., Sauer, Brewer, & Weber, 2008; Wells & Olson, 2003). C. A. Goodsell S. D. Gronlund (&) Department of Psychology, University of Oklahoma, 455 West Lindsey, Norman, OK 73019, USA sgronlund@ou.edu C. A. Carlson Texas A&M University Commerce, Commerce, TX, USA Conducting lineups in a sequential manner (presenting lineups members one at a time) has been proposed as one means to enhance accuracy (Lindsay & Wells, 1985; Wells et al., 1998). Two meta-analyses (Clark, Howell, & Davey, 2008; Steblay, Dysart, Fulero, & Lindsay, 2001) showed that a preponderance of the data exhibited a sequential superiority effect when compared to simultaneous lineups (presenting all lineup members at the same time). But McQuiston-Surrett, Malpass, and Tredoux (2006) argued that it would be better if researchers had an explanation for why sequential lineups were better before policy recommendations were made. We echo Clark (2008) and Wells (2008) in their call for the use of formal models to achieve this goal. The use of formal or quantitatively specified models provides several advantages over mere verbal statements about underlying mechanisms (see Bjork, 1973; Hintzman, 1991; Lewandowsky, 1993). These include critically evaluating and enhancing understanding of the existing data, imposing rigor on reasoning, making assumptions explicit, requiring precise operationalization of constructs and explanations, evaluating the validity and completeness of those explanations, and enhancing the ability to generate novel predictions. The goal of this research was to view an array of simultaneous and sequential lineup data through the lens of the WITNESS (Clark, 2003) model to see what we could learn about explicating the sequential lineup advantage. Clark developed the WITNESS model as the first formal model expressly designed for lineup decision-making tasks. As we shall see, sometimes WITNESS approximated the data without modification. However, sometimes it did not. The question then became how we could modify WITNESS to bring it in line with these data. The inability of WITNESS to match some of the data does not disprove the model. As Clark (2003, p. 630) stated, the model will

2 be shown to be inadequate at some point. Rather, our goal in using WITNESS was to learn about the data. Growing out of our attempts to modify WITNESS to approximate the data will be hypotheses about what produces a sequential advantage. These hypotheses can then be subjected to empirical test, which will enhance our understanding of the sequential advantage. As we shall see, WITNESS lives up to the promise of teaching us despite being wrong (Box, 1979). The remainder of this article is organized as follows. We begin with a description of the WITNESS model. This is followed by an exploration of how the model s performance is influenced by manipulation of its parameters. Next, we describe the data to which we applied WITNESS: the eight direct sequential-simultaneous comparison studies considered by Clark et al. (2008) plus two experiments from our lab (Carlson, Gronlund, & Clark, 2008). Six of these ten experiments were well summarized by WITNESS as originally conceived by Clark (2003). These were the experiments that showed small to nonexistent sequential advantages or a particular type of sequential advantage that paired liberal simultaneous choosing with conservative sequential choosing. We focused our theoretical efforts on two of the remaining four experiments. The suspect always was placed in the same position (the eighth) in those two key experiments, which was ideal for model development because the data were not contaminated by likely suspect position effects. Smith and Batchelder (2009) argued that if there were individual differences across (participants and) items (i.e., different positioning of the suspect), then applying a model to data pooled over items might be misleading. To approximate the data from the two key experiments, we developed decision and memory modifications to determine how the model (and by analogy, a witness) behaved in a sequential lineup. We follow that with a discussion of what this exercise taught us about the factors that produce a sequential lineup advantage and directions for future research on this important issue. WITNESS Model WITNESS is a direct-access memory model (see Clark & Gronlund, 1996). It is ideal for applying to lineup decisionmaking tasks because its parameters tie directly to key components of the eyewitness task. Moreover, at its core, WITNESS shares many characteristics of signal detection applications to recognition memory (Banks, 1970). Therefore, it is likely that what we learn about simultaneous and sequential lineup decision making by applying WITNESS to the data would be similar to what we would learn had we developed an alternative model. That is, we believe that if we developed modified versions of other memory models (e.g., SAM: Gillund & Shiffrin, 1984; MINERVA 2: Hintzman, 1988; REM: Shiffrin & Steyvers, 1997), then we would have uncovered comparable ideas about how witnesses make lineup decisions. In WITNESS (Clark, 2003), faces are represented as vectors of features. These are abstract features and do not represent specific features such as the shape of the nose or the color of the eyes. Most memory models make similar representational assumptions. The degree of overlap between the actual perpetrator and the witness memory for the perpetrator is governed by the parameter c, which specifies whether each feature of the vector representing the perpetrator is encoded correctly into memory (with probability c) or replaced with a random feature (with probability 1 - c). The result is that memory for the perpetrator is an approximation of the actual perpetrator; the better the encoding (the higher the value of c) the more the memory for the perpetrator resembles the actual perpetrator. We set the vector length to 100 (as did Clark, 2003). According to WITNESS, the guilty suspect (the target) is placed into the target present (TP) lineup and is assumed to be a perfect encoding of the perpetrator. This assumption can be relaxed to reflect a change of appearance on the part of the perpetrator between committing the crime and inclusion in the lineup, but we had no need to add that flexibility. The innocent suspect is placed into the target absent (TA) lineup and parameter SSP (a probability, SSP denotes the Similarity between the Suspect and the Perpetrator) governs the similarity between the innocent suspect and the perpetrator. The closer the SSP is to 1.0 the more the innocent suspect resembles the perpetrator. The similarity of the remaining lineup foils to the suspect(s) can take one of two forms (see Clark & Tunnicliff, 2001). The foils either can be matched to the description of the perpetrator or to a likeness of the suspect. In the description-matched case, the same foils are used in both the TP or TA lineups (but see, Cybenko & Clark, 2009). The only difference is that the guilty suspect is removed from the TP lineup and replaced with the innocent suspect to create the TA lineup. The quality of the foils in a description-matched lineup is governed by parameter SFP (a probability, SFP denotes the Similarity between a Foil and the Perpetrator). However, in a suspect-matched lineup, different foils are selected for the TP and the TA lineups because the foils are matched against different suspects: the guilty suspect to create foils for a TP lineup, and the innocent suspect to create foils for the TA lineup. In a suspect-matched lineup, the selection of foils is governed by the parameter SFS (a probability, SFS denotes the Similarity between a Foil and a Suspect guilty or innocent). None of the experiments we modeled in this article utilized a match-to-suspect design; thus, this parameter was not needed and will not be mentioned further.

3 Now that we have described how lineups are represented, we need to specify how retrieval takes place. Retrieval results in the production of a value that represents the extent to which a lineup member matches the approximation of the perpetrator stored in memory. The degree of match between each lineup member and the perpetrator s approximation is based on the dot product of these two vectors. A large match value (dot product) signals strong overlap between a lineup member and memory for the perpetrator; a small match value signals that the lineup member bears little resemblance to memory for the perpetrator. Once a match value(s) is determined, it must be translated into a decision (make a choice or reject the lineup). In a simultaneous lineup, the model finds the lineup member that corresponds to the best match (BEST) and the lineup member that corresponds to the next-best match (NEXT) to the perpetrator. The former represents the contribution of an absolute judgment process, and its contribution to the decision is weighted by wa. A relative judgment contribution is based on the difference between BEST and NEXT; its contribution to the decision is weighted by wr (where wa? wr = 1). The lineup member corresponding to BEST is chosen when [wa * BEST? wr * (BEST - NEXT)] exceeds the decision criterion, CSIM. Otherwise, a lineup rejection is recorded. The sequential lineup decision process is simpler. Lineup members are tested one at a time, and the model chooses the first lineup member whose match value exceeds a criterion (CSEQ). If no match value exceeds CSEQ, then the model records a lineup rejection. In all versions of WITNESS that we consider, the model evaluated no additional lineup members once a match value had exceeded CSEQ. Applying WITNESS to the Data WITNESS provides an explanation for the results of an experiment if the model s parameters can be adjusted to approximate the data. If a satisfactory approximation is achieved, then the workings of the model plus the estimated parameter values provide insight into how participants might solve this same memory problem. If a satisfactory approximation is not achieved, then the model must be modified to bring it in line with the data. Throughout our modeling exercises we operationalized satisfactory approximation by maximizing r 2 as an indication of trend relative magnitude and minimizing root mean squared deviation (RMSD), which is the square root of the squared deviation between WITNESS and the data (see Schunn & Wallach, 2005). We strove for r 2 values above.94 and RMSD values approaching 0.1. To describe this parameter adjustment process, we next describe two hypothetical research studies, both of which use description-matched lineups: Study A includes a crime video that affords a good view of the perpetrator, and Study B includes a crime video that affords a poor view of the perpetrator. A larger c value in Study A would capture the superior encoding of the perpetrator (e.g., c =.35 vs..15). Next, assume that Study A includes an innocent suspect who closely resembles the perpetrator, and the innocent suspect is placed into a biased lineup (i.e., the foils do not resemble the perpetrator). In contrast, Study B includes an innocent suspect who does not resemble the perpetrator, and he or she is placed into a fair lineup (the foils match the perpetrator). To fit the data from Study A, a relatively high value of SSP (e.g.,.7, meaning that the innocent suspect and the perpetrator share 70% of their features) and a relatively low SFP value (e.g., SFP =.1, indicating that the foils are poor matches to the perpetrator) would be appropriate. In contrast, for Study B, a smaller value of SSP (e.g.,.4) and a relatively larger value of SFP (e.g.,.3) would be suitable. Finally, suppose that in Study A the instructions induce a liberal response criterion placement whereas Study B induces a more conservative criterion placement. The values of CSIM and CSEQ would be adjusted to match the respective choosing rates. Decision Weights in Description-Matched Lineups One parameter adjustment we did not make for the simultaneous lineup data involves the decision rules governed by wa and wr. As we will demonstrate, for description-matched lineups (same foils in TP and TA lineups), these parameters had no appreciable impact on performance. Note that in situations where different foils are used in TP and TA lineups, as is the case for a suspectmatched lineup, the foil distributions differ between TP and TA lineups and wa and wr have a differential effect (see Breneman & Clark, 2008). In description-matched lineups, we could reproduce the same predicted values given any value of wa by adjusting only the decision criterion, CSIM (recall that wa? wr = 1). For instance, we fit WITNESS to the results of the meta-analysis in Clark et al. (2008, Table 5). The fit with wa =.5 was excellent (RMSD =.10, r 2 =.98) using parameter values: c =.22, SSP =.5, SFP =.33, CSEQ =.079, and CSIM =.038. However, an equivalent fit was achieved for a completely relative decision rule (wa = 0) and a completely absolute decision rule (wa = 1) by holding the memory parameters constant and adjusting only the value of CSIM (if wa = 0: CSIM =.061, RMSD =.11, r 2 =.97; if wa = 1: CSIM =.01, RMSD =.12, r 2 =.96). To further explore the impact of wa and wr on performance, we took the absolute (wa = 1), relative (wa = 0), and 50/50 (wa =.5) decision rules and swept CSIM across

4 a range that would yield from 0% to 100% choosing rates. We did this for two different sets of parameters: one matched to the Clark et al. (2008) meta-analysis and another matched to Carlson et al. (2008, Experiment 1). Figure 1 plots the correct identification rate (choosing the guilty suspect from a TP lineup) versus the false identification rate (choosing the innocent suspect from a TA lineup) in a manner analogous to an ROC function. The curves do not reach the upper right corner of the graph because only suspect choices are plotted; as the criterion decreases foils can be chosen instead of the suspect. The three ROC functions for the Clark et al. (2008) data overlap, as do the three ROC functions for the Carlson et al. s (2008) data. This indicates that the ability to discriminate guilty from innocent suspects does not differ across the three decision rules. Consequently, the decision rules do not differentially affect the ability to discriminate targets (guilty suspects) from distractors (innocent suspects and foils) in description-matched lineups (see also Breneman & Clark, 2008) and adjusting the decision weights will not improve the ability of the model to fit the data. Because the data sets we considered used a same-foils or description-matched design, we set wa and wr equal to.5 for the remainder of this modeling exercise. We did develop an alternative procedure for varying relative versus absolute contributions in a simultaneous lineup: A witness who adopts a relative judgment strategy does not probe memory as thoroughly as one who adopts an absolute judgment strategy. In WITNESS, this could be Fig. 1 Correct identifications versus false identifications for relative (wa = 0), absolute (wa = 1.0), and mixed (wa = wr =.5) decision rules in simultaneous lineups as criterion position varied from conservative to liberal achieved by shortening the vector length used to probe memory when making relative judgments. Although we will not discuss this modification further, it did inspire the memory modifications described below. Theory Space Before we attempted to fit the model to the data, we varied WITNESS across a wide expanse of the parameter space. As a result of this exploration, we developed a sense of its capabilities and deficiencies regarding the production of sequential advantages. We constructed a 3 (quality of encoding) 9 2 (quality of innocent suspect) 9 3 (level of lineup fairness) 9 3 (choosing rate) factorial exploration of the model s predictions for simultaneous and sequential TP (target present) and TA (target absent) lineups. For quality of encoding, poor, moderate, and excellent memory were represented by c =.1,.25, and.45. For quality of the innocent suspect, SSP =.4 and.8 represented a moderate and good match, respectively. Lineup fairness used a ratio of SSP to SFP of 1/1 for fair, 1/3 for intermediate, and 1/9 for biased. Finally, we adjusted the decision criteria (CSIM and CSEQ) to yield three different choosing rates. We did this by computing the average choosing rate for the TP and TA lineups and then adjusting the decision criterion so that choosing was among 30 35%, %, and 55 60%, representing low, medium, and high choosing. One thousand simulations were conducted per factorial combination. The guilty or innocent suspect was placed in a random position in the sequential lineup for each simulation. The model treated all suspect positions in the simultaneous lineup identically. We held the similarity of encoding, innocent suspect quality, and lineup fairness constant when comparing sequential and simultaneous performance because nothing could be learned from comparisons that confounded these factors with lineup type. For example, it would be meaningless to let the perpetrator be well encoded when a simultaneous lineup was tested (e.g., c =.45) but let that same perpetrator be poorly encoded when a sequential lineup was tested (e.g., c =.1). Such an experiment would exhibit poor research design fundamentals. However, the choosing rates (i.e., criterion placement) plausibly could vary as a function of lineup type. Therefore, we organized our discussion around two types of comparisons. First, we compared simultaneous to sequential performance when the choosing rates across a sequential and a simultaneous lineup were comparable (e.g., low versus low, high versus high). Second, we compared simultaneous to sequential performance when the choosing rates varied (e.g., simultaneous high to sequential low, simultaneous low to sequential medium).

5 We assessed performance by computing the probative value (PV, see Clark & Wells, 2008), which is the TP correct identification (ID) rate divided by the sum of the TP correct ID rate plus the TA false ID rate. The use of PV as a performance index puts the focus on suspect identifications because the suspect is the only unknown individual in the lineup; the foils are known to be innocent. PV approaches 1.0 as a greater proportion of suspect identifications are of the guilty suspect. To determine if there was a significant simultaneous or sequential advantage, we computed the difference between the sequential and simultaneous PVs and compared this difference to a value of.08 (sequential PV - simultaneous PV [.08 = sequential advantage; sequential PV - simultaneous PV \ -.08 = simultaneous advantage). A value of.08 was chosen because it was the smallest significant difference Gronlund, Carlson, Dailey, and Goodsell (2009) observed in their empirical exploration of the simultaneous-sequential study space. Equal Choosing Rates Of the 54 simultaneous-sequential comparisons in which the choosing rates were equal, there were no sequential advantages. There were, however, five simultaneous advantages (c =.1, SSP =.4, fairness = 1/1, high choosing; c =.25, SSP =.8, fairness = 1/1, low, medium, and high choosing; and c =.45, SSP =.8, fairness = 1/1, high choosing). These simultaneous advantages resulted because WITNESS always identifies the best match in the simultaneous lineup. This best match usually is the guilty suspect in the TP lineups, which elevates the correct ID rate. In the sequential lineup, however, the model would not necessarily get to the guilty suspect. If the guilty suspect was late in the sequential lineup, the model might choose a foil before getting to the guilty suspect, especially given that all five of these simultaneous advantages involved fair lineups (i.e., good foils). The same thing happens in the TA lineups. Because the lineups are fair, the innocent suspect in the simultaneous lineup was the best match only one-sixth of the time and, thus, many of the lineup choices were foil IDs. However, in the sequential lineup, the innocent suspect is selected more often because foils that happen to be a better match sometimes occur after the innocent suspect and, therefore, do not compete with the innocent suspect. For example, take a situation where a foil generates a better match value than does the innocent suspect and both match values are above criterion. That foil is the BEST match in the simultaneous lineup and gets chosen (resulting in a foil ID). However, in the sequential lineup, if the innocent suspect is presented earlier than the good foil, then the innocent suspect gets chosen (resulting in a false ID). Consistent with this explanation, in all five of the simultaneous advantages, the sequential TP lineup had a greater number of foil choices and a lower correct ID rate. Moreover, the TA false ID rate was greater for the sequential lineup along with a lower foil ID rate. This was how WITNESS produced a simultaneous advantage when the simultaneous and sequential choosing rates were equal. Unequal Choosing Rates Of the 108 comparisons with unequal choosing rates, there were 18 simultaneous advantages and eight sequential advantages. We begin with the sequential advantages and how WITNESS produced them. The factor that consistently was related to the production of a sequential advantage was the pairing of stringent sequential choosing with less stringent simultaneous choosing. This characterized all of the eight sequential advantages. In other words, the sequential advantage produced by WITNESS was the product of a conservative criterion shift for sequential lineups. Meissner, Tredoux, Parker, and MacLin (2005) found empirical support for this explanation. However, despite finding more conservative sequential choosing throughout their experiment, Gronlund et al. (2009) showed that sequential advantages occurred for other reasons (in particular, placing the suspect late in the sequential lineup). Incidentally, 16 of the 18 simultaneous advantages occurred when more stringent simultaneous choosing was paired with less stringent sequential choosing. In sum, the primary factor that allowed WITNESS to produce sequential (and simultaneous) advantages depended on appropriate shifts in the respective choosing rates. Armed with this knowledge, we were ready to apply WITNESS to enhance our understanding of the sequential advantage. However, before we discuss these results, we will review the relevant data. Application of WITNESS to the Data and Data We considered the same data which Clark et al. (2008) designated as direct-comparison studies. Their inclusion criteria consisted of the set of experiments that covaried simultaneous and sequential lineup presentation with target presence and target absence within the same study. The eight data sets were all published, had only one suspect per lineup, and used adult witnesses. The studies were by Kneller, Memon, and Stevenage (2001), Lindsay, Lea, and Fulford (1991a), Lindsay et al. (1991b), Lindsay and Wells (1985), MacLin, Zimmerman, and Malpass (2005), Melara, DeWitt-Rickards, and O Brian (1989), Parker and Ryan

6 (1993), and Sporer (1993). We also considered data from our own lab that met the aforementioned criteria (Carlson et al., 2008, Experiments 1 and 2 biased 1 ). As reported by Clark et al. (2008), the eight direct-comparisons studies showed that the PV of suspect identifications, on average, was greater for sequential lineups than for simultaneous lineups. However, researchers do not always operationalize sequential lineup advantages in this way. For example, MacLin et al. (2005) and Sporer (1993) found a sequential lineup advantage by separately analyzing correct IDs (sequential was lower, but not significantly so) and false IDs (sequential was significantly lower). PV considers both factors together, which is why we found no sequential advantage where the original researchers reported one. To fit WITNESS to the data, we need to consider the experimental design of each study. This included lineup size (six or eight), suspect position (counterbalanced across multiple positions, placed in a specific position), and foil design (same foils in TP and TA). Lindsay et al. (1991b) used different foils in simultaneous and sequential lineups, but using different values of the SFP parameter for simultaneous and sequential did not improve the ability of the model to fit the Lindsay et al. (1991b) data. In addition, Carlson et al. (2008, Experiment 2) used different foils across TP and TA lineups, but varying SFS parameter did not improve the ability of the model to fit the data. We began by fitting WITNESS to each of the aforementioned 10 experiments. Table 1 presents the data and model comparisons for the experiments ordered from the largest sequential advantage to the largest simultaneous advantage. Included are the minimized RMSD and maximized r 2 to represent each fit. However, because we did not optimize a fitting algorithm to find the best fit, what Table 1 provides is the best fit that we found as a result of our exploration of the parameter space (which was informed by the theory space exploration and our own experience working with the model). Table 2 provides the corresponding parameter values for each model fit. Table 3 highlights the degree to which some of the data are mispredicted. It reports the observed and predicted PV difference (e.g., sequential PV minus simultaneous PV) and correct identification (ID) difference values (sequential correct ID minus simultaneous correct ID). Positive PV difference values signal a sequential advantage. For the subsequent discussion, we divide the ten experiments into two subsets based on whether there was a sequential advantage greater than a PV difference of.08 (the top five experiments) or not. We chose.08 to match the theory 1 Experiment 2 in Carlson et al. (2008) also had intermediate and fair lineup conditions. However, because our focus is on the sequential advantage, and neither of those conditions exhibited a sequential advantage, we excluded them. Table 1 Data, model predictions, and fit statistics for the 10 experiments Data Model Fit TP TA TP TA RMSD r 2 Lindsay and Wells (1991b) Suspect Foil No ID Suspect Foil No ID Lindsay and Wells (1985) Suspect Foil No ID Suspect Foil No ID Lindsay et al. (1991a) Suspect Foil No ID Suspect Foil No ID Melara et al. (1989) Suspect Foil No ID Suspect Foil No ID Kneller et al. (2001) Suspect Foil No ID Suspect Foil No ID

7 Table 1 continued Table 2 Parameter values for the 10 experiments Data Model Fit TP TA TP TA RMSD r 2 Carlson et al. (2008, Exp. 2 biased) Suspect Foil No ID Suspect Foil No ID MacLin et al. (2005) Suspect Foil No ID Suspect Foil No ID Sporer (1993) Suspect Foil No ID Suspect Foil No ID Carlson et al. (2008, Exp. 1) Suspect Foil No ID Suspect Foil No ID Parker and Ryan (1993) Suspect Foil No ID Suspect Foil No ID Note. Data sets are ordered by size of sequential advantage. Parameter values are presented in Table 2 Study c SSP SFP CSIM CSEQ Lindsay et al. (1991a) Lindsay and Wells (1985) Lindsay et al. (1991a) Melara et al. (1989) a Kneller et al. (2001) Carlson et al. (2008, Exp. 2) MacLin et al. (2005) a Sporer (1993) Carlson et al. (2008, Exp. 1) Parker and Ryan (1993) a These studies did not include a designated innocent suspect space exploration as well as the data from Gronlund et al. (2009). We deal with these two sets of experiments separately, beginning with those with no sequential advantage as assessed by the PV difference (the bottom five in Table 1). Experiments that Found No Advantage Parker and Ryan (1993) showed the only simultaneous advantage, but the sample size was very small (n = 12 per condition), which raises concerns about the robustness of these findings. Gronlund et al. (2009) found evidence for simultaneous advantages using sample sizes four times as large, but these advantages were the result of the suspect being placed early in the sequential lineup, the impact of which we will describe in more detail later. Therefore, we did not make modifications to WITNESS that would produce a simultaneous advantage for two reasons: (1) a larger corpus of simultaneous advantage data is needed before applying WITNESS to the vagaries of the simultaneous advantage and (2) the simultaneous advantage may be a sequential effect. That is, early suspect placement in the sequential lineup results in poorer sequential performance relative to simultaneous, and later suspect placement in the sequential lineup results in better sequential performance relative to simultaneous (Carlson et al., 2008; Gronlund et al., 2009). The remaining four experiments in this group showed small to nonexistent sequential advantages and all were well approximated by WITNESS. We based this judgment on two criteria. First, the fits were very good, yielding RMSD values less than.14 and r 2 values greater than.94. Second, Table 3 shows that the model approximated the PV differences (and the correct ID differences) observed in the data. In sum, the data did not show a sequential advantage and neither did the model. Fitting this range of data was an impressive achievement for WITNESS. As the parameter values in Table 2 reveal,

8 Table 3 Experiments ordered by size of sequential advantage PV difference a Correct ID difference b Data WITNESS Data WITNESS Lindsay et al. (1991b) Lindsay and Wells (1985) Lindsay et al. (1991a) Melara et al. (1989) Kneller et al. (2001) Carlson et al. (2008, Exp. 2 biased) MacLin et al. (2005) Sporer (1993) Carlson et al. (2008, Exp. 1) Parker and Ryan (1993) a Probative value (PV) of sequential lineup minus PV of simultaneous lineup. Positive values indicate a sequential advantage b lineup correct identification (ID) rate minus simultaneous lineup correct ID rate these experiments covered a range of lineup bias, encoding quality, and choosing rates. By examining the SSP versus SFP values, it is clear that lineups ranged from fair, in the case of experiments that did not include a designated innocent suspect (MacLin et al., 2005) or had a similar SSP to SFP value (Carlson et al., 2008, Experiment 1; Sporer, 1993), to biased (Carlson et al., 2008, Experiment 2). It also was evident that the encoding strength of the perpetrator varied widely (ranging from c =.18 to c =.33), as did the choosing rates (ranging from 79% to 16%). However, there was little to learn about what causes a sequential advantage from the application of WITNESS to a set of experiments with no sizeable sequential advantages. We will learn more about the factors responsible for the sequential advantage from the inability of WITNESS to produce a sequential advantage of sufficient magnitude for the remaining experiments. Experiments that Found a Advantage The top half of Table 3 indicates that WITNESS overpredicted the difference between the simultaneous and sequential correct IDs while under-predicting the magnitude of the sequential advantage (i.e., the PV difference). We were unable to adjust parameter values to produce a sequential advantage of the magnitude seen in the Kneller et al. (2001), Lindsay et al. (1991a, 1991b), and Lindsay and Wells (1985) data. However, WITNESS did approximate the magnitude of the sequential advantage in the Melara et al. (1989) data. How did the model do so? In the Melara et al. (1989) data, a sequential advantage resulted because of the drastically different choosing rates in the simultaneous and sequential lineups. [Note that the cell size (n = 8) was small in this study as well]. The choosing rate for the simultaneous lineup averaged 94% while the choosing rate for the sequential lineup averaged 31%. We know from the theory space exploration that WITNESS s parameters can produce a sequential advantage of this sort, and it did. Table 3 shows that the PV difference produced for the Melara et al. (1989) data (.102) comes closer to the magnitude seen in the data (.149) than is the case for the remaining four sequential advantage experiments. Table 2 reveals that this was accomplished with the use of an unusually low value of CSIM (i.e., high simultaneous choosing). CSIM generally was about half that of CSEQ for the remaining four experiments. This alone does not mean that participants were more conservative for the sequential lineup as the match values are on different scales unless wa equals 1.0 (completely absolute judgment process). However, when we increased the value of CSIM to make the simultaneous choosing rates in Melara et al. (1989) comparable to these other experiments, the size of the sequential advantage waned. For example, with CSIM equal to.035 and all the other parameters the same as the Melara et al. (1989) fit, the size of the sequential advantage dropped from a PV difference of.102 to.035 (not a sequential advantage, by our criterion). Based on our theoretical analysis, Melara et al. (1989) would not have found a sequential advantage if their participants placed their simultaneous criterion where participants in the other experiments placed theirs. We next turn to the remaining four experiments that showed a sequential advantage. As noted in the theory space exploration, only the coupling of conservative sequential choosing with less conservative simultaneous choosing could produce a sequential lineup advantage. However, these parameter adjustments were insufficient for these four experiments. The first observation we made as we began fitting WITNESS to these four experiments was that the simultaneous TP and TA data were easy to fit.

9 There appears to be something fundamentally correct about what WITNESS assumes regarding how participant-witnesses make simultaneous lineup decisions, probably reflecting the model s signal detection foundations. The problem for the model invariably was fitting the sequential TP and TA data concurrently and thereby producing a sequential advantage of sufficient magnitude. Because of the ability of WITNESS to match the simultaneous data, and its failure to produce a sequential advantage, the modifications we explored involved changes as to how the model functioned in a sequential lineup. In other words, WITNESS is at a disadvantage because it lacks processes or strategies that a witness can bring to bear in a sequential lineup; participants do more than simply select the first match value above criterion. With that in mind, we explored two different approaches to give WITNESS the flexibility to produce a sequential advantage. One approach involved decision modifications, and the other involved memory modifications. We began with the Lindsay et al. (1991a, 1991b) studies because both placed the suspect into only one position (the eighth position). Lindsay and Wells (1985) and Kneller et al. (2001) placed the suspect in various positions but reported the data averaged across position because they detected no position effects. However, in light of our recent work (Carlson et al., 2008; Gronlund et al., 2009; see also Clark & Davey, 2005), suspect position in a sequential lineup can make a difference, and Lindsay and Wells and Kneller et al. (2001) might have lacked the statistical power to detect these effects. Suspect position effects (if present) would produce item differences that when collapsed over produce pooled data unlike the individual subject data (Smith & Batchelder, 2009). Instead, a model would need to be augmented with a random effects component to capture the position effects. Alternatively, a model could be applied to the data separated by the different suspect positions. But the Lindsay and Wells and Kneller et al. (2001) data were not reported that way. That means that the two experiments by Lindsay and colleagues provide the least contaminated data and, therefore, the best starting point for our theoretical modifications. Both the Lindsay et al. (1991a, 1991b) experiments involved eight-person lineups and a designated innocent suspect. Different foils were used in the simultaneous and sequential lineups in Lindsay et al. (1991b), but as stated above, an additional parameter accounting for the different foils did not improve the fit. Lindsay et al. (1991a) stated that the suspect was placed in position 8 in the sequential lineup. This was not stated in Lindsay et al. (1991b), but we made the same assumption given that the two articles were published at the same time. As is apparent from Table 1, WITNESS provided an especially poor approximation to Lindsay et al. (1991b). These two experiments had two of the three largest sequential advantages, which Table 3 shows that WITNESS did not approximate. Both of these studies placed the suspect in the eighth position and showed a large sequential advantage. Supporting this finding, Carlson et al. (2008, their Fig. 3) found a sequential advantage only when a suspect (guilty or innocent) was placed late in the lineup (positions 5 and 6). This was counter to what the WITNESS model could produce. If a suspect was placed late in a sequential lineup, the parameters needed to be set up to reject all foils before the guilty suspect in the TP lineup and choose a foil or reject the innocent suspect in the TA lineup. A unique combination of parameter values would be necessary to accomplish this. First, the decision criterion (CSEQ) must be below the match value for the guilty suspect; second, CSEQ must be above the match value for the innocent suspect; and third, the match values of the foils must fall below those of the innocent suspect. Only then would the model get to the eighth position frequently enough. As can be seen in Table 1, we could not find a combination of parameter values that allowed the model to do this; the foil ID rate was too high and the correct ID rate was too low. Therefore, we explored decision modifications that might capture how a participant-witness might behave as a sequential lineup unfolds. Decision modifications. The rejection of the first few foils might cause a witness to shift from an absolute to a relative decision rule. In other words, a witness might begin comparing subsequent lineup members to an estimate of what they have viewed so far. Consequently, any lineup member that was considerably better than what had been viewed previously may induce a pop out effect (Ross, Benton, McDonnell, Metzger, & Silver, 2007), which could, in turn, produce a decrease in criterion. We implemented the shift to a relative decision rule if the model rejected the first four lineup members. The magnitude of the sequential advantage was insufficient if this shift was made prior to the fourth lineup member. Following the fourth lineup member, the average match value of the preceding lineup members was compared to the match value of the current lineup member. If the difference between the average and current match values was greater than the parameter popout, then the model would decrease the criterion. The amount of criterion decrease was a function of the magnitude of the difference between the average and current match values. That was because the criterion (usually) needed to decrease more in the TP lineup (CSEQ TP ) than in the TA lineup (CSEQ TA )to make the guilty suspect more likely to be chosen than the innocent suspect. For the fit to the Lindsay et al. (1991a) data, the average match value for the guilty suspect was twice as great as the

10 average match value for the innocent suspect; thus, when popout was exceeded, we let the criterion decrease half as far in the TA lineup. As can be seen in Table 4, this modification resulted in a good fit to the Lindsay et al. (1991a) data (RMSD =.16, r 2 =.96, c=.25, SSP =.57, SFS =.31, CSIM =.041, popout =.060, CSEQ =.13, CSEQ TA =.1, and CSEQ TP =.5 * CSEQ TA ). Most importantly, this model produced a PV difference of.12. For the Lindsay et al. (1991b) data, the average match value of the guilty suspect again was twice the average match value of the innocent suspect. Therefore, again the criterion decreased half as far in the TA lineup if popout was exceeded. As seen in Table 4, this modified model fit the Lindsay et al. (1991b) data (RMSD =.16, r 2 =.98, c=.30, SSP =.52, SFS =.15, CSIM =.055, popout =.065, CSEQ =.15, CSEQ TA =.12, and CSEQ TP =.5 * CSEQ TA ) and produced a large sequential advantage with a PV difference of.18. There undoubtedly are other functions that could govern how the criterion could change as a sequential lineup unfolds that would produce a similar result as the shift to a relative decision rule. In fact, we constructed an alternative model with a criterion value that decreased continuously throughout the sequential lineup according to a power function (CSEQ *[p (i-1) ], where p governed how sharply CSEQ decreased as the lineup members progressed from i = 1 to 8). This modification also fit the data and produced a sequential advantage if we allowed a different rate of Table 4 Model fits for decision modification Data Model Fit TP TA TP TA RMSD r 2 Lindsay et al. (1991b) Suspect Foil No ID Suspect Foil No ID Lindsay et al. (1991a) Suspect Foil No ID Suspect Foil No ID Note. Parameter values presented in text decrease for sequential TP and TA lineups. It would be preferable if different parameter values were not necessary in the sequential TP versus the sequential TA lineups; a participant-witness cannot make this adjustment because he or she cannot know in advance whether a lineup contains a guilty or an innocent suspect. The fact that we had to make different adjustments could signal a shortcoming of the proposed modifications, or it could signal that there were factors that differed between the TP and TA lineups that participants differentially responded to that researchers were not aware of and therefore did not control (like the position of a next-best foil). We return to this issue in the General Discussion. Memory modifications. Instead of a witness changing strategy and adjusting a decision criterion, a witness s memory probe could be improved as a sequential lineup unfolds. For example, a witness might view lineup member #1 and note that the ears matched their memory but the nose was too big. After viewing #2, a witness might get a better sense of what the nose looked like, and so on. Alternatively, a better understanding of the culprit s configural characteristics could be obtained as the witness progressed through the lineup. In either case, as the lineup unfolds, a better and more complete memory probe could be constructed in the sequential lineup. The same course of action would be possible in a simultaneous lineup, but perhaps it does not occur to a witness to try, or perhaps making comparisons among faces exhausts the requisite cognitive resources. We developed two memory modifications that allowed WITNESS to produce a sequential advantage by enhancing the quality of the memory probe as the sequential lineup unfolds. For both memory modifications, the model selected the first lineup member whose match value was above CSEQ (i.e., the criterion did not change during the lineup). The first modification increased the quality of the memory probe by gradually increasing how much of it was utilized as the model mimicked a real witness s progress through the sequential lineup. Both the simultaneous and the sequential lineups started with a memory probe consisting of a 50-element vector (though we could have started with 100-element vectors for both simultaneous and sequential and increased the sequential probe from there). As the model simulated progress through the sequential lineup, the memory probe was enhanced through the addition of seven elements (increasing the vector length utilized) after each lineup member was rejected. This Additive Cue modification provided a good summary of the two experiments by Lindsay and colleagues. As evident from Table 5, the model fit the Lindsay et al. (1991a) data (RMSD =.12, r 2 =.97) and produced a comparable sequential advantage, PV difference =.17. The model also

11 Table 5 Model fits for Additive Cue memory modification Data Model Fit TP TA TP TA RMSD r 2 Lindsay et al. (1991b) Suspect Foil No ID Suspect Foil No ID Lindsay et al. (1991a) Suspect Foil No ID Suspect Foil No ID Note. Parameter values are presented in Table 7 Table 6 Model fits for Better Cue memory modification Data Model Fit TP TA TP TA RMSD r 2 Lindsay et al. (1991b) Lindsay et al. (1991a) Suspect Foil No ID Suspect Foil No ID Note. Parameter values are presented in Table 7 fit the Lindsay et al. (1991b) data (RMSD =.19, r 2 =.97) and produced a comparable sequential advantage, PV difference =.15. The parameter values are given in Table 7. A second type of memory modification was inspired by a proposal by (Clark, Marchall, & Rosenthal, 2009; see also Tversky, 1977). Rather than using more of the vector as the sequential lineup unfolds, perhaps a witness always uses the same amount of information, but the quality of the information increases. For example, a witness could replace ineffectual segments of the memory probe with richer segments as he or she hypothesizes which features are diagnostic and which are not. For example, as a witness begins viewing lineup members in a sequential lineup, he or she might realize that the perpetrator had ears like Barack Obama. Or a witness might decide that the nose was too common to be diagnostic. As a result, a witness could enhance aspects of the memory probe related to the ears and discount aspects of the memory probe related to the nose. We implemented this idea in WITNESS by creating two encodings of the perpetrator in memory. Each was governed by the encoding parameter c, for which we estimated two different values, c and c2 (where c \ c2). The entire c-vector governed memory for the perpetrator as the first lineup member was viewed. However, prior to viewing the second lineup member, features 1 to k from the c-vector (which reflected poorer encoding) were replaced by features 1 to k from the c2-vector (which reflected better encoding). Prior to viewing the third lineup member, Table 7 Parameter values for memory modifications c c2 SSP SFP CSIM CSEQ Additive Cue Lindsay et al. (1991b) Lindsay et al. (1991a) Better Cue Lindsay et al. (1991b) Lindsay et al. (1991a) features k? 1to2k from the c-vector were replaced with features k? 1 to 2 k from the c2-vector. This process continued as additional lineup members were viewed. The result was that the degree to which the perpetrator matched memory increased as access to poorly encoded features was replaced with access to better encoded features. As is apparent in Table 6, this Better Cue modification provided an excellent summary of the two experiments by Lindsay and colleagues. The model fit the Lindsay et al. (1991a) data (RMSD =.07, r 2 =.99) and produced a comparable sequential advantage, PV difference =.14. The model also fit the Lindsay et al. (1991b) data (RMSD =.06, r 2 =.99) and produced a comparable sequential advantage, PV difference =.15. The parameter values are given in Table 7. Summary. We have developed several potential explanations for the sequential lineup advantage. Two