Replication of Study 2 from The Intermixed Blocked Effect in Human Perceptual

Transcription

1 Replication of Study 2 from The Intermixed Blocked Effect in Human Perceptual Learning Is Not the Consequence of Trial Spacing by Mitchell, Nash, & Hall (2008). Daniel Lakens Eindhoven University of Technology Author Note: Correspondence can be addressed to Daniël Lakens, Human Technology Interaction Group, IPO 1.33, PO Box 513, 5600MB Eindhoven, The Netherlands. D.Lakens@tue.nl. Competing Interests: The authors declared that they had no competing interest with respect to their authorship or the publication of this article.

2 Replication of Study 2 from The Intermixed Blocked Effect in Human Perceptual Learning Is Not the Consequence of Trial Spacing by Mitchell, Nash, & Hall (2008). Exposure to a visual stimuli allows people to learn things about these stimuli. Simply being exposed to two very similar stimuli can make these stimuli more distinct (Gibson & Walk, 1956). When in a pre-exposure task people are exposed to two very similar stimuli (AX and BX) that share many common elements (X), the features that are unique to each stimulus (A and B) can be differentiated better in a subsequent recognition task. In this type of perceptual learning, it has been shown (e.g., Lavis & Mitchell, 2006) that intermixed pre-exposure (i.e., a trial sequence such as AX BX AX BX, etc.) is much more effective than pre-exposure in which the stimuli are presented in separate blocks of trials (e.g., AX A, etc., followed by BX BX, etc.). This is referred to as the intermixed blocked effect. An earlier study by Lavis and Mitchell (2006) provided support for the intermixed block effect. However, the intermixed and blocked pre-exposure conditions in these studies did not only differ in terms of the opportunity they allowed for participants to compare AX and BX during pre-exposure, they also differ with respect to the temporal spacing of trials. Because in the intermixed pre-exposure each presentation of AX in followed by a presentation of BX, while in the blocked condition each presentation of CX is immediately followed by CX. In two experiments, Mitchell, Scott, & Nash (2008) aimed to control for the effects of spaced practice. In Study 1 there were three phases of pre-exposure. In the intermixed phase, presentations of AX alternated with presentations of BX. In the blocked phase, all presentations of CX were given consecutively, and in the spaced phase all presentations

3 of DX were given sequentially, but a temporal delay was inserted between each trial. In the subsequent test task where participants had to discriminate between stimuli by indicating two stimuli were the same of different performance was better on different trials (where two different stimuli were presented) for stimuli from the intermixed pre-exposure block (AX and BX) but not for stimuli from the blocked pre-exposure (CX) or spaced pre-exposure (DX). Performance on same trials did not differ between stimuli. Although temporal spacing was equivalent in the intermixed and spaced conditions, the interval between DX presentations was empty in the spaced pre-exposure, while the interval between AX in the intermixed pre-exposure contained the presentation of BX (and vice versa). Mitchell and colleagues (2008) reasoned that perhaps the presence of BX between AX presentations increased encoding variability for this cue, and performed Experiment 2 to address this possibility. In Experiment 2, pre-exposure consited either of two similar stimuli (AX and BX or two different stimuli (CX and DY). They found that discrimination performance on the subsequent test trial was better for AX/BX, but not for CX, both on trials where the two presented stimuli were different as when they were the same. They conclude that alternation of trial types is not enough in itself to produce enhancement of discrimination; rather, it is critical that the target cue be presented in alternation with a similar cue (page 240). The current replication study consists of a replication of Experiment 2. Method Participants The final sample consisted of 48 undergraduate students from Eindhoven University of Technology (27 women, mean age 20.5) who received monetary compensation of 5 euro in return for their participation. Four additional participants were run but were replaced before the data was analyzed because they failed to follow instructions during the pre-exposure task

4 (i.e., they did not press the space bar after each trial). Sample size was determined both based on the fact that all 16 counterbalancing conditions could each be presented to three participants, and based on an effect size of dz = The effect size of Cohen s dz was calculated from the raw data that were kindly provided by Dr. Mitchell, (for formula s see Lakens, 2013). Using G*Power, power calculations revealed that with an effect size estimate of dz = 0.51 and a sample size of 48, the experiment would have 93% power for a two-sided test (and 97% power for a one-sided test, which would be correct given the pre-registered nature of the study, and the interest in finding an effect in the same direction as in the original study). With 48 participants, a sensitivity analysis shows we have 80% power for a one-sided test with an effect of dz = Stimuli The stimuli are the same as those used in Mitchell et al. (2008), and were kindly provided by Dr. Mitchell. Two different background stimuli are used (X and Y), with four variations (A, B, C, D), see Appendix 1. Procedure Participants performed the experiment in isolation in individual cubicles and were seated approximately 60 cm from the computer monitor. Instructions were presented on the computer screen (see Appendix 2). The experiments consisted of the pre-exposure task and the test task. There are two phases of pre-exposure. Compounds AX and BX are intermixed in one phase, and CX and DY are intermixed in the other phase (order counterbalanced). Each stimulus was presented 60 times. Across participants, the unique features presented in the stimuli (see Appendix 1) played the role of features A D equally often. Because the method section of the article in which the original finding was described provided no further detail of how the unique features were counterbalanced across participants, this replication study

5 applied a Latin-square design. To be specific, the stimuli were counterbalanced across participants as detailed in Table 1. This results in 16 different counterbalancing conditions. The 48 participants will be distributed equally across the 16 conditions (with three participants in each unique combination). Table 1 Counterbalancing Conditions During Pre-exposure Counterbalance order 1 Counterbalance order 2 Block 1 Block 2 Block 1 Block 2 AX/BX CX/DY CX/DY AX/BX X as background BX/CX DX/AY DX/AY BX/CX CX/DX AX/BY AX/BY CX/DX DX/AX BX/CY BX/CY DX/AX AY/BY CY/DX CY/DX AY/BY Y as background BY/CY DY/AX DY/AX BY/CY CY/DY AY/BX AY/BX CY/DY DY/AY BY/CX BY/CX DY/AY During the pre-exposure phase, each stimulus was displayed 60 times for a duration of 470 ms on each occasion. Each individual stimulus presentation was followed by a blank gray screen during which participants made their space bar presses. In fact, the following trial was initiated after 2,000 ms whether a press was made or not. The area of the screen around the checkerboard was the same gray color as the gray squares of the checkerboard. A thick black border separated the checkerboard from the background. The individual squares within the checkerboard had no borders. The stimuli were presented on a 21 in. (43.18 cm) computer monitor (a 17 inch computer monitor was used in the original study) and were approximately 8-centimeters square. Authorware version 7.02 was used to control stimulus presentation on a computer running Windows 7. Presentations of AX alternated with presentations of BX. There were two phases of pre-exposure. In one phase, pairs AX and BX were intermixed, and CX and DY were intermixed in the second phase. The order of these two

6 phases was counterbalanced across participants (see counterbalancing order 1 and 2 in Table 1). There is no mention of any instructions between the two phases of the pre-exposure. This means that after 60 trials where each pair (e.g., AX/BX) is presented intermixed, 60 trials follow where a second pair is intermixed (e.g., CX/DY) without making this change salient to participants. At the completion of the pre-exposure phase, participants were informed that in a second phase, they would be presented with pairs of checkerboards, one pair at a time. They were told to press the A key if these two stimuli appeared to be the same, and the 5 key on the number pad if the stimuli appeared to be different. A reminder about which keys to press remained on the screen throughout the test period. Participants were also told not to spend too long on each judgment. For detailed instructions, see Appendix 2. The test phase required same different judgments on pairs of stimuli, which were either different, or the same. In the article in which the original finding was presented, the authors note that: In the different test trials, the target stimuli AX, BX, and CX were compared to X alone (pg. 240). No mention is made of target stimulus XD. However, later the authors noted the following: Thus, overall, each unique feature A D was presented in compound both with X and with Y. This produced eight types of different trials in total in the test phase: four in which the stimuli AX DX were compared to X and four in which the stimuli AY DY were compared to Y (pg. 240). This latter description was followed, and eight different trials were created as specified in the original article. Each of these trial types was presented four times making a total of 32 different test trials. Although not specified, we counterbalanced the order of presentation (e.g., first X, then AX, or first AX, then X) such that each order was used twice. There were 10 types of same test trial on which two of each of the following stimuli were presented: AX DX, AY DY, X, and Y. These trial types were presented once each (so

7 there were 10 same trials), making a grand total of 42 trials (see Table 2). The order of stimulus presentation was randomized across test trials. Test trials consisted of the presentation of one stimulus for 800 ms, followed by a blank screen for 550 ms (which was interpreted as meaning a blank grey screen), and the presentation of the second stimulus for 800 ms. A white square was then presented in place of the checkerboard, and it remained on the screen until the response was made. Although not specifically mentioned, all stimuli were presented in the center of the screen. Furthermore, although not specified, we added a 550 ms empty screen between each trial to prevent the next stimulus from appearing immediately after the white square when a key was pressed. Differences from the original study The stimuli were presented on a 21 in. (43.18 cm) computer monitor, whereas a 17 inch computer monitor was used in the original study). Participants were seated in individual cubicles, whereas the there are no details about the location of participants (e.g., whether the experimenter was present in the same room) in the original manuscript. We used a Latin square design to counterbalance conditions during pre-exposure between subjects, but the original article did not specify how conditions were counterbalanced.

8 Table 2 Specification of all 42 Test Trials Trial Type Test Trials # Presentations AX/X 2 X/AX 2 BX/X 2 X/BX 2 CX/X 2 X/CX 2 DX/X 2 Different X/DX 2 AY/Y 2 Y/AY 2 BY/Y 2 Y/BY 2 CY/Y 2 Y/CY 2 DY/Y 2 Y/DY 2 AX/AX 1 BX/BX 1 CX/CX 1 DX/DX 1 Similar X/X 1 AY/AY 1 BY/BY 1 CY/CY 1 DY/DY 1 Y/Y 1 Results Confirmatory results. Correct answers were averaged for trials in which AX (or BX) was presented, or CX was presented (the factor training condition), both for test trials when two different stimuli were presented (e.g., AX/X) and when two of the same stimuli were presented (e.g., AX/AX, the factor trial type), resulting in four performance scores for each individual. As in the original experiment, there were 8 AX/BX different trials, 4 CX different trials, 2 AX/BX same trials, and one CX same trial.

9 The results of the critical test trials, those on which AX (or BX) and CX were presented, are shown in Figure 1. A 2 (test trial: same vs. different) X 2 (training condition: AX/BX vs. AC/DY) repeated measures analyses was used to examine the data. Performance on same test trials was better than it was on different test trials, F(1, 47) = , p <.001, ηp² = 0.79, ηg² = More importantly, unlike the original study, performance on trials in which AX (or BX) was presented did not reliably differ from that on trials in which CX was presented. The main effect of the two types of tests (the statistical equivalent of the contrast comparing performance across the two types of tests reported in the original manuscript, and the main hypothesis we aimed to replicate) was not statistically different from zero, F(1, 47) = 0.87, p =.335, ηp² = 0.02, ηg² = Finally, the interaction between the training conditions (AX/BX and CX/DY) and the test trial type (same and different) was nonsignificant, F(1, 47) = 3.02, p =.089, ηp² = 0.06, ηg² = These results differ from those observed by Mitchell et al. (2008), who observed two main effects of trial type and training condition. 1 For an explanation of the difference between partial eta squared and generalized etasquared effect sizes, see Bakeman, 2005.

10 Figure 1. Mean percentage of correct responses on same and different test trials in Experiment 2 by Mitchell et al. (2008) and the current replication study when stimuli AX, BX, and CX were compared to the common element X. Presentations of AX and BX were intermixed with one another in pre-exposure. Stimulus CX was intermixed with DY in pre-exposure. Error bars indicate standard errors of the mean. Exploratory results. If we explore this interaction in more detail (thus going beyond the confirmatory analyses), we see that the mean performance in the different trials of participants in the AXBX condition was greater (M = 0.35, SD = 0.23) than the mean performance of participants on different trials in the CX condition (M = 0.25, SD = 0.24). The difference between measurements is statistically different from zero, (M = 0.10, SD = 0.33), 95% CI = [0.01, 0.20], t(47) = 2.15, p = 0.037, dav = 0.43, 95% CI [0.03, 0.83]. When directly comparing the effect size in experiment 2 of Mitchell and colleagues (dz = ) with the effect size observed in our data (dz = 0.31, 95% CI [0.02, 0.60] we see that our 95% CI interval of the effect size estimate in the different judgments overlaps with the effect observed by Mitchell et al (2008). There was no effect of training condition (AX/BX and CX/DY) on performance in same trials. The mean performance of participants in the AXBX condition (M = 0.85, SD = 0.25) and the mean performance of participants in the CXDY condition (M = 0.88, SD = 0.33) did not statistically differ, t(47) = 0.33, p = 0.743, dav = 0.07, 95% CI [-0.35, 0.49]. It should be noted that these performance averages are based on either two (AX/BX) or one 2 Cohen s dav and dz differ in using either the averaged standard deviation as a standardizer in the effect size calculation, or the standard deviation of the difference scores (see Lakens, 2013).

11 (CX) judgments. Therefore, these data should be expected more across replications than the data on different trials, which are averages over 8 (AX/BX) or 4 (CX) judgments. Discussion We performed a close replication of Experiment 2 by Mitchell and colleagues (2008) and partially replicated their results. We find a similar main effect of same versus different trials, but no main effect of training condition (AX/BX versus CX). Instead, our data provided a tentative indication of an interaction between trial type and condition, with the benefit of intermixed presentation being present for performance on different trials, but not on same trials. It should be noted that the difference in the data pattern between the original and replication study is completely dependent upon the performance on same CX trials, which consisted only of a single trial. To put this into perspective, if only 5 additional participants in the replication study (out of 48 total participants) would have made an error on the CX same trial, the replication would have yielded the same pattern of results as the original study by Mitchell and colleagues (2008). The performance on the same trials in the replication study mirrors the performance on same trials in Experiment 1 of Mitchell et al. (2008), where there was no difference between the AX/BX condition and the CX condition on same trials. Thus, it remains unclear whether the intermixed-blocked effect is present in same trials. However, the intermixedblocked effect is aimed at examining how people learn to perceptually differentiate visual stimuli, which makes the performance on different trials the most important test of the theory. We can therefore conclude that even though we do not exactly replicate the overall pattern in Mitchell et al. (2008), our data on the different trials provides additional support for the most important theoretical prediction by the intermixed-blocked effect. Our observed effect size estimate is smaller than that in the original study, but confidence intervals in the effect size estimates of the simple effects in the same and different trials overlap. Future studies aimed at

12 examining whether the intermixed-blocked effect influences performance on same trials would benefit from either larger sample sizes and/or additional critical AX/BX and CX same trials.

13 References Gibson, E. J., & Walk, R. D. (1956). The effect of prolonged exposure to visually presented patterns on learning to discriminate them. Journal of Comparative and Physiological Psychology, 49, Lavis, Y., & Mitchell, C. J. (2006). Effects of preexposure on stimulus discrimination: An investigation of the mechanisms responsible for human perceptual learning. Quarterly Journal of Experimental Psychology, 59, Mitchell, C., Nash, S., & Hall, G. (2008). The intermixed-blocked effect in human perceptual learning is not the consequence of trial spacing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34, Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4:863. doi: /fpsyg

14 Appendix 1 Background Stimulus X (center, bottom) and 4 versions with added unique features (top 4, clockwise starting top left A, B, C, D

15 Background Stimulus Y (center, bottom) and 4 versions with added unique features (top 4, clockwise starting top left A, B, C, D)

16 Appendix 2 Instructions Two student assistants translated the instructions from English to Dutch, and two other student assistants translated the instructions back from Dutch to English. The experimenter checked these translations against the original. From Lavis & Mitchell, 2006: In the first phase of this experiment you will see some colored grids, one at a time. Please examine them carefully. The grids are very similar but some of them have small differences. Please try to find these differences. After each grid, please press the spacebar to see the next grid. Pay careful attention because you will be asked what you think about them later. Press spacebar to continue. Dutch Translation: Welkom bij dit onderzoek. In het eerste deel van dit onderzoek zie je enkele gekleurde roosters, één voor één. Bestudeer deze aandachtig. De roosters lijken erg op elkaar, maar sommigen hebben kleine verschillen. Probeer deze verschillen te vinden. Druk alsjeblieft op de spatiebalk na elk rooster om het volgende rooster te zien. Let aandachtig op omdat je later gevraagd zal worden wat je over de roosters denkt. Leg een vinger op de spatiebalk, en klik op de verder knop om met de taak te beginnen.

17 Before the test phase (based on Lavis & Mitchell, 2006, but changed to reflect the differences between studies): In this phase of the experiment, you will be presented with two grids, one after the other. Stimuli will appear briefly on the screen, and you will need to categorize them as same or different. If the pair of stimuli are the same, press A. If the pair of stimuli are different, press 5 on the numerical keyboard. Don t spend too much time on each judgments. Place your fingers on the keys now. Press the spacebar to begin. Dutch Translation: In dit deel van het experiment worden twee roosters met gekleurde vierkantjes gepresenteerd, één na de ander. Stimuli verschijnen kort op het scherm, en jij moet ze categoriseren als hetzelfde of verschillend. Als het paar stimuli hetzelfde is, druk je op de A. Als het paar stimuli verschillend is, druk je op de 5 op het numerieke toetsenbord. Besteed niet te veel tijd aan elke beslissing. Plaats je linker en rechter wijsvinger nu op de toetsen op het toetsenbord. Druk op de verder knop om te beginnen.