Negation-induced forgetting: Is there a consequence to saying "no"?

Transcription

1 Graduate Theses and Dissertations Iowa State University Capstones, Theses and Dissertations 2017 Negation-induced forgetting: Is there a consequence to saying "no"? Rachel Elizabeth Dianiska Iowa State University Follow this and additional works at: Part of the Cognitive Psychology Commons Recommended Citation Dianiska, Rachel Elizabeth, "Negation-induced forgetting: Is there a consequence to saying "no"?" (2017). Graduate Theses and Dissertations This Thesis is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Iowa State University Digital Repository. For more information, please contact digirep@iastate.edu.

2 Negation-induced forgetting: Is there a consequence to saying no? by Rachel Elizabeth Dianiska A thesis submitted to the graduate faculty in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Major: Psychology Program of Study Committee: Christian A. Meissner, Major Professor Jason C.K. Chan Gary L. Wells The student author and the program of study committee are solely responsible for the content of this thesis. The Graduate College will ensure this thesis is globally accessible and will not permit alterations after a degree is conferred. Iowa State University Ames, Iowa 2017 Copyright Rachel Elizabeth Dianiska, All rights reserved.

3 ii TABLE OF CONTENTS Page LIST OF FIGURES... LIST OF TABLES... ABSTRACT.... iv v vi CHAPTER 1 INTRODUCTION... 1 The Negation Effect... 2 Negation in Attention... 4 Negation in Memory... 6 Overview of the Current Studies... 7 CHAPTER 2 EXPERIMENT Method... 9 Experiment 1A Results Experiment 1B Results Discussion Table and Figures CHAPTER 3 EXPERIMENT Method Experiment 2A Results Experiment 2B Results Combined Samples Analysis Discussion Table and Figures CHAPTER 4 EXPERIMENT Method Results Discussion Table and Figures CHAPTER 5 GENERAL DISCUSSION Replicability of the Negation Effect... 42

4 iii Theoretical Mechanisms Leading to Negation Practical Implications of the Negation Effect in Memory Conclusions and Figures REFERENCES APPENDIX A. EXPERIMENT 1 QUESTIONS APPENDIX B. EXPERIMENT 1 RKG INSTRUCTIONS APPENDIX C. EXPERIMENT 2 INSTRUCTIONS APPENDIX D. EXPERIMENT 2 QUESTIONS APPENDIX E. EXPERIMENT 3 QUESTIONS APPENDIX F. EXPERIMENT 3 RKG INSTRUCTIONS APPENDIX G. IRB APPROVAL FOR EXPERIMENTS... 60

5 iv LIST OF FIGURES Figure 1 Experiment 1 Procedure Figure 2 Effect Size Graph for Mayo et al. (2014), Experiment 1A, and Experiment 1B Figure 3 Effect Size Graph for Experiment Figure 4 Graph of Negation Effect by Number of Alternatives Interaction Figure 5 Forest Plot of Negation Effect Sizes Page

6 v LIST OF TABLES Table 1 Means for Final Test Measures in Experiment 1A (False Rejection, Accurate Recognition) and Experiment 1B (proportion recalled) Table 2 Means for Final Memory Test Errors in Experiment Table 3 Means for Final Test Measures in Experiment 3, Load-Absent Condition Table 4 Means for Final Test Measures in Experiment 3, Load-Present Condition Page

7 vi ABSTRACT The negation effect refers to the cognitive detriment associated with correctly saying no (a negation), compared to correctly saying yes (an affirmation). A recent study has shown this detriment for item memory following the negation of a feature of an item (Mayo, Schul, & Rosenthal, 2014). This research examines the replicability of the negation effect using the original paradigm, as well as an adapted list-learning paradigm. Participants studied a set of objects and were then asked questions about features of objects that elicited yes or no responses. After a filler task, participants completed a final memory test during which they indicated whether a given object label was present or not present during the study phase. Experiment 1 failed to conceptually replicate the negation-induced forgetting effect present in Mayo et al. (2014) using a list-learning paradigm. Experiment 2 was a pre-registered replication, and the negation effect was successfully replicated using the original stimulus and test materials from Mayo et al. (2014). Experiment 3 successfully replicated the negation effect using a list-learning paradigm, and found that the magnitude of the negation effect is influenced by the number of alternatives suggested by a feature statement.

8 1 CHAPTER I: INTRODUCTION Imagine the following scene: one afternoon, you hear through your window what you think may be gunshots. Turning around to look, you see one man lying on the ground behind a car, and another man holding a gun a few feet away. You make a note of the gunman s description in your head: tall, average build, white t-shirt and jeans, black baseball cap. While you are phoning the incident in to the police, the man with the gun grabs a backpack from the trunk of the car and runs away. The police arrive a few minutes later, and once the area is secured you are asked to make a statement. A detective asks you a simple question: Was the baseball cap blue? Your answer to this question a yes or no may later influence what you remember about the event. Cognitive psychologists have long been aware of the limitations and malleability of human memory (e.g., Loftus, 2005). An understanding of these limitations has informed decades of research on how memory can be altered or falsely recollected, as well as how memory retrieval can be improved in applied contexts (e.g., interviewing of witnesses and suspects). However, advances in interviewing techniques have only been accompanied by a partial understanding of how the type of question a person is asked can influence his or her memory. Studies have assessed the influence of mnemonic techniques (the Cognitive Interview; Geiselman, Fisher, MacKinnon, & Holland, 1986), generating verbal descriptions (the verbal overshadowing effect; Schooler & Engstler- Schooler, 1990), suggestive questioning and social influence (e.g., Garven, Wood, Malpass, & Shaw, 1998; Hope, Ost, Gabbert, Healey, & Lenton, 2008), and interference due to selective retrieval of event information (e.g., Camp, Wesstein, & Bruin, 2012; Chan, Thomas, & Bulevich, 2009) on subsequent memory for an event or suspect. One

9 2 aspect of questioning that has been overlooked involves the potential influence of negation on subsequent memory that is, the cognitive detriment associated with correctly saying no to a question (negation), compared with correctly saying yes to a question (affirmation). Prior studies have primarily situated negation in a context of lexical comprehension, focusing on how negations themselves are communicated and understood; however, memory researchers utilizing manipulations requiring a yes or no response have also observed differences in performance based upon the response given. The finding that a person s memory or comprehension can be differentially influenced based upon whether one responds affirmatively or negatively appears to have been demonstrated consistently, yet has garnered little attention. The following studies attempted to replicate this negation effect in memory, and to further identify factors that may moderate the effect. Specifically, these studies examined the influence of affirmative or negative responses to forced-choice, yes-no statements related to a feature of an object (e.g., The glass was empty ) on subsequent memory for the object of the question. In this context, the negation effect encompasses a comparative memory impairment based upon an accurate response of no to questions about features of studied objects, rather than an accurate response of yes. The Negation Effect The negation effect has been studied since the 1960s, particularly in the area of psycholinguistics. In this context, negations are represented as sentences describing how a situation is not (e.g., Susan does not bake cookies), whereas affirmations are represented as sentences describing the actual situation (e.g., Stephen tidied up his

10 3 drawers). The nature of how negations are represented and accessed, as well as how accessibility influences understanding and inference-making has been explored. Using lexical comprehension and sentence verification tasks, a negation effect has been shown when participants more quickly and more accurately verify affirmative statements (Gough, 1965; Wason, 1961). Negated words and phrases have also been associated with decreased accessibility and slower response times (Engelkemp & Hormann, 1974; Kaup & Zwann, 2003; MacDonald & Just, 1989; Meyer, 1975). Despite years of research, the manner in which negations are represented in memory is still debated. Two models of negation representation that have been explored include the schema-plus-tag model and the fusion model. The schema-plus-tag model proposes that a negated message is first processed using an affirmative meaning and is then negated (e.g., Mayo, Schul, & Burnstein, 2004). For example, this model suggests that a negated statement such as not red would first be processed as red with the negation operator subsequently added to the representation. The fusion model, in contrast, proposes that the meaning of the negated phrase is the result of merging of the negation operator with the affirmed meaning. The phrase the door is not open would thus be represented as the fusion of not and open, or closed. These competing models of representation yield different implications for associations that are activated when the negated statement is processed, as well as for long-term retention of the meaning of the negated statement. To compare the two models, Mayo and colleagues (2004) examined the inferences participants made as different negations and affirmations were processed. Participants encoded affirmed target sentences (e.g., Tom is a tidy person ) or negated

11 4 target sentences (e.g., Tom is not a tidy person ) and then determined whether a probe sentence describing a behavior (e.g., Tom forgets where he left his car keys ) fit the meaning of the target sentence. Negations in this study were presented three ways: semantically negated, as sentences with negation operators; visibly negated, with a red background that signaled that a sentence should be negated; or dually negated, as sentences with negation operators that were also displayed on a red background. The behavioral probes could be consistent with, inconsistent with, or irrelevant to the meaning of the target sentence. A baseline condition was included for comparison, wherein people first saw the behavioral probe and then assessed the congruency of the affirmed or negated sentences. Compared to this baseline, people were quicker to respond when the probe meaning was consistent with an affirmed target phrase, as well as when the probe meaning was inconsistent with a negated target phrase. Processing a negated message first as an affirmation in the schema-plus-tag model would activate associations that are inconsistent with the negated meaning. Thus, Mayo and colleagues (2004) suggested that the facilitation of incongruent associations with negated sentences in this study supports the schema-plus-tag model (e.g., Hasson & Glucksberg, 2006). In addition to representations, research into negation has extended to other areas of cognition, including attention and memory. Negation in Attention Negations are a crucial element of communication, for instance in directing attention away from or administering instructions not to do something. The use of negation as a communication device can sometimes itself lead to communication errors. Consider the classic studies of thought suppression: when people were instructed, don t

12 5 think about white bears, sometimes that instruction led people to paradoxically think about white bears (Wegner, Schneider, Carter, & White, 1987). Recently, Maciuszek (2013) examined the nature of negations in commands (e.g., don t pay attention to [target], ) in memory and comprehension. Attention focus, as measured by the amount of details recalled about the target, was found to be greater for people who received the negation order, compared to people in control groups who either were not told anything about the target, or who were not ordered to pay attention to the target. In other words, when participants were instructed not to pay attention to something, attention was drawn to the object to a greater extent. Orenes, Beltrán, and Santamaría (2014) used a visual world paradigm to investigate how negations are understood and represented based on the situational context and number of available alternatives. The visual world paradigm allowed for both images and verbal information to be presented simultaneously, and eye movement data was collected to determine the focal point of attention. Four images of colored figures (red, green, blue, yellow) were displayed while a statement was presented. Initial statements that were presented to participants manipulated the situation in which figures would appear: either with two alternatives ( The figure could be red or green ) or with multiple alternatives ( The figure could be red, or green, or blue, or yellow ). Subsequent statements were ether phrased as affirmations ( The figure was red ) or negations ( The figure was not red ). When the initial statement set up a context in which multiple alternatives were present, people tended to direct their attention to the object of a negated sentence. That is, participants who were presented with the negated statement the figure was not red would focus on the red figure, rather than any of the other alternate colored

13 6 figures. On the other hand, when a situation implied only two alternatives, participants attention was directed to the intended color conveyed by the statement. That is, when presented with the negated statement, the figure was not red, participants focused on the figure of the other color present in the scenario. The attentional component of negation thus appears to be sensitive to the number of alternatives suggested by the statement, and this may differentially influence what is remembered following that negation. Negation in Memory Manipulations using yes-no questions to assess memory are prevalent throughout the cognitive literature. While the impact of negation on subsequent memory for word lists has been documented in classic studies of the levels of processing effect (Craik & Tulving, 1975), the implications for object or event memory have only recently been explored (Mayo, Schul, & Rosenthal, 2014). In their original demonstration of the levels of processing effect, Craik and Tulving (1975) required participants to respond yes or no to prompts that systematically manipulated the level of semantic processing for a list of words varying, for example, the physical structure of the word (e.g., Is the word in capital letters? ); whether or not the target word rhymed with another (e.g., Does the word rhyme with? ); or whether or not the target word fit a given category (e.g., Is the word a type of? ) or syntactical structure (e.g., Would the word fit the sentence:? ). Across several experiments, Craik and Tulving noted a consistent effect of response type (yes or no), with the pattern of means suggesting that words associated with a yes response were better remembered compared to words associated with a no response. Positive responses allowed for the encoding prompts to be better

14 7 integrated with the target items, resulting in a more elaborate memory trace. More recently, Mayo, Schul, and Rosenthal (2014) demonstrated that correctly negating a feature of an object can subsequently impair memory for that object, compared to affirming a feature of the object. Mayo and colleagues examined a negation-induced forgetting effect based on the nature of a rehearsal (affirmative or negative). Participants were shown a video tour of an apartment and were later asked questions about features of items that were shown in the video. The initial memory test in Experiment 1 was composed of 16 questions, eight eliciting no responses and eight eliciting yes responses. After a 20-minute unrelated filler task, subjects completed a final recognition test for the objects they saw in the video. Overall, when feature questions elicited correct no responses, participants were less likely to remember the object (e.g., ashtray ) on the final memory test, compared to when feature questions elicited yes responses (d = 0.53 [0.19, 0.87] 1 ). Mayo and colleagues termed this comparative memory impairment negation-induced forgetting. That is, when subjects thought about an object and negated it, the representation of that object was subsequently less accessible. Given the potential applicability of this negation impairment to domains like education and forensic interviewing, I sought to examine the replicability and robustness of the effect. Overview of the Current Studies In three experiments, I examine the replicability of the negation effect in memory. In Experiment 1, I attempt a conceptual replication of Mayo et al. (2014) using a listlearning paradigm. In Experiment 2, I conduct a pre-registered direct replication of Mayo 1 The effect size was provided by the primary author, and confidence intervals for effect sizes were constructed using ESCI software (Cumming, 2012).

15 8 et al. (2014) using the original stimulus and test materials. In Experiment 3, I return to the list-learning paradigm to examine two potential moderators of the negation effect: memory load at encoding, and the number of alternatives suggested by a test statement.

16 9 CHAPTER 2: EXPERIMENT 1 The negation-induced forgetting effect seen in Mayo et al. (2014) prompted a replication attempt using a list-learning paradigm. Subjects in the first of Mayo and colleagues (2014) experiments studied all of the stimuli (i.e., everyday household objects) by watching a video of a computer-simulated tour of an apartment. After watching the video, subjects provided yes and no responses to statements describing features of the objects that they had seen in the apartment. Following a 20-minute unrelated filler task, subjects then completed a final object recognition test. The following experiment examined the replicability of the negation effect when using a single-item presentation list-learning paradigm. Subjects in the current study were shown images of simple objects and presented with statements about features of those objects that elicited yes or no responses. To conceptually replicate the encoding experience of subjects in the Mayo paradigm, subjects in Experiment 1 studied a series of objects in sequence and were then immediately tested on the feature statements (see Figure 1). Following a 20-minute filler period, subjects were administered a final test involving either object recognition (Exp. 1A) or free recall (Exp. 1B). Method Participants. A total of 84 subjects (42.9% male) completed Experiment 1 for partial course credit. Mean age was (SD = 2.28). In Experiment 1A, 49 subjects completed a final recognition test. In Experiment 1B, 35 subjects completed a final free recall test. Three subjects were excluded from the analysis in 1B due to a failure to follow instructions.

17 10 Materials and Design. Stimuli included 32 images of simple objects retrieved from the Massive Visual Memory Stimuli dataset (Brady, Konkle, Alvarez, & Oliva, 2013). Object images were selected to vary based upon the features of both an attribute (e.g., color) and state (e.g., open/closed). The feature statements for each studied object were administered via pre-recorded audio files that involved a female speaker reading the statements aloud. Each statement recording was between 2500ms and 3000ms. Statements that were presented to participants can be found in Appendix A. This study employed a within-subjects design manipulating response to feature questions on an initial memory test (yes or no) 2. Object memory was assessed via performance on a final memory test. In Experiment 1A, subjects indicated that an object was Present or Not Present in the study phase on a final recognition test. In Experiment 1B, subjects freely recalled all of the items that they could remember seeing in the study phase. Procedure. This study was divided into three phases: a study phase, an initial test phase, and a final test phase. A schematic representation of the procedure can be found in Figure 1. Prior to the first phase, subjects provided informed consent and received instructions about the experiment. Specifically, they were instructed that they would study a set of simple objects and be asked questions about features of those objects at a later time. They were not informed that they would take a final test with regard to the 2 This condition was independent of another study manipulation in which the yes/no questions were presented immediately prior to encoding an image, similar to the paradigm used by Craik & Tulving (1975). For subjects in that pre-encoding condition, there was no significant negation effect in either recognition (d =.02 [-.28,.23]) or recall (d =.45 [-.14, 1.03]). For the purpose of this paper, I focus the discussion on this postencoding manipulation of negation, consistent with the Mayo et al. (2014) paradigm.

18 11 images that they studied. The remaining instructions and tasks were presented to subjects via E-Prime 2.0 TM. Subjects were randomly assigned to a condition prior to arriving for the experiment. In the study phase, subjects studied the full set of 32 objects for 250ms each with a 1000ms ISI. Following the presentation of all object images, subjects completed the initial test phase, which was comprised of feature statements that described correct or incorrect attributes of the studied objects. These feature statements were randomly divided into yes or no responses, and counterbalanced so that each object was equally associated with both responses across all conditions. Immediately following the study phase, all subjects completed an unrelated 20- min filler task that required them to locate sequences of numbers vertically, horizontally, or diagonally, similar to a word search. Next, subjects completed a final object memory test. In Experiment 1A, this final test was a recognition test: subjects were shown simple object labels (e.g., highlighter ) and were asked to indicate whether that object was present or not present in the first part of the study. The final recognition test was comprised of 64 object labels half of these object labels corresponded to the 32 studied objects from the first phase of the experiment (and thus elicited correct Present responses), while the other half corresponded to new, unstudied objects (and thus elicited correct Not Present responses). The order of presentation for test items was determined randomly for each subject. For each item, subjects were also asked to provide a confidence estimate and phenomenological memory (i.e., remember-know-guess) judgment. Confidence was assessed using a 1 to 5 scale, with 1 corresponding to not sure at all and 5 corresponding to definitely sure that the object was or was not

19 12 studied. Subjects were provided instructions with respect to remember/know/guess responses and were asked to choose between I (would have) recollected seeing the object, The object is (not) familiar to me, and I am guessing for each decision on whether an object was studied. 3 Specific instructions can be found in Appendix B. The order of presenting confidence and memory basis judgments was counterbalanced such that half of the subjects first rated confidence and then provided the memory basis judgment, while the other half first provided a memory basis judgment and then the confidence rating. The final test in Experiment 1B involved a free recall test. Subjects were given as much time as they needed to list objects recalled from the first part of the experiment. Subjects were prompted twice to recall all of the objects they could remember studying. Free recall responses were coded by a research assistant blind to the images used in the study. A response was counted as correct/present if the research assistant could discern what object was being named, even if the label was not a perfect fit. For instance, the terms marker and highlighter would both be considered as correct for highlighter. After completing the final memory test, subjects were debriefed and dismissed from the study. Given my interest in replicating the negation-induced forgetting effect in Mayo et al. (2014), I will report Bayes Factors (BF 10 ) in addition to the traditional null hypothesis significance tests (NHST). Bayes factors enable the comparison of evidence strength for two models: one in which there is no significant negation effect (the null hypothesis, 3 Subjects provided these responses for both old and new responses. In the results, I focus on the phenomenological memory judgments for only correct old responses.

20 13 Model 0), and one in which there is a significant effect (the alternative hypothesis, Model 1). The BF 10 factor will be used to express the probability of the data given the alternative hypothesis (Model 1) relative to the probability of the data given the null hypothesis (Model 0). However, when this value is less than 1, I will invert it for interpretation. For the subject-level effects, I will use the effect size for Mayo et al. (2014) Experiment 1 as provided by the primary author, d = 0.53 [0.19, 0.87], as the prior distribution. For the item-level analyses, I will use the standard Cauchy prior of to calculate BF 10. The magnitude of BF 10 can be used to interpret the strength of the evidence: a factor of 0 to 3 is considered anecdotal evidence; 3 to 10 is considered substantial evidence; 10 to 100 is considered strong evidence; while a factor greater than 100 is considered decisive evidence (Jeffreys, 1961). Results Experiment 1A Initial Memory Test. The initial memory test was used to elicit yes and no responses from participants. It consisted of 32 statements pertaining to features of objects that were studied. Half of the statements described correct features of the objects and required yes responses; the other half described incorrect features of the objects and were correctly answered with no responses. A paired-samples t-test was used to assess the influence of initial memory test response (yes, no) on the proportion of accurate responses to the feature statements. Subjects were similarly accurate for statements requiring a yes response (M =.72, SE =.02) and for statements requiring a no response (M =.70, SE =.02), t(48) = 1.52, p =.14, d = 0.17 [-0.05, 0.40]. Final Memory Test. Two outcome measures were of primary interest: (1) conditionalized errors on the final memory test, and (2) phenomenological memory bases

21 14 for accurately recognized objects on the final memory test. For each person, the proportion of conditionalized errors made on the final memory test was calculated. Conditionalized errors refer to items that were associated with a correct response on the initial memory test, but were responded to as Not Present on the final memory test. Proportions of Remember and Know responses were also calculated for objects correctly recognized as having been present in the first part of the experiment. Know responses were corrected for independence of the two processes by dividing the number of Know responses (K) by the opportunities to respond Know (1 R; see Yonelinas & Jacoby, 1995). A paired-samples t-test was used to assess the influence of initial memory test response (yes, no) on the proportion of errors on the final memory test (i.e., studied items that were correctly answered on the initial test but were not recognized as having been studied in the first part of the experiment). Although numerically more errors were made following no responses (M =.10, SE =.02) than following yes responses (M =.07, SE =.01), a statistically significant negation effect was not observed, t(48) = 1.49, p =.14, d =.27 [-.09,.62]. To further examine the null effect, a Bayesian paired-samples t-test was conducted using JASP software (JASP Team, 2016). An estimated BF 10 of 0.54 suggests that the data were 1.85 times more likely under the null hypothesis, which is considered weak evidence. In addition to these subject-level analyses, an item-level analysis was conducted to determine if a negation effect was present across the 32 target items. When examined at an item-level, there was a marginally significant negation effect, t(31) = 2.03, p =.05, d = 0.38 [.01,.74]. An estimated BF 10 of 1.15 suggests that the data were

22 times more likely under the alternative hypothesis on an item-level. Again, this Bayes Factor is considered weak or anecdotal evidence. The second measure of interest involved the basis upon which subjects correctly recognized objects. A paired-samples t-test was used to assess the influence of initial memory test response (yes, no) on the proportion of accurately recognized objects based on remembering and knowing. No differences in recollection responses were observed following yes (M =.85, SE =.02) vs. no responses (M =.84, SE =.03), t(48) =.48, p =.63, d = 0.07 [-0.22, 0.35]; nor were differences observed in familiarity responses following yes responses (M =.55, SE =.07) vs. no responses (M =.58, SE =.07), t(48) =.39, p =.70, d = 0.06 [-0.37, 0.25]. Results Experiment 1B Initial Memory Test. A paired-samples t-test was used to assess the influence of initial memory test response (yes, no) on the proportion of accurate responses to the feature statements. Subjects were similarly accurate for statements requiring a yes response (M =.71, SE =.02) and for statements requiring a no response (M =.69, SE =.02), t(31) =.93, p =.36, d = 0.18 [-0.20, 0.55]. Final Memory Test. A paired-samples t-test was used to assess the influence of initial memory test response (yes, no) on the proportion of studied objects correctly recalled. This measure was conditionalized for accuracy on the initial test, such that only objects correctly answered with yes or no on the initial test were included in the final proportion recalled. Subjects recalled fewer objects associated with a no response (M =.29, SE =.02) than those associated with a yes response (M =.32, SE =.03); however, this difference was not significant, t(31) = 1.04, p =.31, d =.21 [-.14,.56]. An estimated

23 16 BF 10 of 0.39 suggests that the data were 2.58 more likely under the null hypothesis. An item-level analysis was used to determine if a negation effect was present across the 32 target items. The item analysis revealed a significant negation effect, t(31) = 2.28, p =.03, d = [-1.52, -.08]. More objects were correctly recalled after a yes response (M =.59, SE =.04) than after a no response (M =.41, SE =.04). An estimated BF 10 of 1.79 indicates that the data provided weak evidence for the alternative hypothesis on an item-level. Discussion The present studies failed to find a significant negation effect for either conditionalized errors on a final recognition test (1A) or for conditionalized accuracy on a final free recall test (1B). A similar effect size for the negation effect was observed for both recognition, d =.27 [-.09,.62] and recall, d =.21 [-.14,.56]. While these effects fall within the confidence intervals of the original effect produced by Mayo et al. (2014) [0.19, 0.87], they are at the lower end of the distribution and were about half the size original effect (d = 0.53). Bayes factors (BF 10 = 0.54 and 0.39, respectively) suggested only weak evidence in favor of the null hypothesis. Despite the presence of a negation effect in an item-level analyses, the lack of subject-level effects takes precedence in the interpretation of the negation effect s non-significance (e.g., Raaijmakers, Schrijnemakers, & Gremmen, 1999).

24 17 Table 1 Means for final test measures in Experiment 1A (false rejection, accurate recognition) and Experiment 1B (proportion recalled) After yes After no Measure Mean SD Mean SD False Rejection Proportion Errors Confidence Accurate Recognition Response Latency Confidence Remember Know (Corrected) Proportion Recalled Accurate

25 18 Figure 1. Graphic representation of the procedure used in the present Experiment 1 to conceptually replicate the procedure used in Experiment 1 of Mayo, Schul, & Rosenthal (2014). Effect Size (Cohen's d) Mayo et al. (2014) Experiment 1 Exp. 1 Recognition Test Figure 2. Effect sizes with 95% confidence intervals for Mayo et al. (2014), Experiment 1A (recognition test), and Experiment 1B (recall test). Exp. 1 Recall Test

26 19 CHAPTER 3: EXPERIMENT 2 Experiment 2 attempted a direct replication of Mayo et al. s (2014) Experiment 1, using the original stimuli and test materials. Following the failed conceptual replication, I contacted the primary author and requested her materials. I also clarified the type of filler task and items used as lures on the final recognition test to elaborate on the concise description provided in the original article. The primary author reviewed the procedure to verify the similarity between my proposed method and the complete method used in the original experiment. Experiment 2 was thus a direct replication using the translated materials (from Hebrew to English) published in the original paper, as well as those obtained from the primary author. This study was pre-registered via the Open Science Framework prior to beginning data collection as part of the Pre-Registration Challenge (osf.io/p2qfv; Spies et al., 2012). As I pre-registered only a laboratory sample, I will first present the results for the pre-registered sample (Experiment 2a) and then discuss the additional unregistered online sample (Experiment 2b). Method Sampling Plan and Participants. The sampling plan was based on a power analysis to detect a small to medium effect size. Using G*Power 3.1 (Faul et al., 2009), a planned sample size of n = 68 was calculated to achieve power of 0.90 using a two-tailed dependent t-test, given a small to medium effect size. In total, the pre-registered laboratory sample (Experiment 2a) was comprised of 75 students (45.3% male) from Iowa State University, who participated in this experiment for partial course credit. Mean age was (SD = 1.05). Due to a program error, demographic information is missing for one subject. Data from ten subjects was

27 20 excluded from analysis due to subjects being non-native English speakers (N = 9) or being identified as multivariate outliers via Mahalanobis distance (N = 1; Tabachnick & Fidell, 1996) 4. Additionally, 80 subjects (41.40% male) completed an unregistered online sample (Experiment 2b) via Amazon s Mechanical Turk. Mean age was (SD = 10.49). Workers were paid $1.00 to complete the task. Data from ten subjects were excluded from the online sample for failing to respond correctly to attention checks. Materials. This experiment was presented to subjects in the laboratory using E- Prime 2.0 TM, and to subjects on Mechanical Turk via Qualtrics. The video presented in this study involves an 8 min, 15 s video provided by the primary author of the original experiment (Mayo et al., 2014). The video depicted a tour of the upstairs and downstairs portions of a digitally simulated apartment. Present in the video were a number of everyday household objects. The original experiment instructions and test items used in Experiment 2 can be found in Appendix C and Appendix D, respectively. Four different randomizations of the initial test were programmed and counterbalanced across participants. Design and Procedure. This study was a fully within-subjects design, manipulating yes-no responses to statements on an initial memory test. Only slight variations in the procedure were present in the online study (when compared with the 4 The original study eliminated the few subjects who erred in more than 50% of initial memory test statements. The registration overlooked the exclusion of these subjects. I performed the analyses with these subjects (N = 6) and without, and the conclusions did not change. To be consistent with the pre-registration, the results presented here include the 6 subjects who were less than 50% accurate on the initial memory test.

28 21 laboratory version). Subjects provided informed consent and were told that they would view a brief film and answer questions about it. Prior to beginning the experiment, subjects in the online sample also answered a set of attention check questions (e.g., Who is the current president of the United States? ) to ensure that subjects attended throughout the duration of the study. With the exception of an additional set of attention check questions present after the filler task was completed, the remainder of the procedure did not differ between the laboratory and online samples. After receiving instructions for how to proceed, subjects were instructed to view the stimulus video. They were then told that they would complete a short memory task involving a series of sentences. The task was comprised of 16 feature statements that described either congruent features of objects in the apartment, and were thus correctly answered with yes responses, or incongruent features of objects in the apartment, and were thus correctly answered with no responses. The questions were randomly split twice, and in each random split the correct yes and no responses were counterbalanced, resulting in four randomization conditions. After responding to all 16 statements, participants completed a 20-minute unrelated filler task comprised of a wordconstruction task in which participants were presented with 15-character words in English (e.g., overadjustments ) and were instructed via the computer program to generate as many new words as possible using only the letters in the given base word (e.g., random, mentors, etc.). Subjects were given 4 min to generate as many words as possible for each of five base words. Once all five base words had been completed, subjects were administered a final object recognition test in which they were presented with a series of object labels (e.g.,

29 22 chair ) and asked to indicate whether the object was seen in the video. Items on the final test included the 16 tested objects on the initial memory test, as well as 16 novel objects that were not featured in the video. The 32 objects on the final test were identical to those used by Mayo and colleagues (2014). After completing the final test, subjects were debriefed and dismissed from the study. Results Experiment 2A Initial Memory Test. The initial memory test consisted of 16 statements pertaining to features of objects seen in the apartment video. Half of the statements described correct features of the objects and required yes responses; the other half described incorrect features of the objects and were correctly answered with no responses. Accuracy on the initial memory test refers to the proportion of correct yes and no responses to the feature statements. Subjects were more accurate for statements requiring a no response (M =.75, SE =.02) than for statements requiring a yes response (M =.65, SE =.03), t(64) = 3.07, p =.003, d = 0.53 [.18,.88]. This effect is inconsistent with Mayo et al. (2014) there was no difference between yes and no response accuracy on the initial test in the original experiment. Final Memory Test. The final memory test consisted of 32 object labels pertaining to 16 tested objects that were present in the video, as well as 16 novel objects that were not present in the video. Subjects indicated whether the object was present or not present in the apartment that they viewed. Accuracy on this final test was calculated as the proportion of the 32 test items correctly indicated as being present (critical items) or not present (lure items) in the video. Subjects were mostly accurate on this test (M =.85, SD =.19).

30 23 The measure of primary interest is that of errors on the final memory test: objects that were present in the video that were incorrectly categorized as not present on the final memory test. This measure was conditionalized for accuracy on the first test, such that only items that subjects correctly answered with yes or no on the initial memory test were included in the analysis. For both yes and no responses, the proportion of target objects that were correctly responded to on the initial memory test but were not recognized as having been present on the final test was calculated. Consistent with Mayo et al. (2014), subjects demonstrated a significant negation effect such that target objects associated with no responses (M =.14, SE =.02) produced more errors in memory, compared to target objects associated with yes responses (M =.07, SE =.02), t(64) = 3.08, p =.003, d = 0.50 [.17,.82]. An estimated BF 10 of suggests strong evidence for the alternative hypothesis (the presence of a negation effect) for a subject-level analysis. An item-level analysis was conducted separately to determine if the negation effect was present across the 16 target items. A paired-samples t-test found only a marginally significant negation effect, t(15) = 2.01, p =.06, d =.60 [-.04, 1.23] 5. An estimated BF 10 of 1.26 suggests anecdotal evidence for an item-level negation effect in the laboratory sample. Results Experiment 2B Initial Memory Test. With regard to the initial memory test involve yes-no responses, accuracy was again greater for statements requiring a no response (M =.66, SE =.02) than for statements requiring a yes response (M =.56, SE =.03), t(69) = 2.51, 5 The registration also overlooked the item-level analysis. I present it here to be consistent with the results presented in Mayo et al. (2014).

31 24 p =.014, d = 0.46 [.09,.82]. Although this effect is inconsistent with Mayo et al. (2014), it replicates what was observed in the laboratory sample in Experiment 2A. Final Memory Test. Performance on the final memory test was assessed with the same measure of conditionalized errors. For each item type (yes, no), a proportion was calculated of objects that were correctly answered on the first test, but were incorrectly categorized as not present on the final memory test. Similar to the laboratory sample, subjects demonstrated a significant negation effect: objects associated with no responses (M =.16, SE =.02) produced more errors in memory, compared to objects associated with yes responses (M =.08, SE =.02), t(69) = 2.54, p =.014, d = 0.41 [.17,.65]. An estimated BF 10 of 3.04 suggests substantial evidence for the alternative hypothesis (the presence of a negation effect) for this subject-level analysis. An itemlevel analysis was conducted separately to determine if the negation effect was present across the 16 target items. A paired-samples t-test found a significant negation effect, t(15) = 2.88, p =.01, d =.88 [.19, 1.55]. Objects associated with a no response were more likely to be incorrectly responded to as Not Present (M =.16, SE =.03) than were objects associated with a yes response (M =.07, SE =.02). On an item-level, an estimated BF 10 of 4.90 suggests substantial evidence for the presence of the negation effect. Results Combined Samples Analysis (Experiments 2a and 2b) The same pattern of results on the final test was observed for both the laboratory (Experiment 2a) and online (Experiment 2b) samples, as confirmed by a 2 x 2 mixed model ANOVA assessing the influence of sample (laboratory, online) and response to feature statements (yes, no) on conditionalized final memory test errors. No main effect

32 25 of sample was observed, F(1, 133) =.23, p =.63, nor an interaction between yes-no response and sample, F(1, 133) =.01, p =.94. However, consistent with the original Mayo et al. (2014) study, a significant negation effect was found such that target objects associated with no responses (M =.15, SE =.02) produced more errors in memory compared to target objects associated with yes responses (M =.08, SE =.01), F(1, 133) = 14.93, p <.001, d = 0.45 [0.21, 0.68]. An estimated BF 10 of suggests that the data were times more likely under the alternative hypothesis, which exceeds the threshold for decisive evidence (greater than BF 10 = 100) of the negation effect. An item-level analysis was conducted separately to determine if the negation effect was present across the 16 target items. A paired-samples t-test found a significant negation effect, t(15) = 2.70, p =.02, d =.81 [.27, 1.34]. On an item-level, more items associated with no responses (M =.16, SE =.03) were erroneously indicated as Not Present on the final test than were items associated with yes (M =.08, SE =.02) responses. Discussion Experiment 2 was a direct replication of the negation-induced forgetting effect using Mayo et al. s (2014) original materials and instructions from Experiment 1. In addition to the pre-registered laboratory sample (Experiment 2a), an online sample using Amazon s Mechanical Turk was included (Experiment 2b). The primary interest was in replicating the negation effect; however, I was also interested in examining the difference in effect size between the Mechanical Turk sample and a traditional laboratory sample. Numerous studies have demonstrated Mechanical Turk s validity as a data collection tool (e.g., Mason & Suri, 2012). The observed negation effect did not significantly differ

33 26 across the samples: the laboratory sample demonstrated a slightly larger effect size, d =.50 [.17,.82], when compared with the online sample, d =.41 [.17,.65], however, the confidence intervals suggest the magnitude of the effect does not differ between the samples (see Figure 3). These findings add further support to the validation of Amazon s Mechanical Turk as an effective pool of subjects. In recent years, replication has become a priority in psychological research, including establishing guidelines for conducting replications (e.g., the replication recipe, Brandt et al., 2014). I was able to replicate the negation effect using the original stimulus and test materials from Mayo and colleagues. Although the observed effect size was marginally smaller in magnitude, the upper limit of the confidence interval encompasses the observed effect size from the Mayo et al. (2014) Experiment 1 (see Figure 3). Thus, this replication is best described as a successful replication: the effect size of the replication was significantly different from the null, and similar to the original effect size. While the Mayo et al. (2014) paradigm is effective in producing the negation effect, there may be external factors that influenced the size of the effect produced in the present replication. Most notably is the potential influence of language. The original study was conducted in Hebrew, while the materials in publication and those provided by the author involved translations into English. However, I cannot identify a theoretical reason that language differences would have resulted in a smaller effect size. Another general limitation is that Mayo and colleagues did not collect confidence or memory basis judgments. Following the successful replication of the effect, I returned to a listlearning paradigm to examine potential moderators of the negation effect.

34 27 Table 2 Means for final memory test errors in Experiment 2 After yes After no Experiment Mean SD Mean SD Experiment 2A Experiment 2B Combined Samples Effect Size (Cohen's d) Mayo et al. (2014) Exp. 2A Laboratory Exp. 2B Online Exp. 2 Combined Figure 3. Comparison of effect sizes (Cohen s d) for the direct replication in a laboratory (Experiment 2A) and online (Experiment 2B) samples.

35 28 CHAPTER 4: EXPERIMENT 3 Following the failure to demonstrate a negation effect in Experiments 1A and 1B, I corresponded with the primary author and more closely examined the procedures and materials used in Mayo et al. (2014). Upon reviewing the stimulus video shown to participants in the original study, I hypothesized that the discrepant findings between the original experiment and the conceptual replication may have been related to differences in how the stimuli were presented to subjects. Specifically, Mayo et al. (2014) demonstrated a significant negation effect using a video tour of an apartment. This video included a large number of household objects and furniture, 16 of which were chosen to be tested after the presentation of the video. I estimated that for every object that was tested, there were 3 to 4 items present within the apartment that were not tested. The replication efforts in Experiment 1 utilized the same basic principles of the paradigm in Mayo et al. (2014) a number of objects presented to subjects, followed by an initial memory test, then a 20-minute filler task, and lastly a final memory test. However, the paradigm in Experiment 1 did not include additional non-tested items during the study task. The presence of the non-tested objects in the Mayo et al. (2014) paradigm may have resulted in an increased memory load for those subjects that was not present in the Experiment 1 single-item presentation paradigm. That is, subjects task in Mayo et al. (2014) may have been more difficult due to encoding necessarily being distributed over more items (both the target items and the non-tested items). In the misinformation literature, people have been shown to be more susceptible to misinformation when memory quality is degraded, due to factors such as the passage of time (e.g., Loftus, Miller & Burns, 1978) or centrality at encoding (e.g., Wilford, Chan, &