Supplemental Materials for Learning absolute meaning from variable exemplars. 1. Additional analyses of participants responses in Experiments 1 and 2

Similar documents
Generation of Quantified Referring Expressions: Evidence from Experimental Data

The Context Repetition Effect: Role of prediction in new memory formation. Honors Research Thesis

04/12/2014. Research Methods in Psychology. Chapter 6: Independent Groups Designs. What is your ideas? Testing

Priming plural ambiguities

URL: <

The role of sampling assumptions in generalization with multiple categories

Supplemental Information. The Evolution of Lateralization. in Group Hunting Sailfish

Flexible Use of Phonological and Visual Memory in Language-mediated Visual Search

EXPERIMENTAL METHODS IN PSYCHOLINGUISTIC RESEARCH SUMMER SEMESTER 2015

COMPARING DECIMALS 1 SCORING GUIDE

Language: English Course level: Doctoral level

When is Likely Unlikely: Investigating the Variability of Vagueness

Comparing Pluralities. Gregory Scontras Department of Linguistics, Harvard University. Peter Graff

Erica J. Yoon Introduction

Chapter 11. Experimental Design: One-Way Independent Samples Design

Data Through Others Eyes: The Impact of Visualizing Others Expectations on Visualization Interpretation

Learning What Others Like: Preference Learning as a Mixed Multinomial Logit Model

Pinball Car Race Online Task

What Is Science? Lesson Overview. Lesson Overview. 1.1 What Is Science?

Statistical Analysis August, 2002

Advanced GCE Psychology

In this chapter we discuss validity issues for quantitative research and for qualitative research.

Invariant Effects of Working Memory Load in the Face of Competition

Visual Scene Understanding

Comparability Study of Online and Paper and Pencil Tests Using Modified Internally and Externally Matched Criteria

A second look at automatic theory of mind: Reconsidering Kovács, Téglás, and Endress (2010)

Supplementary Material: The Interaction of Visual and Linguistic Saliency during Syntactic Ambiguity Resolution

REPEATED MEASURES DESIGNS

Our goal in this section is to explain a few more concepts about experiments. Don t be concerned with the details.

Study Guide #2: MULTIPLE REGRESSION in education

Sample Evaluation Plan. Indicator 4.1.1

Similarity-Based Reasoning is Shaped by Recent Learning Experience

Unit 1 Exploring and Understanding Data

Sawtooth Software. The Number of Levels Effect in Conjoint: Where Does It Come From and Can It Be Eliminated? RESEARCH PAPER SERIES

Natural Scene Statistics and Perception. W.S. Geisler

Encoding of Elements and Relations of Object Arrangements by Young Children

The "Aha! moment: How prior knowledge helps disambiguate ambiguous information. Alaina Baker. Submitted to the Department of Psychology

Supplementary appendix

Lecture 4: Research Approaches

Survey Research. We can learn a lot simply by asking people what we want to know... THE PREVALENCE OF SURVEYS IN COMMUNICATION RESEARCH

Lecture 21. RNA-seq: Advanced analysis

Scale Invariance and Primacy and Recency Effects in an Absolute Identification Task

Chapter 5: Research Language. Published Examples of Research Concepts

ESS/BUDAPEST CROSS NATIONAL PROJECT: Interviewer Note Template for ESS QUESTIONS ONLY

Evaluation of Linguistic Labels Used in Applications

The testing effect for mediator final test cues and related final test cues in online and laboratory experiments

Supplementary Study A: Do the exemplars that represent a category influence IAT effects?

UNIT 4 ALGEBRA II TEMPLATE CREATED BY REGION 1 ESA UNIT 4

Midterm Exam MMI 409 Spring 2009 Gordon Bleil

The influence of (in)congruence of communicator expertise and trustworthiness on acceptance of CCS technologies

A Joint Interference Effect in Picture Naming

66 Questions, 5 pages - 1 of 5 Bio301D Exam 3

Scalar implicature and negative strengthening: What it takes to be happy

ID# Exam 1 PS 306, Spring 2002

Midterm project due next Wednesday at 2 PM

The Effect of Recent Experience on Template Formation during Categorical Visual Search

Research and Evaluation Methodology Program, School of Human Development and Organizational Studies in Education, University of Florida

Within-event learning contributes to value transfer in simultaneous instrumental discriminations by pigeons

Item Analysis Explanation

Diagnostic Study Planning Guide

26:010:557 / 26:620:557 Social Science Research Methods

China Summer Institute 2015 Connie Steinman Connecting Chinese & American Cultures Through Sign Language & Religious Gestures

Nature and significance of the local problem

Research Methods & Design Outline. Types of research design How to choose a research design Issues in research design

Using Condoms Effectively

Where does "analysis" enter the experimental process?

Political Science 15, Winter 2014 Final Review

Overview. cis32-spring2003-parsons-lect15 2

The Meaning of the Mask Matters

Overview EXPERT SYSTEMS. What is an expert system?

Design of Experiments & Introduction to Research

Building Evaluation Scales for NLP using Item Response Theory

Observation and Assessment. Narratives

The essential focus of an experiment is to show that variance can be produced in a DV by manipulation of an IV.

VALIDITY OF QUANTITATIVE RESEARCH

Word Meaning Frequencies Affect Negative Compatibility Effects In Masked Priming

CHAPTER TEN SINGLE-SUBJECTS (QUASI-EXPERIMENTAL) RESEARCH

Midterm Exam ANSWERS Categorical Data Analysis, CHL5407H

b. Associate Professor at UCLA Anderson School of Management

Is Cognitive Science Special? In what way is it special? Cognitive science is a delicate mixture of the obvious and the incredible

The Neural Basis of Financial Decision Making

Evaluating the Causal Role of Unobserved Variables

MMI 409 Spring 2009 Final Examination Gordon Bleil. 1. Is there a difference in depression as a function of group and drug?

English for living in the UK. The Cervical Smear

Dogs Can Discriminate Emotional Expressions of Human Faces

Making a psychometric. Dr Benjamin Cowan- Lecture 9

Research Designs. Internal Validity

THE FIGHTER PILOT CHALLENGE: IN THE BLINK OF AN EYE

Effects of Sequential Context on Judgments and Decisions in the Prisoner s Dilemma Game

Quality Digest Daily, March 3, 2014 Manuscript 266. Statistics and SPC. Two things sharing a common name can still be different. Donald J.

Introduction. Patrick Breheny. January 10. The meaning of probability The Bayesian approach Preview of MCMC methods

aps/stone U0 d14 review d2 teacher notes 9/14/17 obj: review Opener: I have- who has

Analogical Reinforcement Learning

Evidence Based Practice

PLS 506 Mark T. Imperial, Ph.D. Lecture Notes: Reliability & Validity

Rock, Paper, Scissors Investigating traits that are always seen when passed from parents to offspring

Appendix: Instructions for Treatment Index B (Human Opponents, With Recommendations)

Spontaneous Trait Inferences Are Bound to Actors Faces: Evidence From a False Recognition Paradigm

PSY 3393 Experimental Projects Spring 2008

The Typicality Ranking Task: A New Method to Derive Typicality Judgments from Children

Transcription:

Supplemental Materials for Learning absolute meaning from variable exemplars 1. Additional analyses of participants responses in Experiments 1 and 2 As discussed in Section 2.4, our analysis mainly focused on likelihoods with which participants chose the Neither option in the Test Phase. It is a diagnostic tool to assess whether they have learned to mean maximally tight-fitting. (This task was modeled after Syrett et al., (2010) s investigation into children s and adults interpretations of English absolute gradable adjectives). We expected that, if participants semantic representation of includes the maximum standard of comparison (i.e., maximally tight-fitting), they should be more likely to choose the Neither option in Ambiguous test trials. It is, however, also important to evaluate how reliably participants chose the tighter option in the Ambiguous and Unambiguous test trials. In particular, their selection of the tighter option in an Unambiguous trial critically corroborates that they are indeed associating the meaning of with the maximum standard of comparison: when presented with a maximally tight-fitting option and a less tight-fitting option, participants should choose the former as a referent of. To assess participants selection of the tighter-fitting option, we constructed mixed effect regression models similar to what we describe in result sections (Section 2.4). As in the model assessing choices of the Neither option, the analysis was conducted using the glmer function of the lme4 package (Bates, Maechler, Bolker & Walker, 2015) in R (R Core Team, 2016). The dependent measure was participants responses coded as a: Tighter-fitting = 1 and all others = 0. The model included Trial types (Unambiguous vs. Ambiguous), Explanation manipulation (1 = With-Explanation vs. 0 = No-Explanation), and Experiments as fixed effects as well as byparticipant and by-item (at Test) as random effects. Table 1. Model fixed and random effects predicting "tighter-fitting" responses for both Experiments 1 and 2. Experiment 1 and 2: Fixed Effects β Std. Error z-value p-value Intercept 0.49 0.15 3.30 < 0.001 With-Explanation Condition 0.44 0.15 2.93 0.003 Trial (Unambiguous vs. 1.21 0.14 8.46 < 2.00e-16 Ambiguous) Experiment (1 or 2) 0.22 0.15 1.51 0.13 With-Explanation Condition 0.71 0.13 5.37 7.78e-08 x Trial With-Explanation Condition -0.13 0.15-0.87 0.38 x Experiment Trial x Experiment 0.16 0.13 1.24 0.21 With-Explanation Condition x Trial x Experiment 0.11 0.13 0.82 0.41 Experiment 1 and 2: Random Effects 1

Groups Variance Std. Dev. Subject (Intercept) 7.20e-01 8.49e-01 Item (Intercept) 1.56e-16 1.25e-08 Observations: 504, Subjects: 126, Item Type: 2 The results of the analysis are summarized in Table 1. They revealed the main effects of Explanation condition (With- vs. No-Explanation, β =.44, z = 2.93, p <.01) and Trial types (Unambiguous vs. Ambiguous, β = 1.21, z = 8.46, p <.001) such that the tighter-fitting option was more likely to be chosen: 1) when the contextual explanations were provided in Exposure; and 2) when there is an unambiguously tight-fit option was available in a test trial. The interaction between Trials and Explanation manipulation was also significant (β =.71, z =.13, p <.001). Participants were more likely to pick the tighter-fitting option in an Unambiguous test trial when in the With-Explanation condition. Notably, none the following terms including Experiment was significant: main effect of Experiment (Experiment 1 vs. 2), the two-way interaction between the Explanation conditions and Experiment, the two-way interaction between the Trial types and Experiment the three-way interaction between Explanation conditions, Trial types and Experiment This suggests that participants choice of the tighter-fitting option was largely intact when the number of absolute exemplars was halved (i.e., Experiment 2). Together with the choice patterns for the Neither option, these results likely indicate that, in both experiments, participants in the With-Explanation condition inferred a more stringent meaning for the novel adjective: for an object to be it has to be maximally tightfitting (rather than sufficiently tight-fitting). 2. Follow-up experiment: Learning the meaning of with no absolute exemplars We conducted a follow-up experiment to test if participants can infer the maximum standard of comparison for the novel word () exclusively from non-absolute exemplars. We replicated Experiment 2 while replacing all the visually unambiguous (maximally tight-fitting) visual examples with visually ambiguous ones. As a result, all the participants in this experiment watched 24 visually ambiguous exemplars. Half of them were labeled as and the other half were labeled as not. As in Experiments 1 and 2, we used a between subject manipulation and assigned participants randomly to With-Explanation and No-Explanation conditions. 2.1 Participants 99 subjects were recruited to participate from Amazon Mechanical Turk (https://www.mturk.com/mturk/). These participants all identified as monolingual, Englishspeaking adults. They were assigned randomly to one of two between-subject conditions; With- Explanation or No-Explanation. Due to poor catch trial performance (n=1) and past participation in similar experiments (n=25), we excluded 26 participants in total. All remaining 73 participants (34 in With-Explanation and 39 in the No-Explanation conditions) were included in our analyses. 2

2.2. Stimuli In Exposure, participants saw a total of 24 ambiguously tight-fitting examples of. These were constructed from six object types (T-shirt, bracelet, bookshelf, shoe, laptop case, card). Each object type was presented with two unique videos (totaling 12 unique videos). Each video was shown twice while paired with a different description as well as a label of or not. The videos were identical across two between-subject conditions. However, in the With- Explanation Condition, participants received contextual explanations that described the intended use of the novel adjective as an absolute adjective (e.g., I do a lot of hands-on work and don t want to worry about my bracelet falling off. This bracelet is ). In the No-Explanation Condition, the accompanying audio did not provide such information, and instead described the visual scene or irrelevant information (e.g., This is my sister s bracelet and she likes this very much. This bracelet is. The make-up of videos used in the Exposure phase in Experiments 1 through 3 is summarized in Table 2. Table 2. The number of example tokens used in the Exposure phase in Experiments 1 3. Visually unambiguous Visually ambiguous Total labeled as labeled as not labeled as labeled as not Experiment 1 6 6 6 6 24 Experiment 2 3 3 9 9 24 Experiment 3 0 0 12 12 24 2.3 Procedure The procedure was identical to Experiments 1 and 2. 2.4 Planned analysis We sought to determine if the participants learn the meaning of to include a maximum standard of comparison without any exposure to an absolute example. As in Experiment 1, we analyzed participants choice patterns of Neither option in the Unambiguous as well as in the Ambiguous Test trials. To do so, we coded: Neither = 1 and all others = 0 as our dependent variable. We used mixed-effects logistic regression (Breslow & Clayton, 1993; Jaeger, 2008), applying the glmer function of the lme4 package (Bates, Maechler, Bolker & Walker, 2015) in R (R Core Team, 2016). The model included Explanation manipulation (With- Explanation = 1 vs. No-Explanation = 0), Trial type (Unambiguous vs. Ambiguous) as fixed effects. We also random effects of by-participant and by-item (at Test). 2.4 Results Participants were overall most likely to choose the tighter-option regardless of Trial types (i.e., Unambiguous and Ambiguous) and the between-subject Explanation conditions (i.e., Withand No-Explanation conditions) (Figure 1). One notable pattern in the responses in Experiment 3, in comparison to those in Experiments 1 and 2, was the increased proportion of the not-tighter option chosen both in the Unambiguous and in the Ambiguous trials. As a result, participants 3

responses were distributed over the three options in Experiment 3, indicating a great deal of uncertainty and variability in the meaning representations inferred in the Exposure phase. Figure 1. Proportion of responses at Test in the follow-up experiment (Experiment 3). Our planned analysis on participants choice of the Neither option revealed the main effects of explanation conditions (β = -0.45, z = -2.03, p <.05) and Trial types (β = 0.76, z = 4.60, p <.0001) (Table 3). Unlike in Experiments 1 and 2, however, the interaction between these two fixed effect variables was not significant. These results suggest that, even without any exposure to absolute exemplars, participants were capable of relating the meaning of to the positive valency of tightness of fit. Moreover, at least some subjects seem to have been able to infer the maximum standard of comparison. The effectiveness of contextual explanations, however, significantly dissipated in this experiment as compared to Experiments 1 and 2. The presence of contextual explanations did not appear to have supported explaining away of visually ambiguous (non absolute) exemplars. Table 3. Model fixed and random effects predicting "Neither" responses for Experiments 2-3. Experiment 3: Fixed Effects β Std. Error z-value p-value Intercept -1.69 0.25-6.77 <1.32e-11 With-Explanation -0.45 0.22-2.03 <.05 Condition Trial (Ambiguous) 0.63 0.18 3.56 <.001 With-Explanation Condition x Trial 0.16 0.17 0.9 0.36 Experiment 3: Random Effects Groups Variance Std. Dev. Subject (Intercept) 0.99 0.99 4

Item (Intercept) 0 0 Observations: 544, Subjects: 166, Item Type: 2 The results of this follow-up experiment illuminate the importance of visually unambiguous (absolute) exemplars in inferring the maximum standard of comparison. In its absence, participants were unlikely to infer the absolute meaning (maximally tight-fitting) for the novel adjective introduced in the Exposure phase. Together with the results from Experiment 1 and 2, the current result suggests that explaining away comes to effect when some absolute exemplars are available to guide attention to an end point on a relevant scale. 5