Houghton Mifflin Harcourt Participant s Guide Distractor Rationales Fall 2012 User s Conference By Christina Fritz
Topics for Discussion High Quality Items Anatomy of a Multiple Choice Item Types of Distractors Samples The Importance of Designing High Quality Items Assesses with intent Decisions on student proficiency are decided based on assessment items Items should be reliable and valid The Role of Multiple Choice Items Standards from simple to complex can be measured. Highly structured and clear tasks are provided. A broad sample of achievement can be measured. Incorrect alternatives provide diagnostic information. Scoring is easy, objective, and reliable. Item analysis can reveal how difficult each item was and how well it discriminated between the strong and weaker students in the class Can cover a lot of material very efficiently (about one item per minute of testing time). Items can be written so that students must discriminate among options that vary in degree of correctness.
Anatomy of an Item The grid below shows several different areas at a zoo. Which ordered pair shows where the washroom is located? A. (2, 6) B. (3, 5) C. (5, 3) D. (6, 2) Standard: The outcome to be measured by the item. Directions: Clearly written, unambiguous instructions that direct students on how to respond. For example, include details from the text to support your answer, or write your answer on the lines in the test booklet. Stimulus: The contextual situation, graph, map, or passage students will be asked to respond to in the test item. Not required for all items. Stem: The question part of the item. Correct Response: The clear answer to the question. Distractors or Incorrect Answer Choices: For grades 3-12, the three incorrect responses that may represent common misconceptions or errors that students might make if they have not mastered the objective that is being measured. For grades 1-2, only three total answer choices, including the correct response. 3 Distractor Rationales
What is a Distractor? Incorrect answer choices are known as distractors Usually common errors students make or misconceptions that students may have if they have not reached the level of understanding needed to correctly answer the question The rationale is the explanation of what the student does not understand about the standard being assessed that prevented them from answering the question correctly. Importance of Distractors Wrong answer can inform instruction Find out about our students misconceptions Teach critical thinking Provide immediate feedback to the teacher EXAMPLE Which word best describes how Joanne feels at the BEGINNING of the story? A. regretful B. excited C. cautious D. Unsure A. Incorrect. Joanne does not become regretful until the middle of the story. B. Incorrect. She is too nervous to be excited. C. Correct. D. Incorrect. She states that she knows exactly what is supposed to happen. 4 Distractor Rationales
Process and Content Multiple choice items evaluate either procedural understanding or knowledge of the content. Distractor rationales should match the question type, therefore rationales can be either procedural or content oriented based on the question format. Process Example Assess2Know A botanist measures the height of a bamboo plant over a period of several weeks. She notices that the height, in centimeters, of the plant each week (w) can be modeled by the expression 7w 3, where w > 0. What is the height of the plant at the end of week 6? A. 24 cm This answer subtracts 3 from 7 and then calculates 4w. B. 39 cm correct answer C. 42 cm This answer multiplies 7 by 6, but does not subtract 3 from the result. D. 45 cm - This answer multiplies 7 by 6, and then adds 3 instead of subtracting 3. 5 Distractor Rationales
Content Example released Florida Item An understanding of moon phases is needed to answer this question. The picture shows a model of the Moon between Earth and the Sun; therefore, the lit side of the Moon is facing away from an observer on Earth. This arrangement is seen as a new moon. A. A full moon would be seen if Earth were in the middle position of this model (between the Moon and the Sun), so that the entire lit side of the Moon could be seen by an observer on Earth. B. The correct answer is G (new moon). C. A first-quarter moon would be seen if the Moon were to the side of Earth (forming a right angle of Sun-Earth-Moon) in this model, so that only half of the lit side could be seen by an observer on Earth. D. A third-quarter moon would be seen if the Moon were to the side of Earth (forming a right angle of Sun-Earth-Moon) in this model, so that only half of the lit side could be seen by an observer on Earth. Strand: E Earth and Space Benchmark: SC.E.1.2.2 The student knows that the combination of the Earth s movement and the moon s own orbit around the Earth results in the appearance of cyclical phases of the moon. Sample 1 6 Distractor Rationales
Sample 2 Sample 3 7 Distractor Rationales