Pinball Car Race Online Task

Transcription

1 2014 Pinball Car Race Online Task Item Screenshots & Rubrics Performance-based Science Assessment Task Technology-enhanced Aligned to Next Generation Science Standards Evidence Centered Designed Scenario-based SRI Education Division Assessment Team Daisy Rutstein Ron Fried Reina Fujii Geneva Haertel Terry Vendlinski Cindy Ziker Patricia Schank Amy Hafter Michelle Huynh Copyright 2014 SRI International

2 Contents Background... 2 Orientation to Pinball Car Race (Not Scored)... 3 Item Rubric for Item Student Selection of Variables (Not Scored)... 5 Item 2 (Not Scored)... 6 Item Rubric for Item Item 4 (only scored if item 9 is not completed)... 8 Rubric for Item Item 5 (only scored if item 10 is not completed)... 8 Rubric for Item Experiment 1 Trials (Not Scored) Presentation of Experiment 1 Results (Not Scored) Item 6 (Parts a and b are scored together; only scored if item 11 is not completed) Rubric for Items 6a and 6b Item 7 (Not Scored) Item 8 (Not Scored) Item Rubric for Item Item Rubric for Item Experiment 2 Trials (Not Scored) Item 11 (Parts a and b are scored together) Rubric for Items 11a and 11b Item 12a Rubric for Item 12a Item 12b Rubric for Item 12b Task Complete Page 1

3 Pinball Car Race Online Task Items and Scoring Rubrics Background The Pinball Car Race is a middle school science assessment task designed to test a student s knowledge of science content and science practices, including scientific reasoning and the process skills involved in designing and interpreting results of an experimental investigation. An Evidence Centered Design (ECD) approach was used to develop this task, which focuses on knowledge of forms of energy in the physical sciences, specifically, potential and kinetic energy. The student selects a hypothesis and manipulates the variable of interest, while controlling the other variables, to test the hypothesis, interpret the results presented and draw conclusions. For formative purposes, scores on individual parts of the task may be used to provide diagnostic evidence of a student s ability to draw conclusions about an experiment. The Pinball task provides a student with two attempts to design an experiment. Scores may be calculated to diagnose the level of skill a student demonstrates on each of the two attempts, providing an opportunity to determine whether student performance changes. For summative purposes, the scores of item 1, items 3-6, and items 9-12 would be summed to provide an overall score related to a student s experimental investigation ability in the context of energy transfer. For formative purposes, scores on individual parts of the task may be used to provide evidence of a student s performance at a finer grain size. For example, the item associated with posing a hypothesis would provide evidence of the student s ability to perform that particular skill. Other items in the Pinball Car Race task might be used individually to diagnose a student s ability to draw conclusions about an experiment. In addition, the Pinball task provides a student with two attempts to design an experiment. Scores may be calculated to diagnose the level of skill a student demonstrates on each of the two attempts providing an opportunity to determine whether a student s performance changes. Thus, there are several ways to score the Pinball Car Race task depending on the intended purpose of the assessment. This document contains information about the individual items included in the technology-enhanced task. A screenshot and rubric are presented for each item. See URL for Pinball Car Race online task: The final set of items presented in this document were identified using data from the paper/pencil pilot study presented in the Pinball Car Race Pilot Task: Items, Rubrics, Student Work Samples, & Item Statistics document. Page 2

4 Orientation to Pinball Car Race (Not Scored) This page introduces the idea of a Pinball Car Race. The student is able to play the video to watch the car race. Page 3

5 Item 1 On this page, the student plays a video of the start of the Pinball Car Race and is asked to describe the transfer of energy from the spring to the car. Rubric for Item 1 Student should indicate that when the spring hits the car, the potential energy becomes the car s kinetic energy. When the spring comes to a rest, it no longer has potential or kinetic energy. Student may also indicate that the car loses kinetic energy as it slows down. 2 points 1 point 0 points Student indicates that energy is transferred from the spring to the car when there is contact between the two AND that the moving car has kinetic energy. Student indicates that energy is transferred from the spring to the car when there is contact between the two OR that the moving car has kinetic energy. Student s response NEITHER indicates that energy is transferred from the spring to the car when there is contact between the two NOR indicates that the moving car has kinetic energy. Page 4

6 Student Selection of Variables (Not Scored) This page introduces the student to the idea of the experiment he or she will conduct. The student is asked to do an experiment to test which spring characteristics would help the car go farthest in the Pinball Car Race. The student is able to click on the characteristics of the spring to see examples of what each spring would look like, given the choices he or she selects. Page 5

7 Item 2 (Not Scored) Here, a student picks one of the characteristics to use in her or his hypothesis. The student explains why the characteristic was selected in terms of how it would affect the potential energy of the spring. The student is not scored on item 2, as any hypothesis is acceptable. Page 6

8 Item 3 Once a student has picked a hypothesis, he or she is asked to relate the characteristic of the spring in the hypothesis to the potential energy of the spring. The student is allowed to go back and change his or her hypothesis. Information from item 2 will be used in scoring item 3. Rubric for Item 3 The student should state how the characteristic of the spring he or she selected would increase or decrease the potential energy of the spring. The student response should match the student s hypothesis. For example, if the student hypothesizes that the car would travel farther with springs that have more coils, then the student should indicate that the greater the number of coils, the more potential energy the spring could store. Note that this item assesses a student s understanding of how the hypothesis relates to the experiment he or she is conducting. It is not focused on the specific science content of the hypothesis. The student s explanation matches his or her hypothesis AND indicates that he or she 2 points understands the cause and effect relationship between the car and the spring. This explanation must go beyond just restating the hypothesis. The student s explanation matches the hypothesis but does not go beyond just restating 1 point the hypothesis. 0 points The student s explanation does not match the hypothesis. Page 7

9 Item 4 (only scored if item 9 is not completed) In item 4, the student is asked to select settings for each of the springs. These settings allow the student to test his or her hypothesis. The hypothesis that the student selected is shown as a reminder. If the student completes item 9, the score on item 9 will replace the score from this item. Rubric for Item 4 Scores will reflect whether the selected settings are appropriate to the hypothesis. A student should be using different settings for the independent variable and should keep the control variable settings the same in all three trials. The independent variable and control variable should match the student s hypothesis. The independent variable is the variable that the student included in the hypothesis; the control variable is the variable the student will need to hold constant. The student varies the appropriate variable (i.e., the variable that matches the student s 2 points hypothesis) AND keeps the other variable constant. The student varies both variables OR the student varies the incorrect variable (i.e., the 1 point variable that does not match the student hypothesis) but keeps the other variable constant. 0 points The student keeps both variables constant. Item 5 (only scored if item 10 is not completed) In this item, the student explains the reason for choosing the spring settings. The student s hypothesis selected in item 2 is not considered when scoring this item. The student s explanation is an indication of Page 8

10 whether he or she understands the concept of a control variable and an independent variable. If the student completes item 10, the score on item 10 will replace the score from this item. Page 9

11 Rubric for Item 5 A student response should indicate why the use of different settings for the independent variable allows the student to determine what would happen under different conditions. The student response should also indicate the need to keep the control variable constant in all three cases to ensure that any differences in the hypothesized outcome are not due to variation in the control variable. The student response must indicate that the student has knowledge of independent and control variables. 2 points 1 point 0 points A student s response indicates that the use of different settings for the independent variable allows the student to observe what would happen under different conditions, AND a student s response indicates that keeping the control variable the same for all three springs isolates the effect of the independent variable. Explanations must go beyond just restating the settings. A student s response indicates that the use of different settings for the independent variable allows the student to observe what would happen under different conditions OR a student s response indicates that keeping the control variable the same for all three springs isolates the effect of the independent variable. Explanations must go beyond just restating the settings. A student s response does not indicate that the use of different settings for the independent variable allows the student to observe what would happen under different conditions AND a student s response does not indicate that keeping the control variable the same for all three springs isolates the effect of the independent variable. Page 10

12 Experiment 1 Trials (Not Scored) On this page, the student is able to press the play button in order to see how far the car moved with the spring he or she chose. Three trials are shown for each spring. The values are automatically populated in the table. Page 11

13 Presentation of Experiment 1 Results (Not Scored) Multiple representations of the data from Experiment 1 are provided to support students who are unfamiliar with graphical and tabular representations. Page 12

14 Item 6 (Parts a and b are scored together; only scored if item 11 is not completed) The student is provided the results of his or her experiment with the selected variables. In 6a, the student determines how the results of the experiment are related to his or her hypothesis. In 6b, the student is required to explain why he or she made this determination. If the student completes item 11, the score on item 11 will replace the score from this item. Page 13

15 Rubric for Items 6a and 6b A student s answers should correctly interpret the results of the experiment. If the experimental design is correct, then the student s selection in item 6a should correctly indicate whether the results support or do not support the hypothesis. The explanation the student provides in item 6b should discuss how the results from the three trials show which spring made the car go the farthest. If the student did not set up an appropriate experiment (for example, if the student varied both variables or the student varied the variable not identified in the hypothesis), then in item 6a, the student should select that the results do not support the hypothesis; in item 6b, the explanation should discuss why it is not possible to determine the validity of the hypothesis. 3 points 2 points 1 point 0 points Student correctly selects support, or do not support based on the hypothesis AND the explanation the student provides appropriately relates the results to the hypothesis. Student incorrectly selects support, or do not support based on the hypothesis BUT provides an explanation that appropriately relates the results to the hypothesis. Student correctly selects support, or do not support based on the hypothesis BUT his or her explanation inappropriately relates the results to the hypothesis. Student incorrectly selects support, or do not support based on the hypothesis AND provides an explanation that inappropriately relates the results to the hypothesis. Page 14

16 Item 7 (Not Scored) In item 7, the hypothesis and trial results from Experiment 1 are presented. The student has an opportunity to revise the previous hypothesis before conducting Experiment 2. Page 15

17 If the student selects Yes (indicating the student wants to revise his or her hypothesis), the student is presented with an opportunity to select a new hypothesis about which characteristic of the spring makes the car go the farthest. If the student selects No, he or she proceeds to item 8. Page 16

18 Item 8 (Not Scored) In item 8, the trial results from Experiment 1 are presented along with the student s current hypothesis, which may or may not be the same as Experiment 1. The student is asked if he or she wants to change the settings for Experiment 2. If the student selects Yes, the student is taken to item 9 where he or she has an opportunity to revise the previous experimental settings before conducting Experiment 2. If the student selects No, the student is taken to item 12. Page 17

19 Item 9 In item 9, the student is asked to select settings for each of the springs for Experiment 2. These settings allow the student to test his or her current hypothesis, which is presented. Note that if the student chose not to change the experiment settings in item 8 (i.e., selected No ), the student will not see this page, and the settings in Experiment 2 will remain the same as they were in Experiment 1. If the student completes item 9, the score from this item replaces the score from item 4. Rubric for Item 9 Scores will reflect whether the selected settings are appropriate to the hypothesis. A student should be using different settings for the independent variable and should keep the control variable settings the same in all three trials. The independent variable and control variable should match the student s hypothesis. The independent variable is the variable that the student included in the hypothesis; the control variable is the variable the student will need to hold constant. The student varies the appropriate variable (i.e., the variable that matches the student s 2 points hypothesis) AND keeps the other variable constant. The student varies both variables OR the student varies the incorrect variable (i.e., the 1 point variable that does not match the student hypothesis) but keeps the other variable constant. 0 points The student keeps both variables constant. Page 18

20 Item 10 In this item, the student explains the reason for choosing the spring settings in item 9. The student s current hypothesis is not considered when scoring this item. The student s explanation is an indication of whether he or she understands the concept of a control variable and an independent variable. If the student completes item 10, the score from this item replaces the score from item 5. Page 19

21 Rubric for Item 10 A student response should indicate why the use of different settings for the independent variable allows the student to determine what would happen under different conditions. The student response should also indicate the need to keep the control variable constant in all three cases to ensure that any differences in the hypothesized outcome are not due to variation in the control variable. The student response must indicate that the student has knowledge of independent and control variables. 2 points 1 point 0 points A student s response indicates that the use of different settings for the independent variable allows the student to observe what would happen under different conditions, AND a student s response indicates that keeping the control variable the same for all three springs isolates the effect of the independent variable. Explanations must go beyond just restating the settings. A student s response indicates that the use of different settings for the independent variable allows the student to observe what would happen under different conditions OR a student s response indicates that keeping the control variable the same for all three springs isolates the effect of the independent variable. Explanations must go beyond just restating the settings. A student s response does not indicate that the use of different settings for the independent variable allows the student to observe what would happen under different conditions AND a student s response does not indicate that keeping the control variable the same for all three springs isolates the effect of the independent variable. Page 20

22 Experiment 2 Trials (Not Scored) On this page, the student is able to press the play button in order to see how far the car moved with the spring he or she chose. Three trials are shown for each spring. The values are automatically populated in the table. Page 21

23 Item 11 (Parts a and b are scored together) The student is provided the results of his or her experiment with the selected variables. In 11a, the student determines how the results of the experiment are related to his or her hypothesis. In 11b, the student is required to explain why he or she made this determination. If the student completes item 11, the score from this item replaces the score from item 6. Page 22

24 Rubric for Items 11a and 11b A student s answers should correctly interpret the results of Experiment 2. If the experimental design is correct, then the student s selection in item 11a should correctly indicate whether the results support or do not support the hypothesis. The explanation the student provides in item 11b should discuss how the results from the three trials show which spring made the car go the farthest. If the student did not set up an appropriate experiment (for example, if the student varied both variables or the student varied the variable not identified in the hypothesis), then in item 11a, the student should select that the results do not support the hypothesis; in item 11b, the explanation should discuss why it is not possible to determine the validity of the hypothesis. 3 points 2 points 1 point 0 points Student correctly selects support, or do not support based on the hypothesis AND the explanation the student provides appropriately relates the results to the hypothesis. Student incorrectly selects support, or do not support based on the hypothesis BUT provides an explanation that appropriately relates the results to the hypothesis. Student correctly selects support, or do not support based on the hypothesis BUT his or her explanation inappropriately relates the results to the hypothesis. Student incorrectly selects support, or do not support based on the hypothesis AND provides an explanation that inappropriately relates the results to the hypothesis. Page 23

25 Item 12a Item 12 requires a student to draw conclusions from a set of experimental data. Item 12a requires a student to draw an appropriate conclusion about the relationship between the number of coils and the distance a car travels. Rubric for Item 12a A student s answer should correctly interpret the data relating the number of coils and the distance the car goes. Page 24

26 1 point 0 points Student correctly states that increasing the number of coils decreases the distance the car travels OR student states that decreasing the number of coils increases the distance the car travels. Student incorrectly relates the number of coils to the distance the car travels OR student does not address the relationship between the number of coils and the distance the car travels. Page 25

27 Item 12b Item 12 requires a student to draw conclusions from a set of experimental data. Item 12b requires a student to draw an appropriate conclusion about the relationship between the thickness of the wire and the distance a car travels. Page 26

28 Rubric for Item 12b A student s answer should correctly interpret the data relating the thickness of the wire and the distance the car goes. 1 point 0 points Student correctly states that increasing the thickness of the wire increases the distance the car travels OR student states that decreasing the thickness of the wire decreases the distance the car travels. Student incorrectly relates the thickness of the wire to the distance the car travels OR student does not address the relationship between the thickness of the wire and the distance the car travels. Task Complete Page 27