Caribbean Examinations Council

Transcription

1 Caribbean Examinations Council Caribbean Primary Exit Assessment Item Writing Workshop Facilitator: Ms Benita Byer Assistant Registrar Examinations Development and Production Division Benita Byer, CXC 1

2 Can MC items really test a range of cognitive skills? Testing a Range of Cognitive Skills Evaluation Synthesis Analysis Application Comprehension Knowledge Benita Byer, CXC 2

3 Level 1 (Knowledge & Comprehension) A square is BEST described as a quadrilateral with (A) (B) (C) all sides equal two pairs of equal and parallel sides four equal sides and four right angles Benita Byer, CXC 3

4 Level 1 (Knowledge & Comprehension) The thermometer is used to measure (A) (B) (C) length climate temperature Benita Byer, CXC 4

5 Level 2 (Application) The area of a square is 25cm 2. What is its perimeter? (A) (B) (C) 5 cm 20 cm 100 cm Benita Byer, CXC 5

6 Level 2 (Application) The function of the part of the flower labeled 3 is to (A) attract insects (B) produce pollen (C) develop into seeds Benita Byer, CXC 6

7 Level 3 (Analysis & Synthesis) Diagram A shows square floor of length 8 m. Diagram B shows a tile in the shape of a right angled triangle with a base and height of 50 cm. How many of these tiles are needed to completely cover the floor? A B (A) 256 (B) 400 (C) 512 Benita Byer, CXC 7

8 Level 3 (Analysis & Synthesis) The table below gives the stopping distance for a car in different conditions. How much further is the stopping distance at 70 mph in wet conditions than in dry conditions? Speed (mph) Stopping distance (m) Dry Roads Wet Roads (A) (B) (C) 76 m 96 m 172 m Benita Byer, CXC 8

9 Structure of a Multiple-choice Item A multiple - choice item consists of the STEM (statement/question/stimulus) OPTIONS (possible responses) KEY (correct response) DISTRACTORS/FOILS (incorrect responses) Benita Byer, CXC 9

10 Types of MC Items There are three main types of MC Items. 1.Direct questions- (a) Correct answer (b) Best answer 2. Incomplete statements 3. Stimulus material followed by a series of questions Benita Byer, CXC 10

11 1(a) Correct answer variety What is the highest common factor of 24 and 36? A. 4 B.* 12 C. 24 D. 72 Benita Byer, CXC 11

12 1 (b) Best answer variety An equilateral triangle is BEST described as a triangle with A. three sides B. * three equal sides C. equal angles and sides D. a pair of equal sides and angles Benita Byer, CXC 12

13 1 (b) Best answer variety Which material is the best heat conductor? (A) Plastic (B) Wood (C) * Metal Benita Byer, CXC 13

14 2. Incomplete statement The number 762 is divisible by A. * 3 B. 4 C. 7 D. 12 Benita Byer, CXC 14

15 Incomplete statement In science we say that work is done when a force (A) moves something through a distance (B) pushes against a stationary object as a wall (C) is used to hold a weight at a fixed height above the ground Benita Byer, CXC 15

16 3. Multiple response Which of the following describe a cube? I II III IV It has six faces. It has ten edges. It has eight vertices. It is three dimensional. A I and II only B I and IV only C II and III only D* I, III and IV only Benita Byer, CXC 16

17 Advantages of Multiple-Choice Items Enable sampling of a wide content area within a relatively short testing time Flexible several questions may be based on the same stimulus Versatile objectives can be measured across taxonomic range Benita Byer, CXC 17

18 Advantages of Multiple-Choice Items Pre - testing of items allows for prediction and control of test difficulty and reliability Writing time is eliminated and time used for thinking instead Benita Byer, CXC 18

19 Advantages of Multiple-Choice Items Scoring is easy and reliable as keys are determined prior to scoring Item analysis provides the opportunity to examine candidates responses to each item and so identify areas of weakness Benita Byer, CXC 19

20 Guidelines for Writing the STEM Present a single, clearly formulated question, statement or scenario. Use a simple and direct sentence structure. Avoid ambiguous words or phrases. Benita Byer, CXC 20

21 Writing the STEM Avoid excessive verbiage. State the stem in a positive form wherever possible Never use double negatives. If negative words (not, least, etc) are used they should be capitalized. Benita Byer, CXC 21

22 Example Poor: All of the following are NOT prime numbers EXCEPT A. 9 B. 15 C. 21 D. * 29 Better: Which one of the following is a prime number? A. 9 B. 15 C. 21 D. * 29 Benita Byer, CXC 22

23 Writing the STEM Avoid use of personal pronouns (you, we) and especially gender-specific pronouns (he, she) Include any words in the stem that would have to be repeated in each option Benita Byer, CXC 23

24 Example POOR: A prime number is A. a number with no factors B. a number with at least two factors C.* a number with exactly two factors D. a number with fewer than two factors Benita Byer, CXC 24

25 Example BETTER: A prime number is a number with A. no unique factors B. at least two unique factors C.* exactly two unique factors D. fewer than two unique factors Benita Byer, CXC 25

26 Writing the STEM Avoid phrases like what would you do? or what do you think? Avoid situations/content that might be outdated at the time of the exam Ensure that the stem does not contain any clues to the correct answer Benita Byer, CXC 26

27 Writing the STEM Information in the stem of item X should not contain information needed to respond to item Z. Provide as much of the information as possible in the stem Benita Byer, CXC 27

28 Example POOR Which of the following statements is NOT true? A. A square has four equal sides. B. A square has four right angles. C. The diagonals of a square are equal. D. * A square has two lines of symmetry. Benita Byer, CXC 28

29 Example BETTER: Which of the following statements about a square is NOT true? A. It has four equal sides. B. It has four right angles. C. The diagonals are equal. D. * It has two lines of symmetry. Benita Byer, CXC 29

30 Guidelines for Writing the OPTIONS Should follow logically and grammatically from the stem Should be arranged in a logical, chronological or systematic order Benita Byer, CXC 30

31 Example POOR The circumference of a circle is the A. an amount that is half the diameter B. if you measure the distance from the centre to a point on the circle C.* actual distance around the circle D. when a line cuts a circle in half Benita Byer, CXC 31

32 Example BETTER The circumference of a circle is A. half the diameter B. the distance from the centre to a point on the circle C.* the actual distance around the circle D. a line that cuts a circle in half Benita Byer, CXC 32

33 Example POOR If 20% of a number is 80, what is the number? A B. 4 C. * 400 D. 16 Benita Byer, CXC 33

34 Example BETTER If 20% of a number is 80, what is the number? A. 4 B. 16 C. * 400 D Benita Byer, CXC 34

35 Writing the OPTIONS They should be parallel in length grammatical structure content & terminology Benita Byer, CXC 35

36 Example A trapezium A. has no obtuse angles B.* is a quadrilateral with one pair of parallel sides C. is symmetrical D. has four lines Benita Byer, CXC 36

37 Writing the OPTIONS Use professionally acceptable or well-known technical terms in options (distractors as well as the key) Avoid use of ALL OF THE ABOVE & NONE OF THE ABOVE, especially with the Best answer variety Include ONLY ONE correct (or best) option (i.e., the key) Benita Byer, CXC 37

38 Writing the OPTIONS Key should be either the ONLY correct answer or clearly the BEST or MOST APPROPRIATE answer to the informed, well-prepared candidate. Key should be correct and defensible on professional and (if necessary) legal grounds. Benita Byer, CXC 38

39 Guidelines for Writing the DISTRACTORS Should be plausible Should be parallel with the correct answer in length, structure, vocabulary Should provide no unintended clues to the correct answer Benita Byer, CXC 39

40 Writing the DISTRACTORS Should not be too similar in meaning to the correct answer Should not contain unnecessarily technical language Benita Byer, CXC 40

41 Writing the DISTRACTORS Should not overlap with each other Should be free of humorous and ridiculous statements Use common misconceptions and common errors Benita Byer, CXC 41

42 Writing the DISTRACTORS Use statements that are true but do not satisfy the requirements of the problem too broad or too narrow for the requirements of the problem carefully worded but incorrect, and may seem plausible to the uninformed Benita Byer, CXC 42

43 DO NOT Writing the DISTRACTORS Make the distractors vague and ambiguous while the key is clear and concise. DO NOT Use absolutes (no, all, never, always) in distractors but not in the key. Benita Byer, CXC 43

44 The End Benita Byer, CXC 44