WRITING CR ITEMS. Anjuran: Unit Hal Ehwal Kurikulum BHEA

Size: px

Start display at page:

Download "WRITING CR ITEMS. Anjuran: Unit Hal Ehwal Kurikulum BHEA"

Shavonne Crawford
5 years ago
Views:

1 Anjuran: Unit Hal Ehwal Kurikulum WRITING CR ITEMS

2 PART I: WHAT IS A TEST ITEM? The test item has the following key points: Unit of measurement A stimulus and a prescriptive form for answering The response is interpreted in terms of learning about examinees performance in a particular psychological construct.

3 PART II: WHAT IS A CONSTRUCTED RESPONSE (CR) ITEM? It is also known as: Free response Essay question Fill-in-the-blank

4 Example of a CR item STIMULUS But wherefore says she not she is unjust? And wherefore say I not that I am old? O, love s best habit is in seeming trust, And age in love loves not to have years told: Therefore I lie with her and she with me, And in our faults by lies we flattered be. STEM Describe the poetic style evident in the lines above.

5 WHAT IS A RUBRIC? A Scoring Guide Should be clear, specific and include all requirements of the task Provides the validity and reliability for the grading process and the score

6 Component Part Sub Criteria Marks Describing Idea Fails to describe Describes the ideas Describes the ideas Clearly describes the ideas fully. but mumbles. the ideas Conclusion No conclusions Conclusions are The conclusion are The conclusions are stated but drawn and drawn and supporting described. described. Report information is not strong as 4 and 3. Delivering Sentence Fails to deliver correctly The delivery and sentence structure are generally correct. The delivery and sentence structure are generally correct. The delivery is engaging and sentence structure is consistently correct. COMMUNICATION SKILLS Presentation Eye Contact Ignore of the eye contact with audiences Presentation Tools Chalk and talk only. Eye contact to audience can be improved. Only using slide of over head projector (OHP). Concern of eye contact with the audiences Eye contact is made and sustained throughout the presentation. Just presented with Present with Microsoft multimedia PowerPoint only software that present with limited enhanced use of multimedia presentation Confidence No confidence Lack of confidence The confidence level sometime gone wobbly/unstable/s haky Body language bad uses of body postures and movements, inert Suitable uses of body postures and movements, shaky Good uses of body postures and movements, Presentation is full with confidence Excellent uses of body postures and movements, smooth PEMBINAAN and boring all ITEM the time SOALAN discontinue PEPERIKSAAN a little and convincing AKHIR DAN PENAMBAHBAIKAN bit PENYAMPAIAN TOTAL (28)

7 PART III: MISCONCEPTIONS ABOUT CR Can test HOTS than MCQ It is easier to prevent and catch cheating with CR items Examinees have better chances to show what they know CR tests thinking while MC tests memorization CR items are easier and less expensive to write than MCQ

8 Good things about CR Items Responses reflect real-life skills more closely Relatively easy to construct and print Provide examinees multiple ways to answer a question Test skills that cannot be assessed by MCQ items Hard to guess the right response Provide more information about examinees than MCQ

9 Bad things about CR Items Scoring may be difficult, and time consuming Testing time may only allow for a few questions Scoring may be seen as subjective Fairness issues may be more common Language skills may affect results Examinees will usually remember a few CR questions more easily than MCQ questions MCQ may work just as well high correlation between MCQ and CR scores.

10 PART IV: TYPES OF CR ITEMS Type 1: Short Answer Name the bird that lays the largest eggs? Define photosynthesis. Identify the number of dozens in a gross? Identify the third Agong of Malaysia.

11 Types of Multiple Choice Items Type 2 Paragraph Describe with examples, several ways to reduce air pollution. Illustrate the 2 steps we can take to reduce air pollution from automobiles.

12 Types of Multiple Choice Items Type 2 Paragraph Describe with examples, several ways to reduce air pollution. Illustrate the 2 steps we can take to reduce air pollution from automobiles.

13 Types of Multiple Choice Items Type 2 Paragraph Describe with examples, several ways to reduce air pollution. Illustrate the 2 steps we can take to reduce air pollution from automobiles.

14 Types of Multiple Choice Items Type 2 Paragraph Describe with examples, several ways to reduce air pollution. Illustrate the 2 steps we can take to reduce air pollution from automobiles.

15 Type 3 Essay: Document Based Questions Read the attached speech given by Tun Mahathir. Outline the major points of his address and compare them with the Look East Policy that he mooted the year before.

16 Type 4 Essay: Thesis and Theme Stimulus: Before a person can be a great hero, they have to experience suffering and overcome great challenges. Stem: Using examples from three works of literature, present an argument for or against this statement.

17 Type 5 Essay: Explaining a Big Idea Describe how the various organs of the digestive system work together to ensure that an organism obtains appropriate nutrition.

18 Type 6 Mathematics Prove by induction or any other method that n n3, where n is a positive integer, is divisible by 3.

19 Type 7 Mathematics In the figure to the right, the center of the regular octagon is at the origin. a) For how many angles will a rotation of the octagon by an angle map the octagon onto itself? Explain how you arrived at your answer. b) How many reflections of the octagon will map the octagon onto itself? Explain how you arrived at your answer.

20 Item Analysed Quantitatively: Good Item

21 QUESTION Analysing CR items qualitatively 1 Does the question clearly meet the test specifications? 2 Is it worded as clearly as possible, or are multiple interpretations are possible? 3 Are the question and timing appropriate for the intended level? 4 Is the content fair for all students? 5 Is an essay the best way to test this information? Document your comments and any recommendations for improving the question.

22 PART VI: Developing rubrics How will work be graded? Holistic: overall impression Analytical: Specific Skills or Information Holistic or Analytic?

23 Component Part Sub Criteria Marks Describing Idea Fails to describe Describes the ideas Describes the ideas Clearly describes the ideas fully. but mumbles. the ideas Conclusion No conclusions Conclusions are The conclusion are The conclusions are stated but drawn and drawn and supporting described. described. Report information is not strong as 4 and 3. Delivering Sentence Fails to deliver correctly The delivery and sentence structure are generally correct. The delivery and sentence structure are generally correct. The delivery is engaging and sentence structure is consistently correct. COMMUNICATION SKILLS Presentation Eye Contact Ignore of the eye contact with audiences Presentation Tools Chalk and talk only. Eye contact to audience can be improved. Only using slide of over head projector (OHP). Concern of eye contact with the audiences Eye contact is made and sustained throughout the presentation. Just presented with Present with Microsoft multimedia PowerPoint only software that present with limited enhanced use of multimedia presentation Confidence No confidence Lack of confidence The confidence level sometime gone wobbly/unstable/s haky Body language bad uses of body postures and movements, inert Suitable uses of body postures and movements, shaky Good uses of body postures and movements, Presentation is full with confidence Excellent uses of body postures and movements, smooth PEMBINAAN and boring all ITEM the time SOALAN discontinue PEPERIKSAAN a little and convincing AKHIR DAN PENAMBAHBAIKAN bit PENYAMPAIAN TOTAL (28)

24 Descriptor 1 Answer not relevant 2 Demonstrates some competence 3 Demonstrates competence 4 Demonstrates superior competence 5 Demonstrates outstanding competence

25 PART VII: CONCLUSION A good item construction increases: a) Discrimination power b) Score variance c) Validity d) Reliability In addition, it ensures: e) Fairness, and f) Consistency

26 DISCRIMINATION POWER Ability to separate poor, average and good examinees. A desirable quality of a test consists of easy, moderate and difficult items

27 SCORE VARIANCE The span (statistical variance) of scores among examinees. The more varied, the better.

28 VALIDITY The degree to which evidence supports scorebased interpretation and decisions (American Educational Research Association, 1999). During item construction, maximize the intensity of evidence i.e., the alignment of each item to the CLO(s). Hence, scores indicate students attainment of CLOs. This is the very core of the OBE concept.

29 RELIABILITY The precision of test scores. They contain little measurement errors. A test that has discrimination power and a large score variance increases the reliability of scores. During item construction, the degree of reliability can be maximized by ensuring there are easy, moderate and difficult items as stipulated in the JSU blueprint.

30 FAIRNESS The F2F hours of teaching and learning a CLO is mirrored in the total scores. The weightage for easy, moderate and difficult items are fairly distributed according to the JSU blueprint. Fairness also addresses bias, upsetting or offensive content. The rubrics used to mark examinees scripts are based on assessment criteria that ensure fairness.

31 CONSISTENCY All final exam papers are consistent in terms of level of difficulty (weightage for easy, moderate and difficult): At multiple locations. Every semester until the next curriculum review. Also, the marking of examinees scripts using rubrics ensure the consistency of marks awarded according to the criteria.

32 For further information, please refer to the attached article: Evidence-centered Design (ECD) Reference: Dr. Luis Saldivia handouts Thank you

33 THANK YOU

1. Stating the purpose of the speech is the first step in creating a speech.

2 From A to Z: Overview of a Speech True/False Questions 1. Stating the purpose of the speech is the first step in creating a speech. 2. Audience analysis involves making random guesses about how the audience