Challngs in Educational Masurmnt Contnt, Mthods and Consquncs Gothnburg, 12 Oct. 2016 National Assssmnt in Swdn A multi-dimnsional (ad)vntur Gudrun Erickson Univrsity of Gothnburg Dpt. of Education and Spcial Education
Only Swdn has Swdish Goosbrris Carl Jonas Lov Almqvist (1838)
Outlin Contxt Discussions and Dilmmas: Aims Construct(s) Mthods Agncy Uss & Consquncs Ongoing activitis Prospcts and Challngs
Background national assssmnt Long tradition of assssmnt at th national lvl; high dgr of accptanc; Sinc th 1980s, diffrnt univrsitis commissiond by th National Agncy for Education to b rsponsibl for tst dvlopmnt; Currntly, an xtnsiv systm with formativ as wll as summativ matrials; No xams; national tst rsults hav an advisory function and ar to b combind with tachrs continuous obsrvations; Tachrs mark thir own studnts tsts, and award final grads; co-rating strongly rcommndd but not mandatory.
Discussions and Dilmmas Initially, grat caution not to intrfr at th local lvl; schools and tachrs givn vry much rsponsibility; Far of too much advic and support, i.. that national tsts would b prcivd and tratd as xams; A growing numbr of obsrvations and studis indicatd considrabl diffrncs in handling national tsts and in awarding grads; issus of fairnss and quity raisd; Far of national tsts mayb providing too littl advic and support, i.. national tsts not important nough; What is lagom advic and support? (lagom = not too much, not too littl)
Th Qustion of Aim(s) For a numbr of yars, th aims of th Swdish national tsts wr to - nhanc ducational achivmnt - concrtiz standards - clarify goals and indicat strngths and waknsss in individual larnr profils - nhanc quity in assssmnt and grading - provid vidnc for local and national analyss of ducational achivmnt Sinc 2008, only th last two with th first two as possibl ffcts A rcnt national inquiry suggsts a sparation of th two rmaining aims
Th Qustion of Construct Individual subjct syllabuss dfin th construct; Contnt standards (Aim of subjct Goals Cntral contnt ) and Prformanc standards; Prformanc standards similar across subjcts, with gnric valu words to distinguish btwn lvls; A rcnt study by th National Agncy for Education rvals considrabl diffrncs btwn syllabuss rgarding, for xampl, amount of contnt in th prformanc standards; A majority of tachrs do not rgard th prformanc standards clar nough to support grading in a satisfactory way; rvisions suggstd; A thrshold rul offrs no possibility to compnsat for waknsss with strngths; all aspcts of th prformanc standards hav to b mt; Urgnt nd for discussions about th ffct of digitalization on constructs and thus syllabuss.
Th Qustion of Mthod Exampls of issus Format: Constructd rspons vs. Slctd rspons Can rasoning b masurd with multipl choic? Points for scoring: Should points b usd? If ys, what typ? If wightd (diffrntiatd), according to what principls, and how rliabl? Anchor itms: Should anchor itms b usd? If ys, typ and numbr? Data analysis: Classical or mor modrn mthods? IRT? If ys, typ? Qualitativ analyss: which and how? Aggrgation: Should aggrgation b mad? If ys, of what and how? Standard stting: Mthod? Participants Which? How many? Rporting rsults: Lvl of dtail? How? To whom?
Th Qustion of Agncy Exampls of issus Tachrs marking thir own studnts national tsts: rasons and ffcts; Co-marking Extrnal marking Studnts rols and rights; Th issu of dlgation: th rsponsibility and rol of th univrsitis commissiond to b in charg of tst dvlopmnt.
Uss and Consquncs (1) Exampls of issus and rsults Consistnt diffrncs btwn subjcts rgarding rsults du to construct, contxt, curricula, taching/tachrs/taching matrials, tsts? Exampl: Aggrgatd national tst scors, yar 9 (2015) % F E D C B A English 4 11 15 28 22 20 Swdish 4 20 24 29 16 6 Mathmatics 19 31 18 17 9 6 Chmistry 7 22 22 23 15 12 History 17 29 27 16 9 2
Uss and Consquncs (2) Exampls of issus and rsults Tachrs pay diffrnt attntion to national tst rsults in thir final grading what is lagom advic and support? Exampl: Rlation btwn aggrgatd tst grads (ATG) and final subjct grads (FSG), yar 9; spring 2015 FSG<ATG FSG=ATG FSG>ATG English 15% 74 % 11 % Swdish 9 % 64 % 27 % Mathmatics 2 % 60 % 38 % Chmistry 15 % 65 % 20 % History 2 % 40 % 58 %
Uss and Consquncs (3) Exampls of issus and rsults National tst rsults vary ovr tim, within and btwn subjcts; tachrs grads sm mor stabl
A Common Framwork for National Tsts Prparatory work during a numbr of yars; Currnt assignmnt by th National Agncy for Education to a small working group (collaborating with rfrnc groups): Thortically foundd quality assuranc Thortical + practical part Main targt groups: th NAE and th univrsity institutions dvloping th tsts (indirctly also othr stakholdrs); Th Framwork intndd to form th basis for th diffrnt tst dvlopmnt groups to dvlop thir subjct spcific spcifications; clarity and transparncy mphasizd; To b dlivrd to th NAE in Dcmbr 2016; howvr, dpnding on political dcisions about th national assssmnt systm at larg.
Rcnt national inquiry rgarding th national tsts for lowr and uppr scondary school Main suggstions A distinct aim for ach part of th national assssmnt systm: National tsts National assssmnt support matrials National valuation A rducd numbr of national tsts Gradual digitalisation of th national tsts; xtnsiv piloting Piloting of modls for co-rating and xtrnal rating A clarr rlationship btwn national tst rsults and grads Incrasd quality and stability ovr tim SOU 2016:25: Equivalnt, fair and fficint a nw systm for national assssmnt
Prospcts and Challngs Building on, maintaining and furthr dvlop th positiv attituds to, and (intndd/assumd) washback of, th national assssmnt systm; Incrasing stability, thrby contributing to validity and rliability fairnss and quity ; Enabling systmatic studis of dvlopmnt ovr tim; Furthr dvloping and laborating mthods of collaboration with broad groups of stakholdrs, and btwn policy, practic and rsarch.
Thank you! gudrun.rickson@pd.gu.s