Key words: Educational measurement, Professional competence, Clinical competence, Physical therapy (specialty)

Transcription

1 Dalton et al: Assessment of professional competence of students The Assessment of Physiotherapy Practice (APP) is a valid measure of professional competence of physiotherapy students: a cross-sectional study with Rasch analysis Megan Dalton,, Megan Davidson and Jenny Keating Griffith University, School of Allied Health, La Trobe University, School of Primary Health Care, Monash University Australia Question: Is the Assessment of Physiotherapy Practice (APP) a valid instrument for the assessment of entry-level competence in physiotherapy students? Design: Cross-sectional study with Rasch analysis of initial (n = 6) and validation samples (n = 8). Students were assessed on completion of, 5, or 6-week clinical placements across one university semester. Participants: 98 clinical educators and 56 physiotherapy students at nine universities in Australia and New Zealand provided 6 completed APP instruments. Results: APP data in both samples showed overall fit to a Rasch model of expected item functioning for interval scale measurement. Item 6 (Written communication) exhibited misfit in both samples, but was retained as an important element of competence. The hierarchy of item difficulty was the same in both samples with items related to professional behaviour and communication the easiest to achieve and items related to clinical reasoning the most difficult. Item difficulty was well targeted to person ability. No Differential Item Functioning was identified, indicating that the scale performed in a comparable way regardless of the student s age, gender or amount of prior clinical experience, and the educator s age, gender, or experience as an educator, or the type of facility, university, or clinical area. The instrument demonstrated unidimensionality confirming the appropriateness of summing the scale scores on each item to provide an overall score of clinical competence and was able to discriminate four levels of professional competence (Person Separation Index =.96). Person ability and raw APP scores had a linear relationship (r =.99). Conclusion: Rasch analysis supports the interpretation that a student s APP score is an indication of their underlying level of professional competence in workplace practice. Journal of Physiotherapy Key words: Educational measurement, Professional competence, Clinical competence, Physical therapy (specialty) Introduction Workplace-based learning and assessment is an essential component of physiotherapy and other health professional education programs. Professional competence includes understanding and dealing with highly variable circumstances and assessment is therefore difficult to standardise across students (Rethans et al ). Controlled assessments such as Objective Structured Clinical Examinations and the use of standardised patients have been developed in response to concerns regarding standardised and reliable measurement of student competencies. While assessment reliability may be enhanced by standardised testing, the validity of controlled examination procedures has been challenged because competence under controlled conditions may not be an adequate surrogate for performance under the complex and uncertain conditions encountered in usual practice (Southgate et al ). A solution to this complexity is to monitor students over a sufficient period of time to enable observation of practice in a range of circumstances and across a spectrum of patient types and needs. This has been argued as superior to one-off exit style examinations (van der Vleuten ). Longitudinal assessment of professional competence of physiotherapy students in the workplace is the assessment approach used within all Australian and New Zealand physiotherapy programs. Clinical educators (registered physiotherapists) generally rate a student s performance on a set of items on completion of a, 5, or 6-week block of supervised workplace practice. If valid interpretations of such scores are to be made, the assessment instrument must be both psychometrically sound and educationally informative (Prescott-Clements et al 8, Streiner and Norman ). These requirements were fundamental considerations in the development and evaluation of the Assessment of Physiotherapy Practice (APP) instrument (Dalton et al 9), which has been adopted in all but one Australian and all New Zealand entry-level programs. The development of the APP was guided by the framework of Wilson (5). An initial item pool was constructed from all available assessment instruments and reduced by removing redundancy and applying criteria related to good What is already known on this topic: Assessment of clinical competence under controlled conditions of practical examinations may not be an adequate surrogate for performance in clinical practice. A standard assessment tool is needed for physiotherapy students on clinical placements. What this study adds: The Assessment of Physiotherapy Practice (APP) is a valid measure of professional competence of physiotherapy students. It is appropriate to sum the scale scores on each item to provide an overall score of clinical competence. The APP performs in a comparable way regardless of the characteristics of the student, the clinical educator, or the clinical placement. Journal of Physiotherapy Vol. 57 Australian Physiotherapy Association 9

2 Research item design. The test content development included input and collaboration from physiotherapy educators across Australia and New Zealand. The iterative cycles included a pilot trial and two field test stages. A detailed description of these stages is presented in Figure (see eaddenda for Figure ). Continuous refinement of the instrument based on qualitative and quantitative evaluation occurred throughout each stage (Coghlan and Brannick ). There were three phases of workplace-based testing a pilot trial and two field tests (Dalton et al 9). This paper reports the results of the second field test. Rasch analysis of data was used at each stage of testing the APP. This statistical model calibrates the difficulty of items and the ability of persons on a common scale with interval-level units called logits (log-odds units) (Bond and Fox 7, Rasch 96). Rasch analysis provides validity evidence based on instrument internal structure. It enables analysis of unidimensionality (considered an essential quality of an additive scale) and the targeting of item difficulty to the persons abilities (Bond and Fox 7). Rasch analysis also enables assessment of the functioning of the rating scale when applied to students with different characteristics (eg, age and gender) or applied by assessors with different characteristics (eg, years of experience as a clinical educator). If data fit a Rasch model, a number of qualities should be evident in the data. Items should present a stable hierarchy of difficulty. It should be easy to achieve high scores on easy items and difficult on hard items, with items in between ranking in a predictable way. An instrument with these properties would make the user confident that a student who achieved a higher total score was able to cope with the more difficult, as well as the easier, challenges. Educators could identify challenging items and appropriate educational support could be developed to help students achieve these more challenging aspects of practice. Further detail on the methods of Rasch analysis and the applicability of its results in the clinical environment is provided in an excellent paper by Tennant and Conaghan (7). The aim of this study was to ascertain whether the APP instrument is a valid measure of professional competence of physiotherapy students when tested using the Rasch measurement model. Therefore the specific research questions were:. Is the APP a unidimensional measure of the professional competence of physiotherapy students?. What is the hierarchy of difficulty of items from easiest to hardest?. Is there any evidence of differential item functioning, which indicates the scale exhibits item bias?. Are the APP items appropriately targeted for the student population? Method This was a cross-sectional study using Rasch analysis of two samples (n = 6 and n = 8). Students were assessed at completion of clinical placements across one university semester in 8. Approval was obtained from the human ethics committee of each participating university. The APP (Version ) used in this final field trial comprised items, presented in Appendix (see the eaddenda for Appendix ). Each of the items has the response options = infrequently/rarely demonstrates performance indicators, = demonstrates few performance indicators to an adequate standard, = demonstrates most performance indicators to an adequate standard, = demonstrates most performance indicators to a good standard, = demonstrates most performance indicators to an excellent standard, and not assessed. A rating of or indicates that a minimum acceptable standard has not been achieved for that item. A global rating scale of overall performance (not adequate, adequate, good, excellent) is also completed by the educator, but this item does not contribute to the APP score. Examples of performance indicators for each item are provided on the reverse of the APP. A total raw score for the APP ranges from to 8, and can be transformed to a to scale by dividing the raw score by the total number of items scored (ie, excluding any items that were not assessed) and multiplying the result by. Participants Students enrolled in entry-level physiotherapy programs from 9 universities in Australia and New Zealand were assessed by educators using the APP on completion of a, 5, or 6-week full-time clinical placement block scheduled across one university semester. The placements occurred during the last 8 months of the students physiotherapy program and represented diverse areas of physiotherapy practice including musculoskeletal, cardiorespiratory, neurological, paediatric, and gerontological physiotherapy. Recruitment procedures optimised representation of physiotherapy clinical educators by location (metropolitan, regional/rural, and remote), clinical area of practice, years of experience as a clinical educator, and organisation (private, public, hospital based, community based, and nongovernment). Field test procedure Prior to commencement of clinical placements, educators and students were sent an information sheet and consent form and invited to participate. Data were excluded from analysis if either the student or their clinical educator did not consent to participate in the research. All clinical educators received training in the use of the APP through attendance at a -hour workshop, access to the APP resource manual, or both. Compulsory workshop attendance for all clinical educators participating in the field test was not feasible in the authentic clinical education environment where face-toface training opportunities are constrained by geographical, workload, and financial considerations. During the trial a member of the research group was available to answer questions by phone or . Students were educated in the assessment process and use of the APP instrument using a standardised presentation prior to placements commencing and information about the APP was included in each university s student clinical education manual. Data management and analysis On completion of each placement the completed APP forms were returned by mail, de-identified, and entered into a spreadsheet. Data were analysed with RUMM software using a partial credit model (Andrich et al ). The analysis tested the overall fit of data to the model, the overall and individual item and person fit, item threshold order, targeting, item difficulty, person separation, differential item functioning, and dimensionality. Journal of Physiotherapy Vol. 57 Australian Physiotherapy Association

3 Dalton et al: Assessment of professional competence of students The Rasch measurement model Conversion of ordinal data to interval level measurement data: The current approach in workplace-based assessment is to score a physiotherapy student s performance on a rating scale across items that sample behaviours considered essential for professional competence. Rating scale options are allocated sequentially ordered integers, and item scores are summed to give a total score. While this approach is common, there is little evidence to support the proposition that ordinal-level total scores approximate interval-level measurements (Cliff and Keats, Streiner and Norman ). Rasch modeling enables the abstraction of equal units of measurement from raw (ordinal data) scores on items of an assessment tool. These can be calibrated and then used with confidence to measure and quantify attributes such as competence in physiotherapy practice (Bond and Fox 7). This conversion facilitates appropriate interpretation of differences between individuals and tallying of converted scores provides interpretable total scores. Functioning of items: In this study the construct of interest was competence to practice physiotherapy. If scores for items fit a Rasch model, a number of qualities should be evident in the data. Items should present a stable hierarchy of difficulty. It should be easy to achieve high scores on some items and difficult on others, with items in-between ranking in a reliable way. An instrument with these properties would make the user confident that a student who achieved a higher total score was able to cope with the more difficult, as well as the easier, challenges. Educators could identify challenging items and appropriate educational support could be developed to help students achieve these more challenging targets. Item bias: A scale that fits a Rasch model should function consistently irrespective of subgroups within the sample being assessed. For example, male and female students with equal levels of the underlying construct being measured should not be scored significantly differently (Lai et al 5). Rasch analysis enables assessment of item bias through investigation of Differential Item Functioning. In the development of the APP, the research team was particularly interested to determine whether the scale performed in a comparable way regardless of the student s age, gender, or the total number of weeks of clinical experience, the educator s age, gender, or experience as an educator, the type of facility where the clinical placement occurred, the university that delivered the student s education, or the clinical area. Dimensionality: One of the primary tenets underpinning Rasch analysis is the concept of unidimensionality. If the scale scores on each item of the APP are to be added together to provide a total score representing an overall level of professional competence, Rasch analysis should indicate a scale that is unidimensional, a scale that measures one construct. Unidimensionality was explored using the independent t-test procedure (Tennant and Pallant 6). Targeting of instrument: It is important, particularly in clinical practice, that the assessment items are appropriately targeted for the population being assessed. Poorly targeted measures result in floor or ceiling effects, and this would mean that either very weak or very strong students may not be graded appropriately. Rasch modeling provides an indication of the match between the item difficulty and the abilities of people in the sample. A well-targeted scale would have a mean person location around zero (Tennant and Conaghan 7). Functioning of the rating scale: Rasch analysis generates a person separation index that provides an indication of the internal consistency of the scale and the power of the instrument to discriminate amongst respondents with different levels of professional competence. A minimum person separation index of.7 and.85 is required for group and individual use respectively (Tennant and Conaghan 7). Rasch analysis also enables investigation of difficulty that clinical educators may have in discriminating between different levels on the rating scale. For a good fit to the model it is expected that for any item, student with high levels of the attribute (professional competence indicated by total scores) would typically achieve a higher item score than individuals with low levels of the attribute. In Rasch analysis this is demonstrated by an ordered set of response thresholds for each item. Ordered thresholds indicate that the respondents (ie, clinical educators) use the response categories (ie, scoring scale) in a manner consistent with the level of the trait (ie, competence) being measured. This occurs when the educators consistently discriminate between response options in a predictable way. Results A total of 6 APP assessments from 56 students were returned by 98 clinical educators. Tables and present the characteristics of the participating students and educators. Table presents the characteristics of the APP forms received. The mean APP total score was 6 (SD, range 6 8). If converted to the scale, this equates to a mean total score of 76 (SD 5, range ). All 5 points on the rating scale were used for the majority of items. Missing data was rare (.% of all data points) and.% of all items were rated as not assessed. Data were randomly divided into two samples. Sample was used for model development (n = 6) and sample for model validation (n = 8). The data were stratified before randomisation to optimise representation of completed APP instruments according to clinical area of the placement, level of student experience, facility type (hospital, nongovernment agency, community health centre, private practice), and university program type (undergraduate, graduate entry). Overall model fit: The item-trait interaction chi-square statistic for Sample was 65. (df = 8, p =.88) and (df = 8, p =.57) for Sample. The chi-square probability values for Sample (p =.88) and Sample (p =.57) indicated adequate fit between the data and the model. Overall item and person fit: The residual mean value for items for Sample was. (SD.7), and for Sample was. (SD.7), indicating some misfit of items. The residual mean value for persons for Sample was.6 (SD.9) and for Sample was.9 (SD.), indicating no misfit of persons in either sample. Individual item and person fit: In both samples, Item 6 (Demonstrates clear and accurate written documentation) exhibited a positive item fit residual above +.5, suggesting Journal of Physiotherapy Vol. 57 Australian Physiotherapy Association

4 Research. Characteristics of participating students. Characteristic n = 56 Age (yr), mean (SD) () Age (yr), range 8 Gender, n female (%) (66). Characteristics of participating clinical educators. Characteristic n = 98 Age (yr), mean (SD) (8) Age (yr), range 6 Gender, n female (%) 5 (7) Experience as a clinical educator (yr), 6 (5) mean (SD) Experience as a clinical educator (yr), range Self-rated level of experience as a clinical educator, n (%) no experience () some experience 58 (9) average experience 88 () above average experience 86 (9) very experienced 5 (8) poor discrimination. None of the items exhibited a significant chi-square value (Table ). To investigate if the misfit of Item 6 was contributing to the overall item misfit to the model, Item 6 was removed from each sample and Rasch analysis repeated. The residual mean value for overall item fit changed from. (SD.7) to. (SD.5) in Sample and from. (SD.7) to. (SD.5) in Sample. The reduction in score variability indicated a small improvement in the overall fit of items to the model. Threshold order: There were no disordered thresholds for any of the items in either Sample or. The threshold map for Sample is illustrated in Figure. Targeting: The average person location in both samples was close to zero (.6) indicating that overall the item difficulty was well targeted to the students abilities. The person-item threshold graph (Figure ) presents the distribution of the students (top half of the graph) and item thresholds (bottom half of the graph) on a logit scale for Sample. This graph shows that a majority of item thresholds correspond to the main cluster of persons (students). Logits of increasing negative value indicate less difficult items and less able students. Logits of increasing positive value indicate more difficult items and more able students. There appears to be an even spread of item thresholds across the full range of student abilities, suggesting effective targeting of APP items. Similar results were seen for the first field test. At the far right end of the X-axis, there are a few person abilities that have no equivalent item threshold difficulties that could differentiate their performance. These represent high performing students. The number of students who are performing at a level too low to be captured by the scale is negligible. Hierarchy of item difficulty: The sequence or hierarchy of average difficulty of the items on the APP for both samples is presented in Table. In both samples, items. Sources and characteristics of the APP forms received. Characteristic n = 6 Responder burden duration to complete (min), mean (SD) 9 (9) duration to complete (min), range 8 Clinical area, (%) musculoskeletal neurological 5 cardiorespiratory paediatric 6 specialty a 5 unknown 8 Patient age group, (%) children ( to yr) adolescents ( to yr) adults ( to 65 yr) 5 older persons (> 65 yr) 6 unknown 5 Type of facility b, (%) public hospital 5 community based services 9 private hospital 7 non-government organisation 6 private practice unknown University program Monash University (Victoria, Australia) 8 Griffith University (Queensland, Australia) La Trobe University (Victoria, Australia) 9 James Cook University (Queensland, 7 Australia) Curtin University (Western Australia, 7 Australia) The University of Sydney (New South 6 Wales, Australia) Charles Sturt University (New South Wales, Australia) Otago University (New Zealand) Auckland University of Technology (New Zealand) unknown 7 a spinal injuries, burns, women s health, oncology, mental health, hand therapy, plastic surgery, b n = representing professional behaviour and communication were amongst the least difficult items whereas the most difficult items related to analysis and planning, progressing intervention, and applying evidence-based practice. Person separation index: The person separation index was.95 for Sample and.96 for Sample, indicating that the APP is able to discriminate at least four levels of performance. Differential item functioning: The presence of item bias was explored by analysis of differential item functioning with a Bonferroni-adjusted p value of.5. No significant Journal of Physiotherapy Vol. 57 Australian Physiotherapy Association

5 Journal of Physiotherapy Vol. 57 Australian Physiotherapy Association Individual item fit of APP items to the Rasch model: sample (n = 6) and sample (n = 8), with items ordered from least to most difficult. Sample (n = 6) APP item Location Standard error Fit residual DF Chisquare p Sample a (n = 8) APP item Location Standard error Fit residual DF Chi square a Item = understands client rights, = committed to learning, = ethical practice, = teamwork, 5 = communication skills, 6 = documentation, 7 = interview skill, 8 = measures outcomes, 9 = assessment skills, = interprets assessment, = prioritises problems, = sets goals, = intervention choice, = intervention delivery, 5 = effective educator, 6 = monitors intervention effects, 7 = progresses intervention, 8 = discharge planning, 9 = applies evidence-based practice, = assesses risk. DF = degrees of freedom, a Chi-square degrees of freedom was for all items p Dalton et al: Assessment of professional competence of students

6 Research understands client rights committed to learning ethical practice teamwork communication skills documentation interview skill measures outcomes assessment skills Items interprets assessment prioritises problems sets goals intervention choice intervention delivery effective educator monitors intervention effects progresses intervention discharge planning applies evidence-based practice assesses risk Person location (logits). Threshold map in Sample (n = 6). person item 5 7.7% 6.% Frequency 5.6%.% 5.5% Location (logits).% Frequency % 8.% 6.7%. Person-item threshold distribution for Sample (n = 6). Grouping set to interval length of. making 7 groups. Journal of Physiotherapy Vol. 57 Australian Physiotherapy Association

7 Dalton et al: Assessment of professional competence of students 9 8 y = 5.8x +.5 R = Raw APP score Location. Plot of person logit location and raw APP score (Sample n = 6). differential item functioning was demonstrated in either of the two samples for the following variables: the student s age, gender, or amount of prior clinical experience, the educator s age, gender, or experience as an educator, or the type of facility, university, or clinical area. This indicates the APP item ratings were not systematically affected by any of these nine variables. Local independence and dimensionality: Local independence is the assumption that responses to items are independent. Some local dependence was evident, with four items showing positive residual correlations greater than. in both samples. The items showing positive residual correlations were Item (Demonstrates an understanding of patient rights and consent), Item (Demonstrates a commitment to learning), Item (Demonstrates ethical, legal and culturally sensitive practice), and Item 5 (Verbal communication). A unidimensional set of items measures a single underlying construct. APP dimensionality was tested by an independent t-test procedure of person ability locations derived from two subsets of items one loading positively and the other negatively >. on the first residual factor of the principal components analysis in RUM (Tennant and Pallant 6). The proportion of persons with significantly different person estimates based on the two item subsets was 7.% and 6.9% for the two samples. The confidence intervals for a binomial test of proportions both included 5%, providing evidence of the unidimensionality of the scale. Figure shows the relationship between raw ordinal APP scores and person logit location for Sample. Sample exhibited the same relationship. Discussion This second and final field trial of the -item APP confirmed that it is a unidimensional instrument with a response scale that is used as anticipated and that is able to discriminate at least four distinct levels of student performance. The sequence or hierarchy of average difficulty of the competencies on the APP provides an indication of which clinical competencies may be easier to acquire, such as communication and professional behaviours, and those that are more difficult and therefore may be expected to take longer to master. The hierarchies of both samples in the current study revealed that items related to analysis and planning (critical thinking), goal setting, and selection and progression of interventions were the most difficult items for students to perform. Rheault and Coulson (99) demonstrated a similar ranking of a 6-item physiotherapy practice assessment instrument. From easiest to most difficult the items were: exhibits professionalism, exhibits communication skills, performs effective treatment skills, performs safe treatment skills, can problem solve, and works from an adequate knowledge base. While the data collected in the field test demonstrated overall fit to the Rasch model for both participant samples, Item 6 (Written communication) showed misfit to the Rasch model. Pallant and Tennant (7) state that one of the most common sources of item misfit is respondents (educators) inconsistent use of the scoring options resulting in disordered thresholds. However, investigation of threshold ordering of the polytomous items on the APP showed there were no disordered thresholds in either sample. Despite a small improvement in overall item fit to the model when Item 6 was deleted, removal of this item is not justified given that written communication is part of the current Australian Standards for Physiotherapy (Australian Physiotherapy Council 6) and represents an essential aspect of professional competence. Exploration of this issue with clinical educators suggests that there is a lack of consensus with respect to the timing of recording patient- Journal of Physiotherapy Vol. 57 Australian Physiotherapy Association 5

8 Research therapist interactions during or after the encounter, and that agencies did not clearly communicate their expectations to students early in the placement. Further research on this item and how it is being interpreted and scored by educators is warranted. In the final field test no significant differential item functioning was demonstrated for the variables student age and experience, clinical educator age, gender, and experience as an educator, university, or field of practice. This indicates that APP item ratings were not systematically affected by any of these variables and supports nationwide use of this instrument across all clinical areas, facilities and universities. One of the primary advantages of Rasch analysis is that raw ordinal scores may be converted to interval level Rasch scores. Given the almost perfect linear relationship between Rasch logit scores and raw scores shown in Figure, the complexity associated with converting the raw score to a Rasch score does not appear warranted. The APP was developed collaboratively, tested within the constraints of a dynamic and unpredictable clinical environment, and has been taken up almost universally as the assessment instrument in entry-level physiotherapy programs in Australia and New Zealand. The advantages of a single, national instrument are the reduction of assessment burden on clinical educators dealing with students from multiple university programs, and the standardardisation of student assessment for entry-level practice ensuring that students are assessed against the same performance indicators, on the same rating scale, against explicit standards for entry-level practice. The evidence of construct validity provided by Rasch analysis supports the interpretation that a student s score on the APP is an indication of their underlying level of professional competence as demonstrated during workplace-based placements. The reliability of judgements made with the APP will be published separately. eaddenda: Figure and Appendix available at jop. physiotherapy.asn.au Ethics: Approval for the study was provided by the Human Ethics Committees of the nine participating universities. All participants gave written informed consent before data collection began. Support: Funding from the Australian Learning and Teaching Council (ALTC) enabled employment of a research assistant and travel to conduct focus groups and training workshops. Acknowledgements: Thanks go to the clinical educators and students who participated, to the University Clinical Education MAnagers of Australia and New Zealand, and to the Council of Physiotherapy Deans, Australia and New Zealand, who championed the development of a national assessment instrument. Correspondence: Dr Megan Dalton, Department of Physiotherapy, School of Primary Health Care, Monash University, Australia. megan.dalton@monash.edu References Andrich D, Lyne A, Sheridan B, Luo G () RUMM. RUMM Laboratories Perth. Australian Physiotherapy Council (6) physiocouncil.com.au/australian_standards_for_ physiotherapy/. [Accessed 6/8/]. Bond TG, Fox MT (7) Applying the Rasch Model. Fundamental measurement in the human sciences (nd edn). Mahwah, NJ: Erlbaum. Cliff N, Keats JA () Ordinal measurement in the behavioral sciences. Mahwah, NJ: Erlbaum. Coghlan D, Brannick T () Doing action research in your own organisation. Thousand Oaks, CA: Sage. Dalton M, Keating J, Davidson M (9) Development of the Assessment of Physiotherapy Practice (APP): A standardised and valid approach to assessment of clinical competence in physiotherapy. [Australian Learning and Teaching Council (ALTC) Final report pp 6 8]. Brisbane: Griffith University. Available online at: Lai J-S, Teresi J, Gershon R (5) Procedures for the analysis of differential item functioning (DIF) for small sample sizes. Evaluation and the Health Professions 8: 8 9. Pallant JF, Tennant A (7) An introduction to the Rasch measurement model: an example using the Hospital Anxiety and Depression Scale (HADS). British Journal of Clinical Psychology 6: 8. Prescott-Clements L, van der Vleuten CP, Schuwirth LW, Hurst Y, Rennie JS (8) Evidence for validity within workplace assessment: the Longitudinal Evaluation of Performance (LEP). Medical Education : Rasch G (96) Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press. Rethans JJ, Norcini JJ, Baron-Maldonado M, Blackmore D, Jolly BC, LaDuca T et al () The relationship between competence and performance: implications for assessing practice performance. Medical Education 6: Rheault W, Coulson E (99) Use of the Rasch model in the development of a clinical competence scale. Journal of Physical Therapy Education 5:. Southgate L, Hays RB, Norcini J, Mulholland H, Ayers B, Woolliscroft J et al () Setting performance standards for medical practice: a theoretical framework. Medical Education 5: 7 8. Streiner DL, Norman GR () Health Measurement Scales. A practical guide to their development and use (rd edn). New York: Oxford University Press. Tennant A, Conaghan PG (7) The Rasch measurement model in rheumatology: what is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis and Rheumatism 57: Tennant A, Pallant JF (6) Unidimensionality matters! (a tale of two Smiths). Rasch Measurement Transactions : 8 5. van der Vleuten C () Validity of final examinations in undergraduate medical training. British Medical Journal : 7 9. Wilson M (5) Constructing measures: an item response modeling approach. Mahwah, NJ: Erlbaum. Website Dalton MB () Development of the Assessment of Physiotherapy Practice - A standardised and validated approach to assessment of professional competence in physiotherapy. Doctor of Philosophy Thesis, Monash University, Melbourne. URL: hdl/959./79 6 Journal of Physiotherapy Vol. 57 Australian Physiotherapy Association