Therapy outcome measures for allied health practitioners in Australia: the AusTOMs

International Journal for Quality in Health Care 2004; Volume 16, Number 4: pp. 285 291 10.1093/intqhc/mzh059 Therapy outcome measures for allied health practitioners in Australia: the AusTOMs ALISON PERRY 1, MEG MORRIS 2, CAROLYN UNSWORTH 3, STEPHEN DUCKETT 4, JEMMA SKEAT 1, KAREN DODD 2, NICHOLAS TAYLOR 2 AND KAREN REILLY 3 1 School of Human Communication Sciences, 2 School of Physiotherapy, 3 School of Occupational Therapy and 4 School of Public Health, Faculty of Health Sciences, La Trobe University, Melbourne, Victoria, Australia Abstract Objective. The aim of this study was to develop a valid and reliable measure of therapy outcome for three allied health professions in Australia: speech pathology, occupational therapy, and physiotherapy. The Australian Therapy Outcome Measures (AusTOMs) enable measurement of the differences in client profiles and patterns of services provision across health care settings. In this paper we describe phase 1 of the study: the development and preliminary validation of the AusTOMs. Method. The UK TOMs, developed by Enderby, were scrutinized by the research team. A pilot core scale was developed, based on the structure of the TOM. Focus groups of expert clinicians for each profession, across the state of Victoria in Australia, analysed and refined the scales further. A mail-out survey was then sent to therapists across Australia to assess both face and content validity of the AusTOMs. Main results. A new tool, the AusTOM, was developed and tailored to the needs of each profession, with input from specialist clinicians and allied health researchers. The face and content validity of the new scales were assessed, and good consensus was obtained for the wording and content validity of the scales. The discriminative validity, concurrent validity, and reliability of the tool are now being evaluated. Conclusion. We have produced an outcome measure in the Australian context for speech pathology, physiotherapy, and occupational therapy. There are six speech pathology scales, nine physiotherapy scales, and 11 occupational therapy scales in the AusTOMs. A clinician chooses the relevant scale(s) for the client (based on the goals of therapy) and makes a rating across all domains for each scale. Further papers will report on the reliability, validity, and clinical usefulness of the AusTOMs. Keywords: AusTOMs, measurement, occupational therapy, physiotherapy, speech pathology; therapy outcome In order to adopt an evidence-based approach to clinical practice, therapists need to routinely evaluate patient outcomes for a range of health care domains. As well as measuring impairments, recently there has been a shift towards quantifying the effects of therapy on activity limitations and restrictions to participation in societal roles. The well-being of patients and caregivers is another health domain that is now being monitored as a key indicator of therapy outcome. One such tool that is used to measure outcomes across these domains is the Therapy Outcome Measure (TOM), developed by Enderby [1] in the UK. Using the TOM [2,3] involves ordinal ratings to assess a client across four domains, based on the World Health Organization s (WHO) International Classification of Impairment, Disability, and Handicap (ICIDH) [4]. One domain describes impairment of body structure/body function (dysfunction), one describes disability (limitations in the performance of functional activities), and the remaining two address psychosocial status: handicap (social disadvantage to the individual) and well-being/distress (emotional level). These domains capture the targets (goals) of therapy [1]. Using TOM further demands consideration of the person s disorder and its effects on function, as well as social and psychological factors such as societal participation and wellbeing. Several reliability trials involving speech and language therapists, occupational therapists, and physiotherapists have been conducted in the UK [2,3,5], which have demonstrated that the TOM has robust measurement properties, with good face and content validity reported [2]. The aim of this project was to devise and validate a therapy outcome tool for use by Australian allied health practitioners, based on the TOM approach. The relevance of the TOM to the professional culture and terminology used within Australia were first assessed; then substantial modifications were made to contextualize the final tool to Australian clinical Address reprint requests to Alison Perry, School of Human Communication Sciences, Faculty of Health Sciences, La Trobe University, Melbourne, Victoria, Australia. E-mail: a.perry@latrobe.edu.au International Journal for Quality in Health Care vol. 16 no. 4 International Society for Quality in Health Care and Oxford University Press 2004; all rights reserved 285

A. Perry et al. practice. It is our view that an outcome measure that is specific to Australian health care will enhance efforts at quality assurance by enabling clinicians to quantify changes in patient functioning, using a tool that provides a common language across professionals. Materials and methods Task 1: development of one Australian core scale The research team s first task was to investigate whether the descriptors used in the TOM were appropriate for use in Australia and, where necessary, adapt definitions in accordance with local requirements. Since the original development of the TOM, the WHO has revised the ICIDH classification (which underpins TOM) and a new classification has been published: the International Classification of Functioning, Disability, and Health (ICF) [6]. The ICF differs in several ways from ICIDH, including with respect to the terminology and focus. The ICF has a focus on the descriptors of health, rather than the consequences of disease. A more positive focus in terminology is noticeable in this new classification. After discussion, it was agreed that the descriptors used for the domains to be assessed in the Australian context needed to reflect these changes. The research team therefore first developed one Australian core scale from which to expand more specific scales, relevant to each of the three professions. This mirrored the process used by Enderby and colleagues when developing the TOM [2]. An overview of the study process is shown in Figure 1. Task 2: operational definitions for AusTOMs scales Focus groups of expert Australian clinicians were asked to examine the TOM scales relevant to their profession. Although they all favoured using outcome measurement, criticism from those therapists in Australia who had used the TOM included dissatisfaction with the headings and with many of the written descriptors used in the UK scales. Using the core scale developed in task 1 as a starting point, focus groups from each profession set out to develop further scales that would not only be profession-specific, but that would be able to measure outcomes of therapy for patients treated in Australia. The focus groups were made up of invited expert clinicians (i.e. those with 2 years experience in the relevant clinical area). This was a convenience sample, and included staff from centres where data would be collected later in the project. The first step for each focus group was to look at the proposed scale headings for each profession, to determine if these were an accurate reflection of therapy areas covered by that profession. The groups then drafted text, defining each of the 0 5 points across the four domains on the scale. Clinicians provided terminology pertinent to these scales; for example, how they would describe a severe speech impairment. This was done by debating and reaching consensus about, firstly, the extreme ends of each scale s domain (e.g. defining points 0 and 5 under the impairment domain), and then refining the mid-points. Each focus group lasted 2 3 hours. Key discussion at each focus group was recorded onto audiotape and by writing consensus statements onto large sheets of paper. These were later TASK 1: Development of one core, generic AusTOMs scale for SP, OT and PT: 4 domains: Impairment, Activity Limitation Participation Restriction, Wellbeing/Distress Who? Research Team TASK 2: Who? Operational definitions of AusTOMs core SP, OT, & PT and SP, OT and PT specific scales Focus groups clinicians in Victoria SP OT PT TASK 3: Assure face and content validity of AusTOMs Modified Delphi technique Who? SP, OT, & PT clinicians across Australia TASK 4: Who? Consumer views of the tool Consumers and advocates Figure 1 Process of AusTOMs development. 286

AusTOMs therapy outcome measure Table 1 Focus group format across the professions Profession Focus groups Aim... Speech pathology One focus group with five specialist clinicians and five academics Five focus groups of clinicians (total = 32 clinicians) To examine the TOM scale headings for speech pathology in the Australian context, and to propose new scale headings wherenecessary To examine the proposed scale headings and develop specific scale descriptors Occupational therapy Two focus groups of 12 academics To examine the TOM scale headings for occupational therapy in the Australian context, and to propose new scale headings where necessary Seven focus groups of clinicians (total = 45 clinicians) To examine the proposed scale headings and develop specific scale descriptors Physiotherapy One focus group of six academics To examine the TOM scale headings for physiotherapy in the Australian context, and to propose new scale headings where necessary Five focus groups of clinicians (total = 67 clinicians) To examine the proposed scale headings and develop specific scale descriptors typed as summaries by a member of the project team. Table 1 shows the focus group format for each of the three professions. Task 3: face and content validity of the AusTOMs (modified Delphi Survey) The profession-specific scales that had been assessed and adapted by the focus groups were then sent out as part of an Australia-wide survey of clinicians, using a modified Delphi technique [7 ] (i.e. two progressive mail-outs to gain consensus) to assure face and content validity of the scales. A list of 1050 allied health managers from Australia s National Allied Health Casemix Committee (NAHCC) was obtained. A mail-out to a convenience sample of 600 managers of allied health services (200 speech pathologists, 200 physiotherapists, and 200 occupational therapists) across Australia took place. The list of managers was stratified, first by state and then to achieve representation of both rural and metropolitan areas across as wide a representation of each profession as possible. This sample was used to assure the face validity of AusTOMs, via national consensus on the scale headings, and then agreement about the 0 5 point level domain descriptors, which were developed by the focus groups. Each manager was sent a copy of all their own professionspecific scales, and asked to distribute the scales to expert clinicians in each of the specialist clinical areas indicated by the particular scale heading. This was the only efficient way to disseminate the scales across Australia. Speech pathology and occupational therapy. Clinicians who were given the survey were asked: (i) to scrutinize the scale descriptors line-by-line; (ii) to rate the overall accuracy of each line on a four-point scale (where 1 = an extremely inaccurate description and 4 = an extremely accurate description); and (iii) to add or remove words or phrases from the scale as necessary to achieve an accurate portrayal of the relevant clinical area and scalar points. Clinicians were also asked to comment on the face validity of the scale, including whether they felt it would be useful, whether the scale provided logical descriptors across domains, and whether they felt comfortable with the concepts described. Physiotherapy. At this stage, the physiotherapy scales did not have detailed descriptors for each of the 0 5 points in each domain. Hence, the Delphi survey used for physiotherapy differed considerably from the other professions. For the first mail-out, physiotherapy clinicians were asked to do two things; firstly, 5-point scales were used to rate their strength of agreement (1 = strongly agree, 2 = agree, 3 = neutral, 4 = disagree, and 5 = strongly disagree) with the following statements for face validity: This scale would be useful to measure outcomes for clients with this disorder ; It would be easy to decide where a client would fit on this scale ; I feel comfortable with the descriptions given on this scale ; The concepts of impairment, activity limitation, participation restriction, and well-being are relevant to the therapy I do and the outcomes I wish to measure ; and I feel comfortable about using this scale with a range of clients with this disorder. Furthermore, physiotherapists were asked to comment (open-ended) on the suitability of the scale headings and the overall domains. Task 4: consumer focus group While the AusTOMs aims to provide a measure of outcomes to be used by the therapist, it was felt to be important to include consumer (client) views of health in the development of the tool. To this end, a focus group of consumers was convened, which six clients and consumer advocates attended. 287

A. Perry et al. An additional seven clients/advocates were invited, but were unable to attend. The group examined all of the AusTOMs, as developed in phases 2 and 3 of the study. Results Task 1: development of an Australian core scale The initial milestone in this project was a revision, on both a linguistic and cultural level, of the core outcome measurement scale. A core scale was derived for all professions to use with their focus groups. This scale in its final form is shown as Table 2. The scale represented a departure from the TOM in the terminology used for each domain, although the core structure of the TOM was retained. Task 2: operational definitions for AusTOMs Speech pathology. The participants in the focus groups in speech pathology expressed the view that the speech pathol- Table 2 AusTOMs core scale Impairment of either structure or function (as appropriate to age): impairments are problems in body structure (anatomical) or function (physiological) as a significant deviation or loss. 0 = The most severe presentation of impairment (either structure or function) 1 = Severe presentation of this impairment 2 = Moderate/severe presentation 3 = Moderate presentation 4 = Mild presentation 5 = No impairment of structure or function Activity limitations (as appropriate to age): activity limitation results from difficulty in the performance of an activity. Activity is the execution of a task by the individual (in a standard milieu without assistance). 0 = Complete difficulty 1 = Severe difficulty 2 = Moderate/severe difficulty 3 = Moderate difficulty 4 = Mild difficulty 5 = No difficulty Participation restriction (as appropriate to age): participation restrictions are difficulties the individual may have in the manner or extent of involvement in their life situation. Clinicians should ask themselves, given their problem, is this individual experiencing disadvantage?. 0 = Unable to fulfill social, work, educational, or family roles. No social integration. No involvement in decision-making. No control over environment. Unable to reach potential in any situation. 1 = Severe difficulties in fulfilling social, work, educational, or family roles. Very limited social integration. Very limited involvement in decision-making. Very little control over environment. Can only rarely reach potential with maximum assistance. 2 = Moderately severe difficulties in fulfilling social, work, educational, or family roles. Limited social integration. Limited involvement in decision-making. Control over environment in one setting only. Usually reaches potential with maximum assistance. 3 = Moderate difficulties in fulfilling social, work, educational, or family roles. Relies on moderate assistance for social integration. Limited involvement in decision-making. Control over environment in more than one setting. Always reaches potential with maximum assistance and sometimes reaches potential without assistance. 4 = Mild difficulties in fulfilling social, work, educational, or family roles. Needs little assistance for social integration and decision-making. Control over environment in more than one setting. Reaches potential with little assistance. 5 = No difficulties in fulfilling social, work, educational, or family roles. No assistance required for social integration or decision-making. Control over environment in all settings. Reaches potential with no assistance. Well-being (as appropriate to age): the level of concern experienced by the individual. Concern may be evidenced by anger, frustration, apathy, depression, etc. 0 = High and consistent levels of distress or concern. 1 = Severe concern, becomes distressed or concerned easily. Requires constant reassurance. Loses emotional control easily. 2 = Moderately severe concern. Frequent emotional encouragement and reassurance required. 3 = Moderate concern. May be able to manage emotions at times, although may require some encouragement. 4 = Mild concern. Able to manage emotions in most situations. Occasional emotional support or encouragement needed. 5 = Able to cope with most situations. Accepts and understands own limitations. 288

AusTOMs therapy outcome measure Table 3 Final AusTOMs speech pathology headings 1. Voice 2. Speech 3. Language 4. Fluency 5. Swallowing 6. Cognitive communication ogy TOM presented a mixture of terminology, and a suggestion was made to use the terminology used by Speech Pathology Australia in their Competency Based Occupational Standards (CBOS) [8]. These are terms with which all Australian speech pathologists are familiar. After further discussion, it was felt that there may be a need for one additional scale to recognise cognitive communication as an area of speech therapy practice that would not fit into any of the CBOS areas. Table 3 shows the final headings for the speech pathology scales. These terms have two advantages: firstly, they are all descriptions of functional ability in areas where speech pathologists work with clients; and secondly, they all are descriptions in a positive direction, rather than of disorder. The latter is conceptually more in keeping with the new ICF philosophy (see above), describing what clients can do, rather than what they cannot [6]. Speech pathology focus groups then developed terminology for each of these six scales. Occupational therapy. Using the ICF [6] framework, occupational therapists work mostly at the levels of activity and participation. The initial two focus groups of academic occupational therapists suggested a structure for AusTOMs according to the ICF [6] classifications of activity/activity limitation and participation/participation restriction. Scale headings were developed to reflect client activities, such as self-care, functional walking, domestic life, and employment. A chosen scale would then align with the dominant goals of therapy interventions. Clinicians participating in the seven focus groups then confirmed these headings and developed the descriptors for each of the four AusTOMs domains under each scale heading. The set of occupational therapy scales developed is shown in Table 4. Physiotherapy. Based on the ICF [6], the participants of the physiotherapy focus groups felt that basing scale headings on impairments of body structure and function would provide a strong construct for the physiotherapy scales for AusTOMs. Using the body function headings from the ICF, 11 scales were originally proposed for further debate and refining. Later modifications (see below) resulted in nine scales, as shown in Table 5. Overall descriptors for each domain were developed. Task 3: face and content validity of the AusTOMs (Delphi Survey) The modified Delphi [7] survey across Australia resulted in 240 scales returned by speech pathologists, 228 by physiotherapists, Table 4 Final AusTOMs occupational therapy headings 1. Learning and applying knowledge 2. Self-care 3. Functional walking and mobility 4. Domestic life: inside house 5. Upper limb use 6. Domestic life: outside house 7. Carrying out daily life tasks and routines 8. Interpersonal interactions and relationships 9. Transfers 10. Work, employment, and education 11. Using transport 12. Community life, recreation, leisure, and play Table 5 Final AusTOMs physiotherapy headings 1. Balance and postural control 2. Cardiovascular system functions 3. Musculoskeletal movement-related functions 4. Neurological movement-related functions 5. Pain 6. Respiratory system functions 7. Sensory function 8. Skin functions 9. Urinary and bowel continence and 228 by occupational therapists in round 1. Returns represented all Australian states and territories. These same respondents were sent a second mailing with the modified scales. There was a return rate ranging from moderately good for speech pathology (42%) and occupational therapy (52%), to poor for physiotherapy (21%), for the second round of the Delphi. Speech pathology. The results from the first mail-out showed that speech pathologists mostly agreed that the scales had good face validity and were comfortable with the concepts on which the scales were developed. For the impairment domain of the voice, language, speech, cognition, and swallowing scales, the percentage of descriptors rated by speech pathologists as extremely accurate (i.e. with a rating of 4, see Methods) ranged from 55% to 93%, with the exception of the fluency scale, where descriptors were rated as extremely accurate only 8% of the time. The results for fluency impairment were poor as clinicians wished to have a descriptor added: percentage of syllables stuttered across each level of the impairment domain. This was changed before sending out the second round for Delphi consensus. With this addition, the second round consensus resulted in 92% of impairment descriptors of the fluency scale being rated by participants as extremely accurate. For the other scales, clinicians agreement with the descriptors was consistently >75% for the second mail-out, after minor changes in wording were made to the scales, as recommended by clinicians. At the first mail-out, participating speech pathologists rated between 73% and 85% of the descriptors in the activity 289

A. Perry et al. limitation domain as extremely accurate, and 73 89% of the participation restriction domain descriptions. The results for well-being were similar, with participating speech pathologists rating 77 92% of descriptors as extremely accurate. After minor modifications were made to the second mail-out, final results indicated that clinicians believed that well over 75% of the domains in every scale had accurate descriptors. Occupational therapy. From the results for the first mail-out, participating occupational therapists rated the descriptors of the impairment domain across all 12 scales as extremely accurate >80% of the time (range 83 100%). The results were even better for the activity limitation and participation restriction domains, with therapists indicating that 92 100% of these descriptors were extremely accurate. For well-being, excellent results were obtained, with 94 100% of descriptors being judged as extremely accurate. The second round of the process resulted in >90% concordance regarding the scales and scale headings to be used, and for the text descriptors across the 0 5 points of rating. Physiotherapy. Across the 11 initial physiotherapy scales, clinicians were either in moderate agreement with statements, or they were neutral/undecided. No scale showed either full agreement or disagreement/strong disagreement with any of the statements (see Methods). There was a moderate degree of concordance with the scale headings and the overall descriptors on this initial version of the physiotherapy scales. As a number of clinicians disagreed with the wording of some of the descriptors, further revision was necessary. Thus, before the second mail-out, the physiotherapy AusTOMs team ran further in-depth interviews with a small panel of physiotherapists to guide refinement and revision of the scales in areas where disagreement was greatest. More detailed descriptors were developed and revised scales were re-sent to the clinicians who had participated in the first modified Delphi. In the second mail-out, physiotherapy clinicians were given the following instructions: (i) cross out any parts of the scale or the Instructions for use document that you think are unsuitable or need to be re-worded; and (ii) change the words and add any comments or extra information that you feel are necessary to the scale(s) or the instructions document, on the attached papers. The result of this second round of returns was overwhelmingly positive, with very few changes to wording suggested for headings and scalar descriptors. Many physiotherapists expressed satisfaction with the scales and noted that their earlier concerns about the scale descriptors had been addressed. All the suggested wording changes were therefore incorporated into the final versions. There was consensus from the mail-out responses and from the expert clinicians that mental functions and reproductive system functions were not scales that were appropriate or useful to Australian physiotherapy practice. These were therefore omitted from the second phase of development, resulting in nine final scales. Task 4: consumer focus group The consumer focus group made no changes to the wording of the impairment or activity limitation domains for any of the scales. It was generally felt that these domains were carefully thought through, with close attention to detail. The group was positive about these domains, with participants advising that these sections of the tool appeared to be suitable for enabling allied health practitioners to measure outcomes of therapy. Participants pointed out that some of the wording of the participation restriction domain was ambiguous, and would require careful explanation in training and/or guidelines for use of the tool. Minor wording changes were suggested, such as replacing the phrase reassurance and support needed with emotional support or encouragement needed, as the word reassurance was not felt to reflect a person s level of well-being. As a result of this focus group, several minor changes in wording were made to the participation restriction and well-being domains of the scales. Discussion This paper describes the process by which clinical consensus was used to develop new therapy outcome measures, the AusTOMs, for use across speech pathology, occupational therapy, and physiotherapy practices in Australia, thereby providing evidence for the face and content validity of the measure. During the course of developing the tool, clinicians reiterated the importance of using consistent constructs across scales used to measure therapy outcomes. They were also more comfortable in using terminology that reflected practices in their own country (in this case Australia), rather than directly adopting terms that may be used overseas. The three professions that participated in this study have different professional mores and philosophies. Occupational therapy has grown out of the need to understand the relationship between what a person does their occupation and their health and well-being. The core business of occupational therapists is to habilitate and rehabilitate people who have difficulties managing their daily activities and participating in society following physical, emotional, psychological, and/or social disorders. Physiotherapy aims to enable people to move more easily so that they can perform functional activities at their optimal level. Although traditionally physiotherapy practice focused on the treatment of impairment, more recently there has been a paradigm shift and practice is beginning to take into account disability and societal participation. Speech pathology has a mixed emphasis. Addressing communication and/or swallowing disabilities involves tackling the changes at the impairment level (e.g. structural changes that occur after surgery or from neurological deficit), and then the psycho-social, activity, and well-being domains are addressed, as communication impacts upon all three. The AusTOMs provide a common vocabulary for assessing outcomes, thereby crossing philosophical boundaries of the three disciplines. One of the strengths of this study lies in the fact that both therapist and client viewpoints were sought in its development. Clinician involvement was essential, as clinicians will ultimately be those who need to accept and use this tool in clinical practice. The need for a common, patient-understood 290

AusTOMs therapy outcome measure language of health outcomes was emphasized by Paul Ellwood ([9], p. 1551), one of the key contributors to outcome measurement in the United States. The AusTOMs were developed primarily from the clinicians perspective, but have also been scrutinized and modified by clients, resulting in a tool that may be comprehensible and useful to both groups. A limitation of the AusTOMs is that they have been deliberately contextualized to Australian allied health practice. The tool reflects the ways in which speech pathology, occupational therapy, and physiotherapy are provided in Australia, which may vary in other parts of the world. Further testing of the validity, reliability, sensitivity to change, and clinical usefulness have been undertaken, and will be reported in a companion paper. The measurement of therapy outcomes is an essential, yet sometimes under-emphasized, component of clinical practice. In allied health professions such as physiotherapy, occupational therapy, and speech pathology, there are few valid therapy outcome tools that are well matched to contemporary practice, as well as being quick and easy to administer. The AusTOMs aim to address these needs and provide clinicians with scales that measure impairment, activity limitation, participation restriction, and well-being, using terminology that reflect the different cultures of the allied health disciplines. The next paper will report the reliability, validity, and sensitivity of this new therapy outcome measure, as well as its usefulness across clinical settings. Acknowledgements La Trobe University acknowledges that this publication draws heavily upon the AusTOMs; Australian Therapy Outcome Measures project, commissioned by the Commonwealth Department of Health and Ageing, Canberra, Australia. We thank Ms Sue Cotton, Biostatistician from the University of Melbourne, who assisted us with advice on data management and data analyses for this project. Ms D. Benetti provided administrative assistance throughout. Professor Pam Enderby assisted the research team at La Trobe University in the application to the Commonwealth to support the project, and both professor Enderby and Dr Alexandra John from Sheffield University, UK, are associate researchers to this project, providing the research team with advice, discussion, and support in this development of the AusTOMs. We are grateful for their guidance and encouragement. References 1. Enderby P. Outcome measures in speech therapy: impairment, disability, handicap, and distress. Health Trends 1992; 20: 61 64. 2. Enderby P, John A. Therapy Outcome Measures: Speech-Language Pathology Technical Manual. London: Singular, 1997. 3. Enderby P, John A, Petherham B. Therapy Outcome Measures manual: physiotherapy, occupational therapy, rehabilitation nursing. San Diego, CA: Singular, 1998. 4. WHO. International Classification of Impairments, Disabilities and Handicaps (ICID-H). Geneva: WHO, 1980. 5. John A, Enderby P. Reliability of speech and language therapists using Therapy Outcome Measures. Int J Lang Commun Disord 2000; 9: 287 302. 6. WHO. International Classification of Functioning, Disability and Health (ICF). Geneva: WHO, 2001. 7. Sim J, Wright C. Research in Health Care: Concepts, Designs and Methods. Cheltenham: Stanley Thornes, 2000. 8. The Speech Pathology Association of Australia (SPAA). Competency Based Occupational Standards (CBOS) for Speech Pathologists. Melbourne: SPAA Limited, 1994. 9. Ellwood PM. The Shattuck Lecture outcomes management: a technology of patient experience. N Engl J Med 1988; 318: 1549 1556. Accepted for publication 5 April 2004 291