Champlain Assessment/Outcome Measures Forum February 22, 2010
Welcome Table Configurations Each table has a card with the name of the discipline(s) and the mix of professions Select the discipline of interest to you and join that group using the guidelines for the numbers of each profession, per table. Once a table is filled (has it s quota ) please choose another table. You will have an opportunity to work with the tools associated with three (3) disciplines today. Please try to mix up the experience around the table (e.g., community-based, rehabilitation, etc.)
Keynote Speaker Marc Paquette Stroke Rehabilitation Coordinator Regional Stroke Program The Ottawa Hospital
Canadian Stroke Strategy 2006 Consensus Panel convened Goal: core set of rehab outcome measures across continuum Selection Criteria: published use in stroke trials, strong psychometric properties, ease of admin., inter-professional where possible Domains selected based on ICF (body structure, function, activity & participation)
Benefits of using a Core Set of Tools Improve data collection re: stroke rehab outcomes across regions Ensure clinicians know which tools are valid, reliable & support best practice Promote a common language within teams & across transitions Select screening tools which identify when more assessment/tx is needed Allow consistency for pts to track progress
Disadvantages of doing nothing Using tools with poor reliability/validity Fragmented communication of measures Duplication of assessment Increased system costs Meaningless data
Best Practice for Stroke Care Canadian BP Recommendations 2008: 4iv & 5viii. Clinicians should use standardized, valid assessment tools to evaluate the patient s stroke related impairments & functional status Ontario Consensus Panel on the Stroke Rehabilitation system report, 2007: Standard 5, Stroke related impairments and functional status will be evaluated by rehab professional trained in stroke rehab using standardized, valid assessments
Provincial & Regional Supports Ontario Stroke System Provincial regions undergoing process Rehab Network of Champlain (RNOC) Stroke Rehab Sub-Committee (RNOC) Champlain Regional Stroke Steering Committee Management of your organizations
Objectives of the day Gain knowledge of CSN domains & tools Knowledge of Champlain tools in use Identify enablers & barriers of tools & implementation Achieve consensus on some core tools for adoption Report back to stakeholders, RNOC & provincial & national groups
Objectives continued Patient centered Focus on regional standardization Think system wide Consider all parts of the rehab continuum Remember change process takes time Network with your regional colleagues Exchange e-mail addresses Have fun!
Domains for today Activities of Daily Living Upper Extremity Structure & Function Walking/Lower Extremity Upper & Lower Extremity Impairment Functional Mobility/Balance Spasticity Language Cognition Visual Perception
Tools for consideration FIM & Alpha-FIM Chedoke-McMaster Assessment Scale Chedoke Lower Extremity Disability Inventory Timed Up & Go Test Mini Mental Examination Montreal Cognitive Assessment Line Bisection Test Visual Perception Test Motor-Free Rivermead Perceptual Assessment Battery
Tools for consideration Ontario Society of Occupational Therapists Perceptual Evaluation Modified Ashworth Scale 6-Minute Walk Test Berg Balance Scale Boston Diagnostic Aphasia Exam Frenchay Aphasia Screening Test
Domains for future consideration Measure of Stroke Severity Medical Comorbidities Speech Intelligibility Depression Functional Communication Instrumental Activities of Daily Living Participation
Provincial Outcome Measures Working Group - Recommendations Balance: Berg Balance Scale Self-Care Activities of Daily Living: AlphaFIM Instrument /FIM Instrument Cognition: Montreal Cognitive Assessment (MoCA) Upper Extremity/Lower Extremity Structure & Function: Chedoke-McMaster Stroke Assessment (CMSA) Impairment Inventory Language: Boston Diagnostic Aphasia Exam
Results of a Canadian Stroke Strategy/Heart and Stroke Foundation National Consensus Panel
CSS Outcome Measures Methodology Dr. Debbie Timpson, Physiatrist Chief of Rehabilitation Pembroke Regional Hospital
Acknowledgements Canadian Stroke Network Canadian Stroke Strategy Heart and Stroke Foundation of Ontario Dr. Mark Bayley and Consensus Planning Group
Considerations for Selecting Responsive and Interpretable Outcome Measures Your stroke team indicates they want to measure the outcomes of their patients. What are the considerations in selecting measures?
What measures are Canadian Clinicians using? Nicol Korner Bitensky et al surveyed about 1800 rehabilitation clinicians in Canada by telephone Asked to answer scenarios concerning typical stroke patients Asked about what measures they used
Summary of Survey findings Inconsistent use of measures Frequently used only at admission and not at discharge Not necessarily using measures tested for responsiveness
Stroke Outcome measures in SREBR 1968-2004 Total number of outcome citations = 1105 Large number of authors created own study specific outcomes = 175 Physical assessments not using a single standardized scale = 178 citations Citations of previously published scales= 752 Only 35 previously published assessment scales were cited 5 or more times
Consensus Panel Principles for selection Tried to select measures that worked across the continuum Can be interprofessional administration Can be administered in reasonable time at beginning and end of Rehabilitation Minimize cost of training Ideally available in English and French.
Ideal Measures for Outcomes You need a well developed evaluative measure with strong psychometric properties for use in clinical practice. - An evaluative measure assesses an individual or group at baseline and again at one or two points, usually to determine if change has occurred. It needs to be responsive to reflect change in patient status when it occurs.
continued What are important Psychometric properties of Measures?
Reviewing Reliability Reliability: the degree to which A measure is free from random error The observed score is different from the true score
Types of Reliability **Stability test-retest (longitudinal) Inter- and Intra-rater Assessed by correlation coefficients: Cronbach s alpha, Intra-class Correlation Coefficient (ICC), Kappa or Weighted Kappa (K or WK) etc.
Verifying Validity Validity: the extent to which a measure really measures what it claims to measure Reflects an absence of both random and systematic error (bias) Not an all or none property rather a matter of degree
Responsiveness What is it? Unlike reliability and validity it is not a traditional psychometric property. How is it defined? Many ways / no consensus. How is it evaluated? Even more ways. Terwee et al Qual Life Res 2003;12:349-62 How should it be reported? Quantitatively. How should it be interpreted? Two main approaches.
What are we trying to measure? Meaningful change to patient, to health professional, to payer Minimal Clinically Important Difference (MCID) (MID)
Some Thoughts on Choosing Outcomes Less is More avoid the temptation to want to answer all the questions on your first attempt Consider the time in administration Consider the time to train people Consider how much equipment is needed Consider whether it will change your practice
Some Thoughts on Choosing Outcomes Pick tools that are trans-disciplinary if possible Think about who is going to use the Results of all your work in collecting outcomes Is this for you as a clinician to plan your practice? Do you want to show those administration people that you make a difference? Do you want to show the fundors that you make a difference ( What do you think they are interested in?)
Conclusions 1. A core set of rehabilitation outcome measures for Canada is required as there is variability in current practice, need to consistently evaluate system and compare across provinces in Canadian Stroke Strategy 2. Issues that must be considered when selecting responsive outcome measures include the reliability, validity and responsiveness of the measure. These properties can now be estimated quantitatively
Today s Facilitator Stephanie Burgetz Learning & Development Specialist Human Resources The Ottawa Hospital
Change & Transition Clinging to Past Denial Focus on Future Commitment: External concern for new job realities Letting Go Anger Grief Internal Self Involvement Excitement New beginning Goodbye Rituals Fear Active Resistance Exploration Neutral Zone
What is Consensus? I can live with that and support it What it is not Winner vs. Loser Us -Them approach
Discussion Guidelines Listen to understand Minimize the side conversations Keep an open mind Stay focused on the issues Be tough on the issues but soft on each other Everyone s opinion is valued equally Participate Share the air time
Process - AM In your table groups, by domain: Review & Discuss each tool Identify Enablers and Barriers write on flip charts Discuss and make recommendations (e.g., guidelines for when it is best used) Your Goal: To make a recommendation as to whether this tool should be a core tool Work towards achieving a consensus Rank order (1, 2, 3 ) the tools (if applicable) Report back to the larger group
Process - PM Move to another domain keeping in mind the ratios identified Work through case studies using the recommended tool(s) Add any additional enablers/barriers, and recommendations You will have the opportunity to do this twice (total of two domains)
End of Day Decisions Going to the three domains that you looked at today, decide: I Support the adoption of this Outcome Measure Tool as a Core Tool to be used in the Champlain LHIN I am Neutral I Reject
Please place your dot sticker in ONE of the following sections: I Support I am Neutral I Reject
Wrap Up- Next Steps Report back to your Directors/Managers Findings to be shared with regional stakeholders Deliverables = Core Tools agreed upon Start using tools where possible Training of tools where needed Follow-up survey of implementation Review of parking lot items for next year +
Continental breakfast, nutrition breaks & lunch Compliments of