The Switch Access Measure (SAM) Where are we now and where we are going?

Transcription

1 The Switch Access Measure (SAM) Where are we now and where we are going? Dr. Toan Nguyen, Research Officer & Senior Rehabilitation Engineer Project team: Mrs Annabelle Tilbrook Mrs Myriam Sandelance Prof. Virginia Wright Senior Occupational Therapist (Chief Investigator) Profession Leader, Occupational Therapy (Assoc Investigator) Paediatric Physiotherapist and Senior Clinician Scientist, Holland Bloorview Kids Rehabilitation Hospital, Canada (Assoc Investigator) The R&D of the SAM was fully supported & funded by Novita

2 Why use switches? Provided to children with severe & multiple disabilities Successful switch use gives opportunities for independent: communication play/recreation learning/education mobility control of the environment

3 Finding the appropriate switch Assessment is complex and requires consideration of a range of factors: Child s needs & abilities: Motor (physical) Visual Cognitive (process) Which reliable body part(s) should be used? Which switch? multitude of possibilities Where and how should the switch be positioned? Twitch Switch Activation by small muscle movement

4 How could a switch access measure help us? Give clear picture of switch users abilities and challenges Avoid prolonged trial and error approach used to determine solutions Inadequate evaluation carries risk of sub-optimal solutions and compromised participation abilities Changes in abilities in this client group happen slowly and in small increments small important changes often missed without targeted measurement THE BAD NEWS No reliable and valid measure that comprehensively covered all these needs HENCE, we developed the NSASA* (conceived in ) now called the SAM. * We acknowledged the earlier work on the development of the NSASA by UniSA s School of Health Sciences Occupational Therapy Honours Students.

5 What is the SAM?

6 The SAM Form Commercial In Confidence

7 The SAM Form - example Participant 3: Mean Motor Score = 3.5/4 Mean Visual Score = 1/4 Mean Process Score = 2.1/4 Interpretation: Minimal issues with Motor access skills Focus on intervention for visual limitations and further development process skills

8 The reliability study Objective: Determine reliability with 2 Occupational Therapist (OT) assessors trained to criterion on the SAM Participants: 20 current switch users (3 15 years, mean age 10.4) 12 (Cerebral Palsy (CP); MACS* Level III-IV); 8 (other diagnoses: MACSequivalent manual ability) 6 single-switch users; 14 two-switch users *Manual Ability Classification System

9 The reliability study Method: The 2 OT assessors reviewed and scored 20 videos (10-20 mins) of children interacting with their switching technology Their SAM scores were analysed to determine inter-rater reliability for overall Total score and sub-scale scores (Motor, Visual and Process ) 2-3 weeks later each OT assessor re-scored a subset of 10 videos (intra-rater reliability) Each OT assessor rated their scoring confidence level on a 100-point scale 2 of SAM s OT developers also scored all 20 videos for inter-rater reliability benchmarking

10 Results Inter-rater reliability Intraclass Correlation Coefficient (ICC) = degree of consistency/agreement among measurements. Aim: ICC>.80; 95% Confidence Interval (CI)>0.5 % mean (SD) score for all items Reliability estimates SKILL ITEMS Assessor 1 Assessor 2 ICC 95% CI of ICC Total 51.0 (21.8) 49.7 (18.1) Motor 56.7 (19.8) 56.0 (17.0) Visual 41.9 (25.4) 46.2 (28.7) Process 49.1 (21.2) 45.9 (23.8) ICC = Intraclass Correlation Coefficient - degree of consistency/agreement among measurements SD = Standard deviation

11 Results Inter-rater reliability The 2 OT Assessors: n = 20 children's videos % mean (SD) score for all items Reliability estimates SKILL ITEMS Assessor 1 Assessor 2 ICC 95% CI of ICC Total 51.0 (21.8) 49.7 (18.1) Motor 56.7 (19.8) 56.0 (17.0) Visual 41.9 (25.4) 46.2 (28.7) Process 49.1 (21.2) 45.9 (23.8) The 2 OT Developers - n = 20 children's video % score for all items Reliability estimates SKILL ITEMS Developer 1 (SD) Developer 2 (SD) ICC 95% CI of ICC Total 50.9 (22.3) 53.7 (22.4) Motor (16.9) (17.8) Visual 44.4 (34.0) 45.9 (35.9) Process 46.0 (28.5) 49.4 (28.3)

12 Results - Intra-rater reliability and clinical utility The 2 OT assessors (n = 10 children's videos rated) SKILL ITEMS Assessor 1 ICC 95% CI Assessor 2 ICC 95% CI TOTAL Motor Visual Process Clinical Utility - Confidence measures (/100) Assessor 1 Mean (SD) Assessor 2 Mean (SD) Overall 84.5 (6.9) 86.8 (6.0) Motor 82.0 (8.2) 85.3 (12.7) Visual 85.9 (8.7) 86.5 (12.2) Process 85.8 (6.4) 88.2 (8.8)

13 Discussion Process section demonstrated excellent reliability for both assessor groups Visual section also achieved good to excellent reliability notable as it only has 2 items - just a few rating inconsistencies could adversely affect reliability Motor section still challenging for OT assessors but not the OT developers. Not an issue with the SAM itself - likely a training challenge related interpretation of several Motor item skill definitions. These are now further clarified in our training. The SAM was able to measure differences in abilities among the children. Achieved the goal of creating a clear picture of switch use strengths and challenges. Assessors confidence was high indicating ease and comfort with use (clinical utility) Next step is validation as an outcome measure, evaluating SAM s ability to: compare between two switching access technologies, and detect change in switch use competency over time

14 Implications for clinical use To use the SAM, given the complexity of the switch user s performance, requires in-depth clinician training to optimise competency and confidence. Some practical knowledge about switch use is essential before a clinician undertakes training.

15 Training and Certification Clinicians need to be certified before using the SAM Our training process: Two-day face-to-face training workshop Post training online module & assessment Attendees scores our videos and submits of own scored video Certification once this process is completed First ever workshop to be held on 27 & 28 April 2018 at Novita For further information about SAM contact:

16 Acknowledgements We would like to thank the children and their families for participating in this study and also the two Occupational Therapists (OTs)who have agreed to be assessors for the SAM reliability study. We acknowledged the earlier work on the development of the NSASA by UniSA s School of Health Sciences Occupational Therapy Honours Students. We thank Novita for their generous support of the reliability study and the subsequent development of the SAM course and online training materials.

17 Contact details Mrs Annabelle Tilbrook (Senior Occupational Therapist) (Chief Investigator) (Course presenter) Mrs Myriam Sandelance (Profession Leader, Occupational Therapy) (Assoc Investigator) (Course presenter) Dr. Toan Nguyen, Research Officer/Senior Rehabilitation Engineer Prof. Virginia Wright, Paediatric Physiotherapist and Senior Clinician Scientist, Holland Bloorview Kids Rehabilitation Hospital, Canada (Assoc Investigator)

18 Finally, something to ponder on For people without disabilities, technology makes things easier. For people with disabilities, technology makes things possible! IBM training manual 1991