Meet CESR: The Communicative Effectiveness Survey Revised For People With Parkinson s Disease And Dysarthria

Size: px

Start display at page:

Download "Meet CESR: The Communicative Effectiveness Survey Revised For People With Parkinson s Disease And Dysarthria"

Melissa Holland
5 years ago
Views:

1 Meet CESR: The Communicative Effectiveness Survey Revised For People With Parkinson s Disease And Dysarthria Neila J. Donovan, Ph.D., CCC-SLP Louisiana State University Department of Communication Sciences and Disorders ASHA 2014

2 Disclaimer Session #1178 Research funded by the Louisiana Board of Regents Research Competitiveness Program Award # LEQSF ( )-RDA- 07 NIH Student Loan Repayment Program 2-year award ( ) ASHA Presentation 11/20/14 2

3 Thanks to My Rasch Compatriots! Kathryn Yorkston & Carolyn Baylor University of Washington Pat Doyle & Will Hula VA Geriatric Research, Education and Clinical Center (GRECC) & University of Pittsburgh ASHA Presentation 11/20/14 3

4 Goals of Presentation In the Program Describe major elements of the study design Discuss the author s conclusions Identify possible implications for practice In the Presentation Describe rationale for studying communicative effectiveness Briefly describe first 2 phases of CESR refinement Focus on Phase III Field testing Psychometric Analysis Discuss using CESR in clinical practice and research ASHA Presentation 11/20/14 4

In Motor Speech Disorders THE IDEAL SLPs aim to improve communicative effectiveness and thus improve quality of life of people with communication disorders SLPs strive to assess and treat

5 In Motor Speech Disorders THE IDEAL SLPs aim to improve communicative effectiveness and thus improve quality of life of people with communication disorders SLPs strive to assess and treat communication disorders across the WHO ICF domains of impairment, activity and participation. THE REALITY Difficult to measure improved communicative effectiveness without valid and reliable assessments. Difficult to assess or treat disorders when very few evidence based treatments and assessments exist at the levels of activities and participation. ASHA Presentation 11/20/14 5

6 Research Aim To develop a measure of communicative effectiveness for individuals with PD and dysarthria in the participation domain of the ICF model Valid Reliable Precise Efficient ASHA Presentation 11/20/14 6

7 What IS Communicative Effectiveness? Communicative effectiveness is the person's ability to actively and efficiently get his/her message across successfully at home, or in community settings to participate in life roles. (Hustad,1999) ASHA Presentation 11/20/14 7

8 Why Study Communicative Effectiveness in People with PD? Communicative effectiveness affects a person s access to and quality of medical care (Hoffman, et.al., 2005) People in poor health become isolated, depressed (Dalton, et al., 2003), which may lead to loss of independence As people age the chance being diagnosed with a neurologic disorder increases (AHA, 2004; PD Society, 2005) Parkinson disease typically diagnosed in a person s 60s (Watts & Koller, 1997) ~ 80% of people with Parkinson disease (PD) have dysarthria and swallowing problems (Miller, 2007)

9 Communicative Effectiveness Survey (CES) 8 items, 7-unit equal appearing interval scale (Yorkston et al, 1999) Item level psychometric properties tested in 66 individuals with PD and dysarthria (Donovan et al., 2007) Preliminary construct validity study (Donovan et al., 2008) Feedback at end of study need for better understanding of communicative effectiveness Environmental and personal factors ASHA Presentation 11/20/14 9

10 3 Phases/Studies Phase I Older people with and without PD and dysarthria define communicative effectiveness (Donovan & Macauda, manuscript in revision) Phase II Refining the Communicative Effectiveness Survey (CES) Item Pool Expert Input, Focus Groups of participants from Phase I Result 27-item Communicative Effectiveness Revised (CESR) Phase III CESR Paper-pencil field testing Analysis ASHA Presentation 11/20/14 10

11 Phase I Defining Communicative Effectiveness Qualitative Study 15 older people with and without PD and dysarthria define communicative effectiveness(ce) Results Effective communication has to be a two-way street must listen and talk. CE requires attention and memory Must be able to present information in a way the partner understands, be knowledgeable about the topic, be concise, use appropriate gestures, tone of voice and facial expressions. CE requires adequate hearing, and volume and precision of speech (Donovan & Macauda, manuscript in revision) ASHA Presentation 11/20/14 11

12 Phase II CESR Item Development & Refinement Input from Phase I guided further item development Results A priori rating agreement of item difficulty hierarchy There was a strong positive correlation among 3 raters r s (24) =.879 and.846 respectively, both with p <.01. Expert Review (n = 4) Revised 8 items for clarity and jargon Added 2 less difficult items Focus Group Review (n = 5 PD, 5 caregivers) Revised 10 items to improve readability (shortened), and clarity (provided examples) CES 27 items, 4-unit rating scale ASHA Presentation 11/20/14 (Donovan, manuscript in preparation) 12

13 Phase III - Research Questions 1. Do CESR data meet the assumptions for Rasch analysis? 2. How well do the CESR items fit the Rasch model? 3. How well does CESR measure communicative effectiveness, based on item-level psychometric properties? 4. How sensitive is CESR to different ability levels? 5. Can CESR ratings differentiate PD from NOPD respondents? ASHA Presentation 11/20/14 13

14 Participant Demographics Age (years) (6.87) Sex 1.35 (.48) Race/Ethnicity 56 Caucasian 4 African American Education Level 4.60 (1.53) Years in LA (21.0) MMSE (2.22) GDS 3.05 (2.82) Apathy 9.70 (5.15) H&Y 1.9 DRS ASHA Presentation 11/20/ PD NOPD t(df=121) p (8.83) 1.81 (.40) 59 Caucasian 4 African American 4.32 (1.62) (22.81) (1.47) 1.73 (1.90) 7.0 (4.5) <.05*

15 1. Do the CESR data meet the assumptions for Rasch analysis? Unidimensionality Exploratory Factor Analysis (Mplus v. 6.11) Data fit criteria for one factor EFA Fit Actual Ideal Statistics RMSEA.09 <.10 CFI.93 >.90 TLI.92 >.90 Muthén & Muthén, 2011 Local Independence Inter item residual correlations Fisher s Z transformation ( 2) Range 0.39 to 0.24 Positive evidence for local independence

16 2. How well do CESR items fit the model? Rasch analysis (rating scale model) showed no item misfit all 27 items remained in the subsequent analyses Significant positive correlation between proposed and actual item hierarchies (r s =.96, df = 25, p <.01). Demonstrates evidence for content validity of the CESR Demonstrates the benefit of participant feedback

17 Communicating your wants and needs to a family member at home. ASHA Presentation 11/20/14 CESR Analysis Item Difficulty 5 Easiest Items 5 Items Mean Difficulty 5 Most Difficult Items Speaking when you are emotionally upset. Having a conversation with family members or friends at home. Having a conversation with a familiar person over the telephone. Having a conversation outdoors in a quiet environment (for example a café). Having a conversation with a health care provider. Participating in conversations with a small group of people (for example several couples over dinner). Having a conversation while riding as a passenger in a car. Engaging in small talk with someone you've just met. Communicating your wants and needs to a family member away from home. Speaking when you are angry. Having a conversation indoors in a noisy environment (for example a social gathering). Talking over the telephone with someone who has an unfamiliar accent. Having a conversation when you re tired or fatigued. Having a conversation outdoors in a noisy environment (for example a party held outdoors). 17

18 MEASURE PERSON - MAP - ITEM <more> <rare> 4 + PD326 3 PD428 PD626 T+ PD286 PD356 PD266 PD517 PD347 A b 2 + i l PD207 PD496 T S ConvNoi PD087 ConvFat ConvPhA PD117 PD476 PD486 ConvHOH ConvNoi SpkUpse 1 PD047 PD068 PD296 PD558 +S PD396 PD406 PD446 ConvF2F ConvUHO PD237 PD276 PD307 PD377 EnterCo TlkRoom PD456 PD467 PD157 PD437 M SpkAng ipd166 PD317 PD337 PD416 PD527 PD546 WantsAw t y D i f f i c u 0 PD038 PD106 PD188 PD226 +M SmTkNew PD217 PD566 PD587 ConvPas ConvSmG PD136 PD616 PD027 PD368 PD388 ConvChi PhoneSv PD017 PD077 PD247 PD536 Call he ConvCou SpkExci l PD257 S Greet -1 +S ConvMDR F2FSvc t y PD177 PD148 ConvHom ConvPho ConvQui WantsHo PD597 PD056 PD096 PD578 T -2 + PD127 T PD <less> <frequent> 3. How well does CESR measure communicative effectiveness, based on item level psychometric properties? Item Mean = 0 Person Mean =.41 No ceiling or floor Adequate logit range ~ 2 to +2 Logits items 3 to + 3 Logits ability 18

19 4. How sensitive is CESR to different ability levels? Person reliability (comparable to Cronbach s alpha) was high at.93 Resulting in a person separation index of 3.65 = 3.65 statistically significantly different groups based on ability Demonstrates that CESR is sensitive to range of ability levels

20 Standard Error (mean =.21).1O 0 ASHA Presentation 11/20/14 20

21 5. Can CESR ratings differentiate PD from NOPD respondents? Statistically significant difference PD group (mean = sd =13.654) NOPD group (mean = , sd = ) t(121) = , p <.001. Large effect (standardized mean difference) (Cohen, 1988) d = (95% CI to -0.81) Demonstrates evidence for CESR construct validity Corroborates findings of Donovan et al., 2008

22 Factors Used to Determine Item level Psychometrics CES (Donovan et al., 2008) CESR Sample size > 50 N= 66 N = 60 Mean Match Items/Persons 0/.05 0/.41 Person to Measure Correlation (comparable to Cronbach s α) Unidimensionality Local Independence In fit Statistics Items In fit Statistics Persons Principal Components Analysis 75.1% variance accounted for MnSq =.98 MnSq =.99 EFA LD MnSq = 1.0 MnSq = 1.0 Significant Levels of Difficulty Actual Item hierarchy agreement with theoretical item hierarchy 6/8 (±1 item) (r s =.96, df = 25, p <.01) Reliable use of rating units 7 point scale not used reliably Revised 4 point scale *4 unit scale was used reliably for all but 2 items

for others to understand Easy to compare pretreatment/post treatment Easy to compare

23 Clinicians & Researchers Clients with PD and dysarthria Family members/caregivers Rehabilitation Team Payors Converted Logit Interval Scale to Interval Scale Easy for others to understand Easy to compare pretreatment/post treatment Easy to compare clients Easy to identify what a person can do and what more needs to be done ASHA Presentation 11/20/14 23

24 Limitations IRT continues to evolve Many using more complex IRT Models Increasingly larger sample sizes needed I used Rasch Analysis Information benefit Permits smaller sample size because it tests a simpler model A couple of surprises in item hierarchy Conversation w/healthcare professional ASHA Presentation 11/20/14 24

25 1. Collect CESR ratings from a very large group Less controls, broader range of problems 2. CESR validation continues Does CESR differentiate between PD on and off medication on and off Deep Brain Stimulator Those with PD and dysarthria from those with PD alone Changes in communicative effectiveness longitudinally Is CESR sensitive to change pre-post treatment Clinically significant differences 3. CAT and/or short form development ASHA Presentation 11/20/14 25

26 Researchers and clinicians now have a way to measure treatment outcomes for people with PD and dysarthria in the participation domain. CESR is an objective measure numbers relay meaningful information CESR is valid, reliable, and sensitive to different ability levels of people with PD and dysarthria (per Rasch Analysis) ASHA Presentation 11/20/14 26

27 Acknowledgments Nina Simmons Mackie, PhD Mark Macauda, PhD Nicole Simmons, M.A., CCC SLP COR Team Heidi Huckabee Michels Ashley Marino Elizabeth Fall Amanda Cross Meghan Collins Kelsey Copperberg ASHA Presentation 11/20/14 27

28 What we wish, we readily believe, and what we ourselves think, we imagine others think also. Julius Caesar ASHA Presentation 11/20/14 28

The Functional Outcome Questionnaire- Aphasia (FOQ-A) is a conceptually-driven

Introduction The Functional Outcome Questionnaire- Aphasia (FOQ-A) is a conceptually-driven outcome measure that was developed to address the growing need for an ecologically valid functional communication