From CG-CAHPS Visit to PCMH: Issues and Implications for MN Health Care Homes Dale Shaller, MPA Shaller Consulting Group Meeting with Minnesota Department of Health March 14, 2013
Overview What is the PCMH Survey? Why Transition from CG-CAHPS Visit to PCMH? Transition Issues and Implications Discussion
CAHPS Clinician & Group Survey Multiple versions to meet user needs 12-month version Visit version Patient-centered medical home (PCMH) version Adult and child versions Core questions are the same across versions Supplemental questions can be added for specific topics
CG-CAHPS PCMH Version
CG-CAHPS Survey Versions
PCMH Survey Domains Adult Survey Core Composites Access Communication Office Staff Provider Rating PCMH Composites Comprehensiveness Self-Management Coordination Shared Decision Making Child Survey Core Composites Access Communication Office Staff Provider Rating Child Development Prevention Advice PCMH Composite Health Goals 6
CG-CAHPS PCMH Survey Item Count
Factors Favoring Use of PCMH Alignment with external reporting requirements Medical home recognition Pay-for-performance Public reporting Other Federal initiatives Characteristics of the survey itself Content Psychometric properties Response rates Cost considerations
Medical Home Recognition
Pay for Performance
Public Reporting: Federal and State
AF4Q Markets Reporting 12-Month or PCMH
Maine Patient Experience Matters
Other Federal Initiatives
Survey Content PCMH survey developed to meet needs of medical home assessment and improvement but applies to all primary care Expanded content provides richer picture useful for both assessment and improvement Experience shows that 12-month timeframe can be used effectively for improvement Ex: Dean, MGH, UCLA, HealthPlus Visit survey is really a hybrid anyway (includes the 12-month Access domain)
Psychometric Properties Visit survey 3-point response scales (communication and office staff) have large ceiling effects Items are too easy to endorse and thus lead to a high percentage of top box responses 12-month 4-point Never-to-Always response scales have greater discrimination Useful for both external assessment and internal improvement
CG-Visit Percentile Distributions
Response Rates Longer surveys do not depress response rates Medicaid respondents are as likely to complete a long survey (95 items) as a short survey (23 items)* Vendors report that adding CAHPS items to proprietary surveys (HCAHPS and CG-CAHPS) actually increases response rates Recent experience with PCMH survey shows response rates averaging between 40-44% *Gallagher P. and Fowler F. Size doesn t matter: response rates of Medicaid enrollees to questionnaires of varying lengths. Center for Survey Research, University of Massachusetts at Boston.
Cost of Administration Average cost per completed survey varies widely 4-fold variation in cost/complete for 2012 MN Visit survey ($5-$20) Statewide PCMH survey cost in Maine ranges between $7.50 - $9.50/complete Statewide 12-month core survey in Puget Sound averaged $7/complete Email administration can reduce data collection costs dramatically regardless of survey version
Recommendations for Minnesota Require one survey: CG-CAHPS 12-month core Clinics seeking HCH certification can add PCMH supplemental items Coordinate as much as possible with other external requirements (NCQA, CMS, HRSA, etc.) Encourage health systems to align internal survey content with 12-month and PCMH content Continue to accept all comparable data collection modes
Appendix Comparison of CG-CAHPS Visit Questions Added to the CG-CAHPS PCMH Survey
CG-CAHPS Study Objectives Test effects of adding core CG-Visit items to the PCMH survey If equivalency is found, it would be possible to compare CG-Visit core measures across all clinics HCH clinics could use PCMH + Visit core Other clinics could use just the Visit core Consistent with internal surveys and previous voluntary reporting initiatives
Split Sample Study Design
Comparison of Study Versions
Participating Clinic Sites
Sampling and Data Collection NRC Picker conducted sampling and data collection Random sample drawn at site level Adult English-speaking primary care patients At least one visit in previous 12 months Target of 250 completed surveys per study arm for each of the 6 clinic sites Two-wave mail methodology Field period: May- June 2012
Results: Response Rates
Results: Respondent Characteristics
Results: Respondent Characteristics (continued)
Results: Top Box Scores
Results: Mean Scores
Results: Impact on Clinic Ranking
Conclusions: Visit vs. PCMH+Visit No differences are found when comparing the two rating questions Small but consistent differences are found when comparing the composite scores Access, communication, and office staff measures Visit scores are always higher than PCMH+Visit Top box scores are higher by 2-4% points Differences could affect clinic rankings Bottom line: Visit core survey items are not comparable across versions