Satisfaction with care is an important, patient-centered

Transcription

1 MANAGERIAL The Assocition of Mentl Helth Progrm Chrcteristics nd Ptient Stisfction Austin B. Frkt, PhD; Jodie Trfton, PhD; nd Steven D. Pizer, PhD Stisfction with cre is n importnt, ptient-centered mesure of helth system performnce becuse it cn identify gps in qulity tht could be missed by other mesures, 1 help detect cross-popultion disprities, 2 nd serve s ctlyst for qulity improvement. 3,4 Yet, the reltionship between ptient stisfction nd qulity of cre, lthough generlly positively correlted, 5 is not fully understood. 6 Studies hve found ptient stisfction to be ssocited with hospitl process qulity 7 ; lower rtes of redmissions, hert ttck mortlity, 8,9 nd surgicl qulity 10 ; nd better long-term outcomes. 11 Fenton et l 12 found it lso positively correlted with higher helthcre utiliztion, costs, nd ll-cuse mortlity. Assessment of wht objective helth system mesures drive ptient stisfction is importnt for 2 resons. First, becuse it relies on ptient surveys, stisfction remins expensive nd chllenging to mesure t high frequency. 13 However, mny other mesures of helth system performnce re esily obtined t high frequency from dministrtive dt (eg, process qulity mesures or redmissions). If stisfction is highly correlted with these other dministrtive mesures, they offer supplements to stisfction surveys wys to monitor nd improve spects of cre relted to stisfction during longer intervls between mesurement. Second, stisfction is not directly modifible; improvements must come from chnges in the processes of cre or investments in services tht ptients vlue. For these resons, we studied the reltionships between set of ptient stisfction mesures nd lrge collection of mentl helth progrm chrcteristics for ptients with recent mentl helth encounter in the Veterns Helth Administrtion (VHA), the lrgest provider of mentl helth cre in the United Sttes. 14 Prior work hs documented vrition in stisfction cross VHA ptients with psychitric dignoses. Rosenheck et l 15 found tht VHA ptients who were dischrged from the hospitl with primry psychitric or substnce use dignosis were more likely to be stisfied with their cre if they were older, in better helth, or hd long length of sty. ABSTRACT OBJECTIVES: Stisfction with cre is n importnt ptient-centered domin of helth system qulity. However, stisfction mesures re costly to collect nd not directly modifible. Therefore, we ssessed the reltionships between veterns stisfction nd mesures of modifible spects of Veterns Helth Administrtion (VHA) mentl helth cre progrms. STUDY DESIGN: For smple of 6990 ptients who received mentl helth cre from the VHA in 2013, we used survey nd dministrtive dt to investigte the ssocition of suite of ccess nd encounter stisfction mesures with lrge collection of mesures of progrm chrcteristics. METHODS: We estimted risk-djusted correltions between 6 stisfction mesures (cross 2 domins: ccess nd encounter stisfction) nd 28 mentl helth cre progrm chrcteristics (cross 4 domins: progrm rech, psychosocil service ccess, progrm intensity, nd tretment continuity). RESULTS: We found tht stisfction with ccess to cre ws higher thn experiences with cre encounters, but tht brod mesures of mentl helth cre progrm rech nd intensity were positively ssocited with both kinds of stisfction. No mesures of psychosocil service ccess were positively ssocited with ccess nd encounter stisfction. Most mesures of tretment continuity were consistently nd positively ssocited with both kinds of stisfction. CONCLUSIONS: As the VHA strives to increse ccess to, nd provision of, mentl helth cre, policy mkers nd progrm mngers should be wre tht stisfction with cre, s it is currently mesured, my not rise s more ptients initite tretment, unless continuity of cre is mintined or enhnced. Am J Mng Cre. 2017;23(5):e129-e137 THE AMERICAN JOURNAL OF MANAGED CARE VOL. 23, NO. 5 e129

2 MANAGERIAL nd Lerning (SAIL) includes 25 dministrtive TAKEAWAY POINTS dt bsed performnce mesures relted to Brod mesures of mentl helth cre progrm rech nd intensity nd most mesures of ccess, continuity of cre, ptient sfety, nd tretment continuity were consistently nd positively ssocited with ptient stisfction. qulity of cre t fcility level. 25 We used the Becuse psychosocil services re cliniclly vluble, policy mkers nd mngers should 2013 MHIS Dshbord nd precursors to SAIL not interpret lck of ssocition with stisfction to justify reducing their vilbility. mentl helth domin report metrics (MHIS Policy mkers nd mngers should be wre tht stisfction with cre, s currently mesured, my not rise s more ptients initite tretment, unless continuity of cre is nd SAIL re refined on n ongoing bsis), mintined or enhnced. shred with us by OMHO, to predict ptientlevel stisfction responses to the 2013 SHEP. Policy mkers nd mngers should continue to trck ptient stisfction nd to specificlly trget stisfction with mentl helth cre. SHEP surveys ptients with recent VHA encounter (the index encounter). To merge Burnett-Zeigler et l 16 reported tht VHA ptients with psychitric dignoses who were younger, nonwhite, or lower-income; hd istics, we ssocited ech SHEP respondent with the VHA fcility fcility-level MHIS/SAIL-bsed mentl helth progrm chrcter- service-connected disbility; or hd received posttrumtic stress where they hd the index encounter. For risk djustment, we lso disorder (PTSD) or substnce use disorder dignosis were less merged, t the ptient level, demogrphic nd Elixhuser 26 comorbidity dt from VHA dministrtive files. likely to be stisfied with their cre. Hepner et l 17 exmined perceptions of behviorl helth cre mong VHA ptients who received Our interest ws in the reltionships between mentl helth cre mentl helth cre. Seventy-four percent sid they were helped by progrm chrcteristics nd ptient stisfction, so we used dt tretment, but only 32% reported n improvement in symptoms. from subset of SHEP respondents those with recent mentl Holcomb et l 18 found tht the stisfction of midwestern VHA helth encounter. To ccomplish this, we restricted the SHEP smple ptients with psychitric dignoses positively correlted with better self-reported outcomes. Ptients with co-occurring substnce s encounters for mentl helth. Becuse most SHEP respondents to respondents with index encounters in the sme qurter nd yer use nd psychotic disorders who were treted in VHA residentil complete nd return surveys 2 or more months fter the index visit, substnce use disorder tretment progrms tht hd more positive this pproch gurnteed tht the mjority would hve hd recent perceptions nd stisfction exhibited greter enggement in cre mentl helth encounter prior to providing stisfction feedbck. nd experienced better outcomes. 19 Finlly, Hoff et l 20 reported Therefore, lthough some of the survey questions sk ptients to lower levels of stisfction mong VHA ptients with psychitric report stisfction bsed on the prior 12 months of cre, it would dignoses thn those with medicl dignoses. be likely tht ptients impressions were more hevily influenced by their most recent mentl helth encounter. Nevertheless, unlike prior nlyses of stisfction mong VHA mentl helth ptients, METHODS 1,17 we were not directly ssessing stisfction with mentl helth cre Since 2002, the VHA Office of Qulity nd Performnce hs fielded services. Our finl smple included 6990 ptients cross 165 VHA the Survey of Helthcre Experiences of Ptients (SHEP), n ongoing fcilities, lthough not ll ptients responded to ll survey items monthly mil survey of ptients experiences during their most due to SHEP question skip ptterns (eappendix Tble A2). All recent VHA encounter. Modeled on the Consumer Assessment of nlyses were conducted t the ptient level. Helthcre Providers nd Systems survey nd bsed on strtified design tht selects from the specilty cre domins s well s new Ptient Stisfction Vribles nd estblished primry cre ptients within ech fcility, 21 SHEP Stisfction with timeliness of cre, which we termed ccess smples bout 30,000 mbultory cre ptients ech month who stisfction, is mesured by SHEP sking respondents how often visit the VHA nd who were not surveyed in the prior yer. The 2013 they were ble to obtin needed cre right wy nd were ble version of SHEP is our source of stisfction mesures, with n overll response rte of bout 44% nd slightly higher response rtes cre, excluding the times they needed urgent cre. Access to VHA to get VHA ppointments s soon s they thought they needed for mles nd substntilly higher response rtes for older ptients tests or tretments is mesured by SHEP sking how esy it ws (eappendix Tble A1 [eappendices vilble t jmc.com]). to ccess tht cre in the lst 12 months. Response options for In 2010, the Deprtment of Veterns Affirs (VA) s Office of the bove 3 mesures included lwys, usully, sometimes, Mentl Helth Opertions (OMHO) implemented the Mentl Helth or never. There ws no crdinl mening to these ctegoricl Informtion System (MHIS) Dshbord, 22 which includes fcilitylevel qulity metrics consistent with the gols of the VA s Uniform grdtions in the ordinl scle Specificlly, following Prentice responses. Therefore, we dichotomized them to eliminte fine Mentl Helth Services Hndbook. 23,24 In ddition, the mentl et l, 27 we dichotomized these to 1 for responses of lwys or helth domin of the VHA Strtegic Anlytics for Improvement usully nd 0 otherwise (Tble 1). e130 MAY

3 Mentl Helth Cre nd Ptient Stisfction Encounter stisfction, which mesures stisfction with the cre received or provider seen, is mesured by SHEP sking respondents to rte VHA helthcre in the lst 12 months on scle of 0 to 10, where 0 indictes the worst helthcre possible nd 10 the best helthcre possible. Stisfction with the respondents personl doctor/nurse is lso ssessed on 0-to-10 scle. For the sme resons given bove, we dichotomized these to 1 for responses of 9 or 10 nd 0 otherwise. 27 Stisfction with the most recent VHA visit is ssessed on SHEP with scle rnging from 1 to 7, where 1 indictes completely disstisfied nd 7 completely stisfied. We dichotomized this to 1 for responses of 6 or 7 nd 0 otherwise. 27 Progrm Chrcteristics Vribles The mentl helth progrm mesures we considered re listed nd defined in Tble 2 nd re orgnized into 4 res of focus: 1) progrm rech (eg, the proportion of ptients receiving mentl helth cre), 2) psychosocil service ccess (eg, the proportion of ptients inititing psychosocil tretment or psychotherpy), 3) progrm intensity (eg, the number of encounters per yer), nd 4) tretment continuity (eg, the proportion of dischrged ptients with follow-up within 7 dys). Within ech re, we exmined 5 or more performnce metrics. Trnsitionl work visits nd supportive employment visits mentioned in Tble 2 re occuptionl therpy tretment modlities. 28,29 TABLE 1. Ptient-Level Demogrphics nd Stisfction Mesures Vrible Description Men SD Demogrphics Age In yers Mrried 1 = mrried, 0 = not mrried Femle 1 = femle, 0 = mle Blck 1 = blck, 0 = nonblck Access stisfction Got needed ppointment Got wnted ppointment Got needed cre Encounter stisfction VHA rting Clinicin rting Visit rting In the lst 12 months, when you needed cre right wy, how often did you get cre s soon s you thought you needed? (1 = lwys or usully ; 0 = sometimes or never ) In the pst 12 months, not counting the times you needed cre right wy, how often did you get n ppointment s soon s you thought you needed? (1 = lwys or usully ; 0 = sometimes or never ) In the pst 12 months, how often ws it esy to get the cre, tests, or tretment you thought you needed through the VHA? (1 = lwys or usully ; 0 = sometimes or never ) Using ny number from 0 to 10, where 0 is the worst helthcre possible nd 10 is the best helthcre possible, wht number would you use to rte ll your VHA helthcre in the lst 12 months? (dichotomized to 1 for response of 9 or 10 nd 0 otherwise) Using ny number from 0 to 10, where 0 is the worst personl doctor/nurse possible nd 10 is the best personl doctor/nurse possible, wht number would you use to rte your personl VHA doctor/ nurse? (dichotomized to 1 for response of 9 or 10 nd 0 otherwise) All things considered, how stisfied were you with the VHA during your recent visit? (Responses were on 7-point scle where 1 = completely disstisfied nd 7 = completely stisfied; dichotomized to 1 for response of 6 or 7 nd 0 otherwise.) VHA indictes Veterns Helth Administrtion. Number of responses nd skip ptterns provided in eappendix Tble A2. 76% 43% 76% 43% 78% 42% 53% 50% 66% 47% 47% 50% RESULTS Descriptive Sttistics Tbles 1 nd 2 report mens of demogrphic control vribles, dependent vribles (ptient-level stisfction), nd key independent vribles (fcility-level progrm chrcteristics). eappendix Tble A3 reports the mens for dignostic risk-djustment vribles. Tble 1 shows tht the verge ge of ptients in our smple ws 62 yers, 55% were mrried, 8% were femle, nd 12% were blck. In ddition, 17% of our smple hd n lcohol use disorder; 9%, drug use disorder, 37%, psychosis; nd 48%, depression dignosis (ll s defined by Elixhuser 25 nd listed in eappendix Tble A3). These figures were higher thn the generl popultion becuse we delibertely selected smple of ptients with VHA mentl helth visit. Tble 1 lso shows tht, cross our smple, most ptients reported high levels of stisfction for ll but 1 mesure. Tble 2 shows fcility-level progrm chrcteristics orgnized by the 4 domins. The fcility-level progrm chrcteristics in Tble 2 were ech computed by OMHO on the full smple of ptients implied by ech chrcteristic, not just the ptients in our study smple. For exmple, the progrm rech chrcteristic of PTSD is defined s % of ptients with PTSD who receive specilty outptient cre for PTSD. This mens tht this mesure cptures, for ech fcility nd yer, the percentge of ptients with PTSD seen by the fcility in tht yer who received specilty outptient cre for PTSD. THE AMERICAN JOURNAL OF MANAGED CARE VOL. 23, NO. 5 e131

4 MANAGERIAL TABLE 2. Fcility-Level Progrm Chrcteristics (independent vrible) Definitions nd Mens Vrible Description Men SD Progrm rech MH ptients % of veterns service-connected for MH condition who reside in the fcility ctchment re who received VHA MH cre 51% 4% Trns l work % of ptients with SMI with trnsitionl work visits b 2% 2% Sup empl % of ptients with SMI who received supportive employment visits b 5% 6% Homeless % of homeless VHA ptients who received ny MH cre 89% 5% PTSD % of ptients with PTSD who received specilty outptient cre for PTSD 20% 9% SUD spec % of ptients with SUD who initited intensive tretment in specilty setting 8% 6% Cse mgmnt % of ptients with SMI who received MH intensive cse mngement for psychosis 3% 3% Psychosocil service ccess SMI % of ptients with SMI who initited psychosocil tretment or psychotherpy 61% 10% PTSD % of ptients with PTSD who initited psychotherpy 59% 9% SUD psy % of ptients with SUD who initited psychosocil tretment or psychotherpy in ny setting 47% 7% Depression % of ptients with depression who initited psychotherpy 39% 7% Progrm intensity Overll Number of MH encounters per VHA ptient MH ptients Number of MH encounters per ptient with ny MH encounters PRRC Number of encounters t psychosocil rehb nd recovery centers per VHA ptient Homeless Number of MH encounters per homeless vetern with ny MH encounters SUD Weeks of intensive outptient SUD tretment per ptient with ny intensive outptient SUD tretment visits Trns l work Number of trnsitionl work visits b per ptient with ny trnsitionl work visits Sup empl Number of supportive employment visits b per ptient with ny supportive employment visits Tretment continuity Num resid Number of outptient visits in the 6 months post residentil MH dischrge Num inpt Number of outptient visits in the 6 months post inptient MH dischrge F/u resid % of ptients with 7-dy follow-up fter residentil dischrge 52% 23% Gp % of ptients with n MH outptient visit in 1 yer who went 6 months without second visit 43% 5% SMI % of ptients with SMI with 8 psychotherpy or psychosocil tretment visits in 14 weeks 8% 4% PTSD % of ptients with PTSD with 8 visits for PTSD psychotherpy in 14 weeks 14% 5% Inptient detox % of ptients who hd inptient detox with outptient follow-up within 7 dys 42% 12% Outptient detox % of ptients who hd outptient detox with outptient follow-up within 7 dys 37% 16% SUD % of ptients with SUD with 8 SUD psychotherpy or psychosocil tretments in 14 weeks 21% 8% Depression % of ptients with depression with 8 psychotherpy visits in 14 weeks 6% 3% Empl indictes employment; F/u, follow-up; inpt, inptient; mgmnt, mngement; MH, mentl helth; num, number; PRRC, psychosocil rehb nd recovery center; psy, psychosocil; PTSD, posttrumtic stress disorder; resid, residentil; SD, stndrd devition; SMI, serious mentl illness; SUD, substnce use disorder; sup, supportive; trns l, trnsitionl; VHA, Veterns Helth Administrtion. All mesures clculted within yer unless specified otherwise. N = 6990 ptients. b Trnsitionl work visits nd supportive employment visits re occuptionl therpy tretment modlities. 27,28 Multivrite Anlysis Seprtely, for ech stisfction mesure (the dependent vrible) nd ech progrm chrcteristic (the key independent vrible), we estimted n ordinry lest squres (OLS) model, controlling for ge (in yers), mritl sttus (1 = mrried, 0 = not mrried), sex (1 = femle, 0 = mle), rce (1 = blck, 0 = nonblck), nd comorbidities. 26 We ccounted for heteroscedsticity with robust stndrd errors. In sensitivity nlyses, we lso rn models with clustering, fcility rndom effects, nd logistic regression. These produced similr results, which re not shown. Bsed on the OLS models, for progrm rech, psychosocil service ccess, progrm intensity, nd tretment continuity mesures, e132 MAY

5 Mentl Helth Cre nd Ptient Stisfction TABLE 3. Risk-Adjusted Sttisticlly Significnt Associtions of Progrm Rech Mesures With Stisfction Mesures Performnce Mesure MH ptients: % of MH service-connected veterns in fcility ctchment re who received VHA MH cre Trns l work: % of ptients with SMI who hd trnsitionl work visits Sup empl: % of ptients with SMI who received supportive employment visits Homeless: % of homeless VHA ptients who received ny MH cre PTSD: % of ptients with PTSD who received specilty outptient cre for PTSD SUD spec: % of ptients with SUD who initited intensive SUD tretment in specilty setting Cse mgmnt: % of ptients with SMI who received MH intensive cse mngement for psychosis Got Needed Appointment Access Stisfction b Got Wnted Appointment Got Needed Cre Encounter Stisfction b VHA Rting Clinicin Rting Visit Rting or indicte positive or negtive ssocition, respectively; empl, employment; F/u, follow-up; inpt, inptient; mgmnt, mngement; MH, mentl helth; num, number; PRRC, psychosocil rehb nd recovery center; psy, psychosocil; PTSD, posttrumtic stress disorder; resid, residentil; SMI, serious mentl illness; SUD, substnce use disorder; sup, supportive; trns l, trnsitionl; VHA, Veterns Helth Administrtion. Significnt t the P <.05 level. Results bsed on ordinry lest squres estimtes, djusted for ge, mritl sttus, sex, rce, nd Elixhuser comorbidities 26 (eappendix Tble A3). b Defined in Tble 1. Tbles 3, 4, nd 5 indicte positive, negtive, or not sttisticlly significnt ssocition of progrm chrcteristics with stisfction mesures. (Coefficient estimtes re provided in eappendix Tbles A4-A7.) Progrm Rech Tble 3 shows tht the brod mesure of progrm rech, MH ptients the percentge of veterns service-connected for mentl helth condition who received mentl helth cre nd the percentge of VHA ptients dignosed with psychotic disorders, bipolr, mjor depression, or PTSD who hd trnsitionl work visits ( trns l work ) were positively ssocited with t lest 2 of 3 mesures of ccess stisfction over the prior 12 months. They were lso ech positively ssocited with 2 of 3 mesures of encounter stisfction. None of the other 4 (more nrrow) progrm rech mesures were positively ssocited with ccess or encounter stisfction. All but the percentge of ptients with serious mentl illness (SMI) who received mentl helth intensive cse mngement for psychosis ( cse mgmnt ) hd no sttisticlly significnt ssocition with stisfction. Consistent with prior work, 30 condition-specific mesures my hve been negtively ssocited with stisfction if ptients with those conditions generlly rted stisfction lower (becuse of their condition, not their cre) nd their representtion in our smple ws higher t fcilities tht tret more of them. 16 No mtter the reson, the use of mesures tht were negtively ssocited with stisfction, or not ssocited with it t ll, should be justified nd vlidted on other grounds (eg, they mesured some spect of cliniclly pproprite cre). In cse mgmnt, for exmple, vilbility of this tretment hs been shown to improve the clinicl outcomes of ptients with chronic SMI. 31 Psychosocil Service Access Of ll the ctegories of progrm chrcteristics, those pertining to psychosocil service ccess were lest ssocited with stisfction mesures (Tble 4). This could be becuse inititing psychosocil services is chllenging time for ptients, so fcilities with greter ccess to it lso hve more ptients who exhibited less stisfction. Progrm Intensity With 1 exception (psychosocil rehb nd recovery center), ll progrm intensity mesures were either positively ssocited with stisfction or were not ssocited with ny stisfction mesure (Tble 4). Although the number of mentl helth encounters per unique ptient seen t fcility ( overll ) ws not ssocited with the stisfction of its ptients who received mentl helth cre, the number of encounters per ptient with ny encounters ( MH ptients ) ws positively ssocited with ll 6 stisfction mesures. Becuse the denomintor of the overll mesure ws ll VHA ptients who were seen t fcility, it reflects both the proportion of ptients in the helthcre system who received mentl helth tretment nd the intensity of services felt by those who received cre. A more focused decomposition of the overll mesure into metrics tht ssessed 1) the rech of mentl helth services mong THE AMERICAN JOURNAL OF MANAGED CARE VOL. 23, NO. 5 e133

6 MANAGERIAL TABLE 4. Risk-Adjusted Sttisticlly Significnt Associtions of Psychosocil Service Access nd Progrm Intensity Mesures With Stisfction Mesures Performnce Mesure Psychosocil Service Access Mesures Got Needed Appointment Access Stisfction b Got Wnted Appointment Got Needed Cre Encounter Stisfction b VHA Rting Clinicin Rting SMI: % of ptients with SMI inititing psychosocil tretment or psychotherpy PTSD: % of ptients with PTSD inititing psychotherpy SUD psy: % of ptients with SUD inititing psychosocil tretment or psychotherpy in ny setting + Depression: % of ptients with depression inititing psychotherpy Progrm Intensity Mesures Overll: number of mentl helth encounters per VHA ptient MH ptients: number of MH encounters per ptient with ny MH encounters PRRC: number of encounters t psychosocil rehb nd recovery centers per VHA ptient Homeless: number of MH encounters per homeless vetern with ny MH encounters SUD: weeks of intensive outptient SUD tretment per ptient with ny intensive outptient SUD tretment visits + + Trnsitionl work: number of trnsitionl work visits per ptient with ny trnsitionl work visits Supportive employment: number of supportive employment visits per ptient with ny supportive employment visits + + or indicte positive or negtive ssocition, respectively; empl, employment; F/u, follow-up; inpt, inptient; MH, mentl helth; num, number; PRRC, psychosocil rehb nd recovery center; psy, psychosocil; PTSD, posttrumtic stress disorder; resid, residentil; SMI, serious mentl illness; SUD, substnce use disorder; trns l, trnsitionl; VHA, Veterns Helth Administrtion. Significnt t the P <.05 level. Results bsed on ordinry lest squres estimtes, djusted for ge, mritl sttus, sex, rce, nd Elixhuser comorbidities 26 (eappendix Tble A3). b Defined in Tble 1. Visit Rting ptients requiring them (eg, the MH ptients rech mesure), nd 2) the intensity of services mong mentl helth cre utilizers (eg, the MH ptients intensity mesure) ws esier to interpret. Mesures of intensity of therpeutic nd supportive employment progrm services ( sup empl nd trns l work ) were either not correlted with stisfction or only ssocited with 1 ccess mesure. It is possible tht ptients thought of these principlly s employment progrms nd did not consider their experience with them when responding to helthcre stisfction surveys. As with other mesures of services for ptients with serious mentl illness, the psychosocil rehb nd recovery centers mesure ws negtively ssocited with stisfction. Agin, this my reflect enrichment of the ptient popultion with ptients who tended to rte helthcre services poorly. Tretment Continuity With few exceptions, continuity, vriously mesured, ws positively ssocited with hlf or more of the ccess nd/or encounter stisfction mesures (Tble 5). In the cse of the percentge of outptients who received mentl helth cre without second visit in 6 months ( gp ), the ssocition ws negtive with ll 6 stisfction mesures, which is still consistent with the ide tht less continuity of cre is less stisfying to ptients. Limittions Our nlysis hs few limittions. First, it is observtionl, so we cnnot infer cuslity. Also, our smple is of ptients with recent VHA mentl helth encounter. As such, it is not necessrily representtive of ll VHA enrollees or even ll VHA enrollees with mentl helth dignoses, mny of whom my not hve hd recent mentl helth visit. Third, the SHEP survey response rte is reltively low for ptients younger thn 50 yers, which could threten the generlizbility of findings for tht group. Finlly, the survey instrument ws not specificlly designed to elicit impressions of mentl helth cre only. It is possible they were lso influenced by other spects of VHA cre. e134 MAY

7 Mentl Helth Cre nd Ptient Stisfction TABLE 5. Risk-Adjusted Sttisticlly Significnt Associtions of Tretment Continuity Mesures with Stisfction Mesures Performnce Mesure Num resid: number of outptient visits in 6 months post residentil MH dischrge Num inpt: Number of outptient visits in 6 months post inptient MH dischrge F/u resid: % of ptients with 7-dy follow-up fter residentil dischrge Gp: % of ptients with MH outptient visit in 1 yer who went 6 months without second visit SMI: % of ptients with SMI who hd 8 psychotherpy or psychosocil tretment visits in 14 weeks PTSD: % of ptients with PTSD who hd 8 visits for PTSD psychotherpy in 14 weeks Inptient detox: % of ptients who hd inptient detox with outptient follow-up within 7 dys Outptient detox: % of ptients who hd outptient detox with outptient follow-up within 7 dys SUD: % of ptients with SUD who hd 8 SUD psychotherpy or psychosocil tretments in 14 weeks Depression: % of ptients with depression who hd 8 psychotherpy visits in 14 weeks Got Needed Appointment Access Stisfction b Got Wnted Appointment Got Needed Cre Encounter Stisfction b VHA Clinicin Visit Rting Rting Rting or indicte positive or negtive ssocition, respectively; empl, employment; F/u, follow-up; inpt, inptient; MH, mentl helth; num, number; PRRC, psychosocil rehb nd recovery center; psy, psychosocil; PTSD, posttrumtic stress disorder; resid, residentil; SMI, serious mentl illness; SUD, substnce use disorder; trns l, trnsitionl; VHA, Veterns Helth Administrtion. Significnt t the P <.05 level. Results bsed on ordinry lest squres estimtes, djusted for ge, mritl sttus, sex, rce, nd Elixhuser comorbidities 26 (eappendix Tble A3). b Defined in Tble 1. DISCUSSION In smple of ptients who visited the VHA for mentl helth conditions, we ssessed the reltionship between stisfction nd progrm chrcteristics, spnning multiple domins. Our results provide some importnt lessons for policy mkers nd helthcre mngers. We found tht stisfction with VHA ccess mong ptients with mentl helth conditions ws higher thn stisfction with cre encounters. Brod mesures of the progrm s rech of mentl helth cre tretment (ie, the proportion of ptients served) nd intensity (ie, the number of visits received) tended to be positively ssocited with both ccess nd encounter stisfction. No mesures of ccess to psychosocil services (ie, the proportion of ptients who received psychosocil services regrdless of setting of cre) nd most mesures of tretment continuity (ie, mesures of outptient follow-up fter inptient cre) were positively ssocited with both kinds of stisfction. Also, more nrrow performnce mesures those tht focused on specific dignostic popultions (eg, those with PTSD nd SMI) were less likely to be positively ssocited with stisfction. This is consistent with prior work tht suggests certin types of ptients who receive mentl helth cre re less likely to be stisfied with cre, perhps more of chrcteristic of the ptients thn the tretment progrms tht serve them. 16 Policy mkers nd progrm mngers should be wre tht s they ttempt to increse psychosocil service ccess, they my not see positive reltionship to stisfction. Efforts to ensure initil ccess to psychosocil services to ll ptients who need them my negtively impct the vilbility of ongoing or more intensive services for those who initite, s lrger pool of ptients inititing services would compete for vilble tretment slots. We found tht becuse mesures of ccess to psychosocil services hd the wekest reltionship to stisfction, while tretment continuity hd the most consistent reltionship, further investigtion is needed. If one took stisfction s the only ssessment of helth system performnce, they might conclude tht psychosocil tretment ccess is not vluble. Tht is not the right interprettion. Psychosocil tretment ccess is vluble for other resons. For exmple, the subpopultion tht does not respond dequtely to medictions my rely on this modlity of cre for improvement. On the other hnd, continuity ws most consistently ssocited with greter stisfction. This is in line with growing body of work showing positive outcomes ssocited with continuity, such THE AMERICAN JOURNAL OF MANAGED CARE VOL. 23, NO. 5 e135

8 MANAGERIAL s better qulity of life, community functioning, symptom reduction, 32 incresing Globl Assessment of Functioning scores, 33 nd lower mortlity risk. 34 Boden nd Moos 19 lso showed tht greter enggement with cre is ssocited with higher stisfction. It is likely tht continuity of cre ws ssocited with greter stisfction becuse those who were not stisfied with cre tended to be lost to follow-up. Continuity of cre my directly cuse greter stisfction (becuse ptients wnt it), but the reverse my lso be true: ptients who re stisfied (for other resons) my be more likely to return for subsequent ppointments, incresing mesured continuity of cre. One mesure for which we found no reltionship to stisfction is hrd to justify on nonstisfction-bsed grounds: the progrm intensity overll mesure of the number of mentl helth encounters per unique VHA ptient. Becuse this mesure could go up through reduction in VHA ptients, independent of their mentl helth cre needs, nd becuse there re other vlid mesures of intensity more specific to mentl helth ptients, this did not pper to be mesure of high vlue. Severl mesures in ech domin my cpture the sme or similr spects of cre. For instnce, severl rech mesures were positively correlted nd exhibited similr ptterns of reltionships to stisfction. Such redundncy is useful for mngers, prticulrly in the context of incentivized or prioritized mesure performnce. As hs been observed in other work, 13,27 incentives cn led to loss of fidelity in the dt underlying metrics, 34 s behvior my be modified in direct response to the mesure. If metric tied to incentives strts to devite considerbly from nother mesuring the sme thing in different wy, but it is not tied to incentives, tht is signl tht the integrity of the underlying dt my hve been ffected by the incentives. CONCLUSIONS With few exceptions, this reserch demonstrtes tht the set of mentl helth progrm chrcteristics used by the VHA exhibits the expected ssocitions with ptient stisfction nd should be useful in monitoring ptient-centered spects of cre qulity. n Acknowledgments This work ws funded by Deprtment of Veterns Affirs, Helth Services Reserch nd Development grnt (CRE ), by the Deprtment of Veterns Affirs Progrm Evlution nd Resource Center, Office of Mentl Helth Opertions, nd by Deprtment of Veterns Affirs, Qulity Enhncement Reserch Inititive grnt (PEC ). It ws pproved by the VA Boston Helthcre System Institutionl Review Bord. The views expressed re those of the uthors nd do not necessrily reflect the position or policy of the Deprtment of Veterns Affirs, Boston University, Northestern University, or Hrvrd University. Author Affilitions: Helthcre Finncing & Economics, VA Boston Helthcre System (ABF, SDP), Boston, MA; School of Medicine, Boston University (ABF), Boston, MA; Hrvrd T.H. Chn School of Public Helth, Hrvrd University (ABF), Boston, MA; Progrm Evlution nd Resource Center, VA Office of Mentl Helth Opertions (JT), Plo Alto, CA; Center for Innovtion to Implementtion, VA Plo Alto Helthcre System (JT), Plo Alto, CA; School of Phrmcy nd Deprtment of Economics, Northestern University (SDP), Boston, MA. Source of Funding: This work ws funded by Deprtment of Veterns Affirs, Helth Services Reserch nd Development grnt (CRE ), by the Deprtment of Veterns Affirs Progrm Evlution nd Resource Center, Office of Mentl Helth Opertions, nd by Deprtment of Veterns Affirs, Qulity Enhncement Reserch Inititive grnt (PEC ). Author Disclosures: The uthors report no reltionship or finncil interest with ny entity tht would pose conflict of interest with the subject mtter of this rticle. Authorship Informtion: Concept nd design (ABF, SDP, JT); cquisition of dt (ABF, SDP, JT); nlysis nd interprettion of dt (ABF, SDP, JT); drfting of the mnuscript (ABF, JT); criticl revision of the mnuscript for importnt intellectul content (ABF, SDP, JT); sttisticl nlysis (ABF, SDP); obtining funding (SDP); nd supervision (SDP). Address Correspondence to: Austin B. Frkt, PhD, VA Boston Helthcre System, 150 S Huntington Ave, Boston, MA E-mil: frkt@bu.edu. REFERENCES 1. Blonigen DM, Bui L, Hrris AH, Hepner KA, Kivlhn DR. Perceptions of behviorl helth cre mong veterns with substnce use disorders: results from ntionl evlution of mentl helth services in the Veterns Helth Administrtion. J Subst Abuse Tret. 2014;47(2): doi: /j.jst Zickmund SL, Burkitt KH, Go S, et l. Rcil differences in stisfction with VA helth cre: mixed methods pilot study. J Rcil Ethn Helth Disprities. 2015;2(3): doi: /s Jh AK, Orv EJ, Zheng J, Epstein AM. Ptients perception of hospitl cre in the United Sttes. N Engl J Med. 2008;359(18): doi: /NEJMs Vitl signs: core metrics for helth nd helth cre progress (2015). The Ntionl Acdemies Press website. Accessed April 4, Chtterjee P, Tsi TC, Jh AK. Delivering vlue by focusing on ptient experience. Am J Mng Cre. 2015;21(10): Dvies E, Shller D, Edgmn-Levitn S, et l. Evluting the use of modified CAHPS survey to support improvements in ptient-centred cre: lessons from qulity improvement collbortive. Helth Expect. 2008;11(2): doi: /j x. 7. Tjeu GS, Kzley AS, Menchemi N. Do hospitls tht do the right thing hve more stisfied ptients? Helth Cre Mnge Rev. 2015;40(4): doi: /HMR Boulding W, Glickmn SW, Mnry MP, Schulmn KA, Stelin R. Reltionship between ptient stisfction with inptient cre nd hospitl redmission within 30 dys. Am J Mng Cre. 2011;17(1): Glickmn SW, Boulding W, Mnry M, et l. Ptient stisfction nd its reltionship with clinicl qulity nd inptient mortlity in cute myocrdil infrction. Circ Crdiovsc Qul Outcomes. 2010;3(2): doi: /CIRCOUTCOMES Scks GD, Lwson EH, Dwes, Jcobson NB, Aynin JZ. Reltionship between hospitl performnce on ptient stisfction survey nd surgicl qulity. JAMA Surg. 2015;150(9): doi: /jmsurg Fremont AM, Clery PD, Hrgrves JL, Rowe RM. Ptient-centered processes of cre nd long-term outcomes of myocrdil infrction. J Gen Intern Med. 2001;16(12): Fenton JJ, Jernt AF, Bertkis KD, Frnks P. The cost of stisfction: ntionl study of ptient stisfction, helth cre utiliztion, expenditures, nd mortlity. Arch Intern Med. 2012;12;172(5): doi: / rchinternmed Prentice JC, Frkt AB, Pizer SD. Metrics tht mtter. J Gen Intern Med. 2016;31(suppl 1): doi: / s Chen S, Smith MW, Wgner TH, Brnett PG. Spending for specilized mentl helth tretment in the VA: Helth Aff (Millwood). 2003;22(6): Rosenheck R, Wilson NJ, Meterko M. Influence of ptient nd hospitl fctors on consumer stisfction with inptient mentl helth tretment. Psychitr Serv. 1997;48(12): Burnett-Zeigler I, Zivin K, Ilgen MA, Bohnert AS. Perceptions of qulity of helth cre mong veterns with psychitric disorders. Psychitr Serv. 2011;62(9): doi: /ppi.ps Hepner KA, Pddock SM, Wtkins KE, Solomon J, Blonigen DM, Pincus HA. Veterns perceptions of behviorl helth cre in the Veterns Helth Administrtion: ntionl survey. Psychitr Serv. 2014;65(8): doi: /ppi.ps Holcomb WR, Prker JC, Leong GB, Thiele J, Higdon J. Customer stisfction nd self-reported tretment outcomes mong psychitric inptients. Psychitr Serv. 1998;49(7): Boden MT, Moos R. Predictors of substnce use disorder tretment outcomes mong ptients with psychotic disorders. Schizophr Res. 2013;146(1-3): doi: /j.schres Hoff RA, Rosenheck RA, Meterko M, Wilson NJ. Mentl illness s predictor of stisfction with inptient cre t Veterns Affirs hospitls. Psychitr Serv. 1999;50(5): Burgess DJ, Grvely AA, Nelson DB, et l. A ntionl study of rcil differences in pin screening rtes in the VA helth cre system. Clin J Pin. 2013;29(2): doi: /AJP.0b013e e. 22. Trfton JA, Greenberg G, Hrris AH, et l. VHA mentl helth informtion system: pplying helth informtion technology to monitor nd fcilitte implementtion of VHA Uniform Mentl Helth Services Hndbook requirements. Med Cre. 2013;51(3, suppl 1):S29-S36. doi: /MLR.0b013e31827d Uniform mentl helth services in VA medicl centers nd clinics. VHA hndbook Deprtment of Veterns Affirs website. Published September 11, Updted November 16, Accessed April 4, e136 MAY

9 Mentl Helth Cre nd Ptient Stisfction 24. Frkt AB, Trfton J, Pizer SD. Mintennce of ccess s demnd for substnce use disorder tretment grows. J Subst Abuse Tret. 2015;55: doi: /j.jst Strtegic Anlytics for Improvement nd Lerning (SAIL) fct sheet. Veterns Affirs website. blogs.v.gov/vantge/wp-content/uplods/2014/11/sailfctsheet.pdf. Published November Accessed April 4, Elixhuser A, Steiner C, Hrris DR, Coffey RM. Comorbidity mesures for use with dministrtive dt. Med Cre. 1998;36(1): Prentice JC, Dvies ML, Pizer SD. Which outptient wit-time mesures re relted to ptient stisfction? Am J Med Qul. 2014;29(3): doi: / Penk W, Drebing CE, Rosenheck RA, Krebs C, Vn Ormer A, Mueller L. Veterns Helth Administrtion Trnsitionl work experience vs. job plcement in veterns with co-morbid substnce use nd non-psychotic psychitric disorders. Psychitr Rehbil J. 2010:33(4): doi: / Kinoshit Y, Furukw TA, Kinoshit K, et l. Supported employment for dults with severe mentl illness. Cochrne Dtbse Syst Rev. 2013;13;(9):CD doi: / CD pub Fiorentini G, Rgzzi G, Robone S. Are bd helth nd pin mking us grumpy? n empiricl evlution of reporting heterogeneity in rting helth system responsiveness. Soc Sci Med. 2015;144: doi: /j. socscimed Dieterich M, Irving CB, Prk B, Mrshll M. Intensive cse mngement for severe mentl illness. Cochrne Dtbse Syst Rev. 2010;6(10):CD doi: / CD pub Adir CE, McDougll GM, Mitton CR, et l. Continuity of cre nd helth outcomes mong persons with severe mentl illness. Psychitr Serv. 2005;56(9): Greenberg GA, Rosenheck RA. Specil section on the GAF: continuity of cre nd clinicl outcomes in ntionl helth system. Psychitr Serv. 2005;56(4): doi: /ppi.ps Hrris AH, Gupt S, Bowe T, et l. Predictive vlidity of two process-of-cre qulity mesures for residentil substnce use disorder tretment. Addict Sci Clin Prct. 2015;10:22. doi: /s Full text nd PDF t THE AMERICAN JOURNAL OF MANAGED CARE VOL. 23, NO. 5 e137

10 eappendix Tble A1. SHEP Response Rtes Ptient Chrcteristic Response Rte All ptients 43.6% Mle 44.7% Under ge 50 yers 13.4% Age yers 38.4% Age yers 55.2% Age 80 or more yers 61.0% SHEP indictes Survey of Helthcre Experiences of Ptients

11 Tble A2. SHEP Skip Pttern nd Item Nonresponse Vrible Question nd Skip Pttern N (of 6990) Access stisfction Got needed pp t Got wnted pp t Got needed cre Q: In the lst 12 months, when you needed cre right wy, how often did you get cre s soon s you thought you needed? Skipped if nswered no to prior question bout whether needed cre right wy. Q: In the pst 12 months, not counting the times you needed cre right wy, how often did you get n ppointment s soon s you thought you needed? Skipped if nswered no to prior question bout whether mde ny ppointments excluding those for urgent cre. Q: In the pst 12 months, how often ws it esy to get the cre, tests, or tretment you thought you needed through the VA? Skipped if nswered no to prior question bout whether tried to get ny VA cre nswered of 3848 eligible (92%) becuse 3142 nswered no to the indicted prior question 5333 nswered of 5557 eligible (96%) becuse 1433 nswered no to the indicted prior question 6070 nswered of 6215 eligible (98%) becuse 775 nswered no to the indicted prior question Encounter stisfction VHA rting Q: Using ny number from 0 to 10, where 0 is the worst helthcre possible nd 10 is the best helthcre possible, wht number would you use to rte ll your VHA helthcre in the lst 12 months? 6917 nswered of 6990 eligible (99%)

12 Clinicin rting No skip pttern Q: Using ny number from 0 to 10, where 0 is the worst personl doctor/nurse possible nd 10 is the best personl doctor/nurse possible, wht number would you use to rte your personl VHA doctor/nurse? 6289 nswered of 6425 eligible (98%) becuse 565 indicted they didn t hve personl doctor/nurse Visit rting Skipped if don t hve personl VA doctor/nurse Q: All things considered, how stisfied were you with the VHA during your recent visit? No skip pttern 6990 nswered of 6990 eligible (100%) SHEP indictes Survey of Helthcre Experiences of Ptients; VA, Deprtment of Veterns Affirs; VHA, Veterns Helth Administrtion

13 Tble A3. Mens for Elixhuser Comorbidities 26 (N = 6990 ptients) Vrible Description Men CHF Congestive hert filure 0.04 crd rr Crdic rrhythmis 0.08 vlve disese Vlvulr disese 0.02 pulm circ Pulmonry circultion disorder 0.00 periph vsc Peripherl vsculr disorder 0.05 Hypertension Hypertension 0.62 Prlysis Prlysis 0.01 oth neuro Other neurologicl disorder 0.04 chron pulm Chronic pulmonry disese 0.18 DM, uncomp Dibetes, uncomplicted 0.30 DM, comp Dibetes, complicted 0.08 hypothy Hypothyroidism 0.08 Renl filure Renl filure 0.05 Liver disese Liver disese 0.04 Peptic ulcer Peptic ulcer disese 0.01 AIDS AIDS 0.00 Lymphom Lymphom 0.01 metst cnc Metsttic cncer 0.00 Tumor Solid tumor without metstsis 0.11 Arthritis Rheumtoid rthritis/collgen vsculr disese 0.02 cog Cogulopthy 0.02 Obesity Obesity 0.24 Weight loss Weight loss 0.00 Fluid Fluid nd electrolyte disorders 0.03 Blood loss Blood loss nemi 0.00

14 Deficiency Deficiency nemi 0.08 Alcohol Alcohol use disorder 0.17 Drug Drug use disorder 0.09 Psychoses Psychoses 0.37 Depression Depression 0.48

15 Tble A4. Risk-Adjusted OLS Coefficients for Progrm Rech Mesures Models Access stisfction Encounter stisfction Performnce (defined in Tble 1) (defined in Tble 1) mesure (defined in Tble 2) got needed pp t got wnted pp t got needed cre # positive # negtive VHA rting clinicin rting visit rting # positive # negtive MH ptients trns l work sup empl homeless PTSD SUD spec cse mgmnt Results bsed on OLS estimtes, djusted for ge, mritl sttus, sex, rce, nd Elixhuser comorbidities 26 (Appendix Tble A3) Significnt t the P <.05 level OLS indictes ordinry lest squres; VHA, Veterns Helth Administrtion; MH, mentl helth; PTSD, post-trumtic stress disorder; SUD, substnce use disorder

16 Tble A5. Risk-Adjusted OLS Coefficients for Psychosocil Service Access Mesures Models Access stisfction Encounter stisfction Performnce (defined in Tble 1) (defined in Tble 1) mesure (defined in Tble 2) got needed pp t got wnted pp t got needed cre # positive # negtive VHA rting clinicin rting visit rting # positive # negtive SMI PTSD SUD psy depression Results bsed on OLS estimtes, djusted for ge, mritl sttus, sex, rce, nd Elixhuser comorbidities 26 (Appendix Tble A3) Significnt t the P <.05 level 1 0 OLS indictes ordinry lest squres; PTSD, post-trumtic stress disorder; SMI, serious mentl illness; SUD, substnce use disorder; VHA, Veterns Helth Administrtion

17 Tble A6. Risk-Adjusted OLS Coefficients for Progrm Intensity Mesures Models Access stisfction Encounter stisfction Performnce (defined in Tble 1) (defined in Tble 1) mesure (defined in Tble 2) got needed pp t got wnted pp t got needed cre # positive # negtive VHA rting clinicin rting visit rting # positive # negtive Overll MH ptients PRRC Homeless SUD trns l work sup empl Results bsed on OLS estimtes, djusted for ge, mritl sttus, sex, rce, nd Elixhuser comorbidities 26 (Appendix Tble A1) Significnt t the P <.05 level OLS indictes ordinry lest squres; PTSD, post-trumtic stress disorder; SMI, serious mentl illness; SUD, substnce use disorder; VHA, Veterns Helth Administrtion

18 Tble A7. Risk-Adjusted OLS Coefficients for Tretment Continuity Mesures Models Access stisfction Encounter stisfction Performnce (defined in Tble 1) (defined in Tble 1) mesure (defined in Tble 2) got needed pp t got wnted pp t got needed cre # positive # negtive VHA rting clinicin rting visit rting # positive # negtive num resid num inpt f/u resid Gp SMI PTSD inpt detox outpt detox SUD Depression Results bsed on OLS estimtes, djusted for ge, mritl sttus, sex, rce, nd Elixhuser comorbidities 26 (Appendix Tble A1) Significnt t the P <.05 level

19 OLS indictes ordinry lest squres; PTSD, post-trumtic stress disorder; SMI, serious mentl illness; SUD, substnce use disorder; VHA, Veterns Helth Administrtion