VETERANS RAND 36 ITEM HEALTH SURVEY (VR-36)

Transcription

1 VETERANS RAND 36 ITEM HEALTH SURVEY (VR-36) Please do this: Instructions: This survey asks for your views about your health. This information will help keep track of how you feel and how well you are able to do your usual activities. Please answer every question by filling in one circle on each line. If you are unsure about how to answer a question, please give the best answer you can. 1. In general, would you say your health is: EXCELLENT VERY GOOD GOOD FAIR POOR 2. The following questions are about activities you might do during a typical day. Does your health now limit you in these activities? If so, how much? YES, LIMITED A LOT YES, LIMITED A LITTLE NO, NOT LIMITED AT ALL a. Vigorous activities, such as running, lifting heavy objects, participating in strenuous sports? b. Moderate activities, such as moving a table, pushing a vacuum cleaner, bowling, or playing golf? c. Lifting or carrying groceries? d. Climbing several flights of stairs? e. Climbing one flight of stairs? f. Bending, kneeling, or stooping? g. Walking more than a mile? h. Walking several blocks? i. Walking one block? j. Bathing or dressing yourself? 3. During the past 4 weeks, have you had any of the following problems with your work or other regular daily activities as a result of your physical health? NO, NONE YES, A LITTLE YES, SOME YES, MOST YES, ALL a. Cut down the amount of time you spent on work or other activities. b. Accomplished less than you would like c. Were limited in the kind of work or other activities. d. Had difficulty performing the work or other activities (for example, it took extra effort).

2 4. During the past 4 weeks, have you had any of the following problems with your work or other daily activities as a result of any emotional problems (such as feeling depressed or anxious)? NO, NONE YES, A LITTLE YES, SOME YES, MOST YES, ALL a. Cut down the amount of time you spent on work or other activities. b. Accomplished less than you would like. c. Didn t do work or other activities as carefully as usual. 5. During the past 4 weeks, to what extent has your physical health or emotional problems interfered with your normal social activities with family, friends, neighbors, or groups? NOT AT ALL SLIGHTLY MODERATELY QUITE A BIT EXTREMELY 6. How much bodily pain have you had during the past 4 weeks? NONE VERY MILD MILD MODERATE SEVERE VERY SEVERE 7. During the past 4 weeks, how much did pain interfere with your normal work (including both work outside the home and house work)? NOT AT ALL A LITTLE BIT MODERATELY QUITE A BIT EXTREMELY 8. These questions are about how you feel and how things have been with you during the past 4 weeks. For each question, please give the one answer that comes closest to the way you have been feeling. How much of the time during the past 4 weeks: ALL MOST A GOOD BIT OF THE SOME OF THE A LITTLE NONE a. Did you feel full of pep? b. Have you been a very nervous person? c. Have you felt so down in the dumps that nothing could cheer you up? d. Have you felt calm and peaceful? e. Did you have a lot of energy? PLEASE CONTINUE

3 8. Continued from page 4 How much of the time during the past four weeks: ALL MOST A GOOD BIT OF THE SOME A LITTLE NONE f. Have you felt downhearted and blue? g. Did you feel worn out? h. Have you been a happy person? i. Did you feel tired? 9. During the past 4 weeks, how much of the time has your physical health or emotional problems interfered with your social activities (like visiting with friends, relatives, etc.)? ALL OF THE MOST OF THE SOME OF THE A LITTLE OF THE NONE OF THE 10. Please choose the answer that best describes how true or false each of the following statements is for you. DEFINITELY TRUE MOSTLY TRUE NOT SURE MOSTLY FALSE DEFINITELY FALSE a. I seem to get sick a lot easier than other people. b. I am as healthy as anybody I know. c. I expect my health to get worse. d. My health is excellent. Now we d like to ask you some questions about how your health may have changed. 11. Compared to one year ago, how would you rate your physical health in general now? MUCH BETTER SOMEWHAT BETTER ABOUT THE SAME SOMEWHAT WORSE MUCH WORSE 12. Compared to one year ago, how would you rate your emotional problems (such as feeling anxious, depressed or irritable) now? MUCH BETTER SOMEWHAT BETTER ABOUT THE SAME SOMEWHAT WORSE MUCH WORSE

4 HOW TO SCORE THE VR- 36 QUESTIONNAIRE STEP1: SCORING QUESTIONS: QUESTION NUMBER ORIGINAL RESPONSE RESPONSE ASSIGNED SCORE VALUE 1, 11, 5, 7, 10b, 10d a, 2b, 2c, 2d, 2e, 2f, 2g, 2h, 2i, 2j a, 3b, 3c, 3d, 4a, 4b, 4c , 8a, 8d, 8e, 8h b, 8c, 8f, 8g, 8i , 10a, 10c

5 HOW TO SCORE THE VR- 36 QUESTIONNAIRE Step 2: Averaging Items to Form Scales for VR-36 Scale Number of items After scoring/recoding as per Step 1, average the following items Physical functioning 10 2a, 2b, 2c, 2d, 2e, 2f, 2g, 2h, 2i, 2j Role limitations due to physical health 4 3a, 3b, 3c, 3d Role limitations due to emotional problems 3 4a, 4b, 4c Energy/fatigue 4 8a, 8e, 8g, 8i Emotional well-being 5 8b, 8c, 8d, 8f, 8h Social functioning 2 5, 9 Pain 2 6, 7 General health 5 1, 10a, 10b, 10c, 10d

6 January 2000 Volume 5 Issue 1 A Publication for Members of Medical Outcomes Trust MEDICAL OUTCOMES TRUST The Veterans SF-36 Health Status Questionnaire: Development and Application in the Veterans Health Administration Lewis E. Kazis, Sc.D. Dr. Kazis is Director of the Veterans SF-36 Project for the Office of Quality and Performance for the Veterans Administration, Washington, D.C. He is the Chief of Health Outcomes for the Center for Health Quality, Outcomes and Economic Research, a Health Services Research and Development Field Program, Veterans Administration Medical Center, Bedford, Massachusetts. He is also Associate Professor of Health Services at the Boston University School of Public Health. MONITOR Contents Feature 1 Research Summary 3 Initiatives 5 Clinical Practice Applications 7 Research & Policy 9 Health System Improvement 11 Most policy makers concede that reducing costs, maintaining quality of care and ensuring optimal patient outcomes are key priorities for restructuring the healthcare system in the United States. In this rapidly changing environment, accountability for patient care outcomes is a key priority. To meet these challenges of increased competition and accountability, patient-based measures of health are increasingly being adopted by healthcare systems as measures of outcomes. These measures define, from the patients perspectives, what is important in terms of both physical and psychological functioning. Patient-based measures are comprehensive assessments of functional status (e.g., physical, psychological, social and role functioning) and overall perceptions of health. They provide accurate and valid summaries of health status that synthesize domains of morbidity, disability, and disease burden. They also give an important summary description of the complex mix of patient characteristics. The Veterans Health Administration (VHA), is the nation s largest integrated healthcare system with approximately 4 million enrollees in the VA system and about five percent of the total market share for hospital services in the nation. 1 It serves a select group of veterans with complex healthcare needs. Compared to veterans nationally and the general US population, users of the VA healthcare system not only have disabilities that are connected to their military service, they are also older, poorer, less educated, and sicker with more disease. 2, 3 Lessons learned from the VA can provide a model for monitoring the outcomes of care in other managed care systems, especially those serving older populations. Like other healthcare systems that are seeking to adapt to a changing healthcare environment, the VA needs information about the health of current and prospective patients to anticipate patient demand, design effective services, and evaluate the outcomes of its care. In a memorandum in March 1997, the Under Secretary for Health, Dr. Kenneth W. Kizer stated: The Veterans Health Administration s performance measurement system was designed to ensure the delivery of excellent healthcare value as originally defined by four domains of value: access, cost/price, technical quality, and customer satisfaction. VHA has decided to include functional status as the fifth domain of value, given its increasing importance in clinical medicine. 4 (sic) In an environment of healthcare restructuring the VHA is aggressively moving towards implementing an outcomes management system. This system will capitalize on well established measures that have been developed both inside and outside the VHA. Measures such as the Veterans SF-36 are well-validated assessments of functional status that can be used to characterize the case-mix of patient populations. These measures can also be used to monitor the process and outcomes of care at the program or system levels. When administered to a patient population at a point in time, the Veterans SF-36 provides an indication of the case-mix, or disease complexity of a patient group. 5 Such measures have a direct bearing on patient needs, workload, and costs of care. Through the Office of Quality and Performance in the VHA, systems using the Veterans SF-36 are currently being developed. The Veterans Health Study The Veterans SF-36 and scoring algorithms are available on request from the author. Historically, the information regarding ambulatory care that is available to VA providers,

7 administrators, and policy makers has been quite limited. The Veterans Health Study (VHS), a service directed project that was launched in 1992, has as its primary goal the development and application of methodology for assessing healthrelated quality of life (HRQoL) within the VHA. These assessments are primarily structured questionnaires administered to the patient and are reliable and valid in ambulatory populations. The VHS implements and demonstrates the validity of using patient-derived measures of health, since this information reflects the healthcare needs of veterans that can be used for monitoring the outcomes of their care. The VHS was designed to develop, test, and disseminate information systems for monitoring patient-derived outcomes of ambulatory care within the VA. The VHS has as its principal goal to generate measures of functional status that are tailored to the veterans needs and represent the complex health characteristics of its patient population. These health characteristics of veterans, or patient-mix, include the sociodemographics and mix of diseases for an ambulatory patient population that uses the VA system of care. The conceptual framework and methods of the VHS are described elsewhere. 6-7 We characterize the health status of patients using multidimensional assessments of HRQoL. In doing this, we build on the work of previous studies, most notably the Rand Health Insurance Experiment and the Medical Outcomes Study (MOS). 8-9 These studies have demonstrated the value and utility of patient-based measures of HRQoL. The VHS has gone beyond this work and has developed a core set of general measures of health status and disease as part of a comprehensive set of assessments, which can be used by the VA and other healthcare systems for research and patient management purposes. Unlike the MOS where administrative data was not available, the VHS has capitalized on the rich secondary administrative databases available in the VA; these include diagnoses, data on utilization of services and procedures, on an inpatient and outpatient basis. Patient-derived measures of health have been merged with administrative data at the individual patient level. The access to these databases has provided a unique and powerful opportunity to examine patient outcomes in the context of casemix measures obtained that are external to the patient-derived assessments. The VHS is a four-year prospective observational study of health outcomes in patients receiving ambulatory care from the VHA. A panel of 2425 patients have been followed annually for clinical and HRQoL assessments. The details of the VHA design and sampling procedures are described elsewhere. 5 Both general and disease-specific HRQoL assessments have been validated in this study. This methodology is particularly germane to VA healthcare, which serves patients with a complex profile of conditions, sociodemographics, and military history that is strongly related to their eligibility for VA care. The VHS has published 15 articles in peer-reviewed journals with general and disease-specific assessments for the Veterans SF-36, diabetes, osteoarthritis, chronic low back pain, chronic lung disease and alcohol related problems (references and scoring algorithms available on request). In addition, a comorbidity index has also been validated as a case-mix adjuster for measures of health status and healthcare utilization. One of the cornerstones in the VHS is the development of the Veterans SF- 36, a short form health status assessment designed specifically for use among veterans. The Veterans SF-36, adapted from the MOS SF-36, spans the range of health concepts from physical to psychological status. The Veterans SF-36 provides a means to measure patient outcomes of care. The Veterans SF-36 The Veterans SF-36 is our primary measure of health-related quality of life. This assessment has been documented as reliable and valid in ambulatory VA patient populations. 7,12 This assessment builds on the MOS SF-36 and has been adopted by the VHA as one of the performance Veterans Health measures of functional status. Modifications to the MOS SF-36 include changes to the role items (role limitations due to physical and emotional problems), where response choices that were originally dichotomized yes/no choices are now five-point ordinal choices ( no, none of the time to yes, all of the time ). These changes to the SF-36 increased the precision and discriminant validity of the role scales and physical and mental component summaries. The changes to the role scales of the SF-36 demonstrated increases to the precision of the scales by more than 100 percent for the role-physical and 80 percent for the role-emotional. The changes to these two scales also lowered the floor and raised the ceiling of the metric as reflected in each of the scales distributions. In addition, the modified scales when used in calculating the physical and mental component summary scales have improved precision by five percent (see summary scale description below). The Veterans SF-36 has been widely used, disseminated, and documented in the VA. 13 Like the MOS version of the SF-36, the Veterans SF-36 measures eight concepts of health: physical functioning (PF), role limitations due to physical problems (RP), bodily pain (BP), general health perceptions (GH), energy/vitality (VT), social functioning (SF), role limitations due to emotional problems (RE), and mental health (MH). A measure of reliability, or the internal consistency, of items measuring a concept about health is given by Cronbach s alpha. This statistic ranges from 0.00 to 1.00, where higher values indicate greater reliability. Scales or concepts with values of 0.70 or higher are considered acceptable. Results for the Veterans SF-36 scales give Cronbach s alphas ranging from 0.93 for physical functioning to 0.78 for social functioning. Items from each concept are summed and rescaled with a standard range from 0 to 100, where 100 denotes the best health. These eight concepts have also been summarized into two summary scores: a physical component summary (PCS) and a mental component summary (MCS). The summary scales are based upon distinct physical and mental constructs well documented Monitor, January 2000, volume 5, issue 1 2 Feature continues on page 13

8 Veterans Health Research Summary: These abstracts were located after conducting a Medline search for veterans and functional health status or quality of life. Welsh CH, Thompson K, Long-Krug S. Evaluation of patient-perceived health status using the Medical Outcomes Survey Short- Form 36 in an intensive care unit population. Crit Care Med 1999 Aug;27(8): OBJECTIVE: Baseline patient functional status as assessed by providers is correlated with mortality after intensive care unit (ICU) admission. We wanted to see if patient self-perception of health status before admission to an ICU correlated with functional outcome. DESIGN: Prospective survey on a convenience sample. SETTING: Single urban university-affiliated Veterans Affairs Medial Center. PATIENTS: One hundred ninety-nine patients in surgical and medical/coronary ICUs. INTERVEN- TIONS: None. MEASUREMENTS: Patientassessed baseline health status was monitored with the Medical Outcome Survey Short-Form 36 (SF-36), consisting of 36 questions that evaluate eight health status concepts. In addition, baseline functional status (Zubrod scale) was determined and severity of illness (Acute Physiology and Chronic Health Evaluation [APACHE] II) data were collected. Zubrod functional status, which includes mortality, was determined 6 wks and 6 months after ICU admission, and correlation coefficients were calculated. MAIN RESULTS: We found it feasible to collect SF-36 health status data on a 9% sample in this setting. Less than 1% of responses were completed by proxy. The SF-36 data were internally consistent, and several of its scales including general health perception and physical functioning correlated with patient Zubrod functional status (r2 =.08, p <.001; r2 =.14, p <.001) at 6 wks as did vitality (r2 =.04, p <.01), social function (r2 =.03, p <.05), and physical role function (r2 =.02, p =.053), although to a lesser extent. Similar correlations were also found with 6-month functional status. CON- CLUSIONS: We conclude that use of the SF-36 is time efficient in an ICU setting and correlates with 6-wk and 6-month functional outcome. It correlates as well with functional outcome as either the baseline Zubrod functional status or the APACHE II severity of illness measurement. The five-question general health evaluation portion correlated almost as well with outcome as the more extensive 36-item questionnaire. Use of the SF-36 may define patient populations for comparison across hospitals. It may also target individuals with needs for additional posthospitalization care, including rehabilitation services or nursing home placement. Skinner KM, Furey J. The focus on women veterans who use Veterans Administration health care: the Veterans Administration Women s Health Project. Mil Med 1998 Nov;163(11):761-6 Women have served in every war and conflict in our history, but the effects of military duty on women are largely unknown. This article discusses the history of women in the military and presents findings from the Veterans Administration Women s Health Project. This study was designed to describe the health-related quality of life of women veterans who receive Veterans Administration ambulatory care. We assessed health status using the Short Form 36, an easily administered 36-item health survey that has been widely used and that has been shown to be reliable and valid. We compared Short Form 36 scores between veteran and nonveteran women. The results indicate that veteran women score lower on every scale compared with nonveteran women. Unwin C, Blatchley N, Coker W, Ferry S, Hotopf M, Hull L, Ismail K, Palmer I, David A, Wessely S. Health of UK servicemen who served in Persian Gulf War. Lancet 1999 Jan 16;353(9148): BACKGROUND: Various symptoms in military personnel in the Persian Gulf War have caused international speculation and concern. We investigated UK servicemen. METH- ODS: We did a cross-sectional postal survey on a random sample of Gulf War veterans (Gulf War cohort, n=4248) and, stratified for age and rank, servicemen deployed to the Bosnia conflict (Bosnia cohort, n=4250) and those serving during the Gulf War but not deployed there (Era cohort, n=4246). We asked about deployment, exposures, symptoms, and illnesses. We analysed men only. Our outcome measures were physical Research Summary

9 health, functional capacity (SF-36), the general health questionnaire, the Centers for Disease Control and Prevention (CDC) multisymptom criteria for Gulf War illness, and post-traumatic stress reactions. FINDINGS: There were 8195 (65.1%) valid responses. The Gulf War cohort reported symptoms and disorders significantly more frequently than those in the Bosnia and Era cohorts, which were similar. Perception of physical health and ability were significantly worse in the Gulf War cohort than in the other cohorts, even after adjustment for confounders. Gulf War veterans were more likely than the Bosnia cohort to have substantial fatigue (odds ratio 2.2 [95% CI ]), symptoms of posttraumatic stress (2.6 [ ]), and psychological distress (1.6 [ ]), and were nearly twice as likely to reach the CDC case definition (2.5 [ ]). In the Gulf War, Bosnia, and Era cohorts, respectively, 61.9%, 36.8%, and 36.4% met the CDC criteria, which fell to 25.3%, 11.8%, and 12.2% for severe symptoms. Potentially harmful exposures were reported most frequently by the Gulf War cohort. All exposures showed associations with all of the outcome measures in the three cohorts. Exposures specific to the Gulf were associated with all outcomes. Vaccination against biological warfare and multiple routine vaccinations were associated with the CDC multisymptom syndrome in the Gulf War cohort. INTERPRETA- TION: Service in the Gulf War was associated with various health problems over and above those associated with deployment to an unfamiliar hostile environment. Since associations of ill health with adverse events and exposures were found in all cohorts, however, they may not be unique and causally implicated in Gulf-War-related illness. A specific mechanism may link vaccination against biological warfare agents and later ill health, but the risks of illness must be considered against the protection of servicemen. Roche VM, Kramer A, Hester E, Welsh CH. Long-term functional outcome after intensive care. J Am Geriatr Soc 1999 Jan;47(1):18-24 Veterans Health OBJECTIVE: Although age-related mortality after intensive care unit (ICU) admission has been studied, functional recovery for different age groups following ICU admission is not well characterized. We hypothesized that compared with younger age groups, fewer patients older than age 65 admitted to an ICU would regain their full prehospitalization functional ability and that their recovery would be slower than that of younger patients. DESIGN: A prospective observational cohort study with convenience sampling. SETTING: Intensive care units of an urban university-affiliated Veterans Administration Medical Center. PARTICIPANTS: A total of 222 patients during the first 72 hours after entry to a medical or surgical ICU at the Denver Veteran s Administration Medical Center between September 1991 and July MEA- SUREMENTS: We collected baseline data on patient demographics and on the severity of acute illness using the Acute Physiology and Chronic Health Evaluation (APACHE II), Acute Physiology Score (APS), and functional status (highest level of physical activity level 1 month before admission). We recorded survival and patient-perceived global functional status at 6 weeks and 6 months after admission. Post-ICU function was adjusted for baseline function, age, APACHE II, and APS using multiple regression. RESULTS: Average patient age was 62+/-.74 years (mean +/- SEM). Fifty-two percent of the entire cohort returned to baseline function at 6 months. Although baseline function was better for younger people, there was no difference in recovery at 6 weeks in older compared with younger patients. Most functional recovery occurred by 6 weeks, with maintenance of this recovery at 6 months. Baseline function was the major determinant of both 6 week recovery (P <.001) and 6 month recovery (P =.002), whereas APACHE II was not (P =.3). Age predicted recovery significantly (P =.04) at 6 months but not at 6 weeks (P =.26). APACHE II (P <.001) and baseline function (P =.03) predicted mortality. CONCLUSIONS: Older people had worse functional ability at ICU admission, but the proportion of older people who recovered and their rate of recovery was the same as for younger people. Baseline functional status, rather than abnormal physiologic status (as measured by APACHE II) on admission, was the major determinant of recovery, whereas APACHE II was the main correlate of mortality. Together, baseline function and physiologic status provide valuable complementary information for clinically relevant outcomes following an ICU admission. Taft CT, Stern AS, King LA, King DW. Modeling physical health and functional health status: the role of combat exposure, posttraumatic stress disorder, and personal resource attributes. J Trauma Stress 1999 Jan;12(1):3-23. This study examined associations of combat exposure and posttraumatic stress disorder (PTSD) with physical health conditions and also incorporated hardiness and social support as mediators and functional health status as an outcome. Data were derived from 1,632 male and female Vietnam veterans who participated in the National Vietnam Veterans Readjustment Study. Path analysis revealed that hardiness and social support operated primarily as intermediary variables between combat exposure and PTSD, and PTSD emerged as the pivotal variable explaining physical health conditions and functional health status. Gender-based differences in means and patterns of associations among variables were found. The results stress the importance of assessing trauma in clinical settings as a meaningful determinant of health outcomes. Kunik ME, Benton CL, Snow-Turek AL, Molinari V, Orengo CA, Workman R. The contribution of cognitive impairment, medical burden, and psychopathology to the functional status of geriatric psychiatric inpatients. Gen Hosp Psychiatry 1998 May;20(3):183-8 In order to define the contributions of cognitive impairment, medical burden, and psychopathology to the functional Monitor, January 2000, volume 5, issue 1 4 Research Summary continues on page 15

10 Veterans Health Initiatives Summaries of Efforts to Evaluate Outcomes of Veterans Health These citations were selected from VA HSR&D Seventeenth Annual Meeting February 24-26, 1999 Abstracts at Do Oral Health-Related Quality of Life Measures Relate to Use of Dental Care? Contact: Judith Jones, DDS, MPH, Nancy Kressin, PhD, A Spiro III, PhD, Donald Miller, ScD and Lewis Kazis, ScD. Bedford VA Medical Center, Bedford, MA. RI Garcia. whom use routine dental care. Impact: Self-reported oral health measures may be useful to monitor the effects of dental care on patients quality of life in users of VA health care. Is Depression Associated with Oral Health- Related Quality of Life? Objectives: Valid dental outcome measures should vary with the use of dental services. The purpose of this analysis is to examine the relationship of oral health-related quality of life measures to past use of dental care in two populations. Methods: We examined the retrospective relationships of self-reported oral health measures to selfreported use of care in two contrasting samples of veterans, the Veterans Health Study (VHS, N=538, mean age=62) and the VA Dental Longitudinal Study (DLS, N=278, mean age =71). Self-reported oral health measures included a single-item self-report measure of oral health (OH1), the 3-item Oral Health-Related Quality of Life measure (OHQOL, Kressin, et al, 1996) the 12- item Geriatric Oral Health Assessment (GOHAI, Atchison & Dolan, 1990), and the 49-item Oral Health Impact Profile (OHIP, Slade & Spencer, 1994). Use of care was categorized into <=1year, >1year; and <=2years, >2years. Reason for last visit was divided into emergency and routine care (exam and cleaning, fillings, other). Results: In the VHS sample, better oral health (OH1) was associated with recency of dental visit, i.e., better oral health was associated (p<0.05) with more recent use (in last year and last 2 years). Better scores on the OH1, OHQOL and OHIP were significantly associated with reason for last visit, with approximately 0.5 sdev lower scores, on average, in persons who used emergency as compared to routine care. In the DLS sample, there were no significant differences in mean self-reported oral health scores by recency of use or reason for last visit; however trends were in the expected directions. Conclusions: The validity of these self-report measures of oral health is suggested by the association with recency of dental care and reason for last visit in VA health care users. However, no significant associations were observed in the DLS, most of Contact: Nancy Kressin, PhD, Avron Spiro III, PhD, Katherine Skinner, PhD and Judith Jones, DDS, MPH. Bedford VA Medical Center, Bedford, MA. Objectives: The health-related quality of life (functional status, emotional well-being) of patients with depression is often as low as, or lower than, that of patients with chronic medical conditions. However, we do not know whether depression has a similar effect on oral health- related quality of life (oral QOL). VA dental policymakers, clinicians and researchers are increasingly relying on oral QOL ratings to evaluate dental treatment needs and outcomes of care. Thus, it is important to understand what factors influence such ratings. Methods: We examined the association between depression (measured by the CES-D) and oral QOL, using two different indices: the Geriatric Oral Health Assessment Index (GOHAI) and the Oral Health-Related Quality of Life measure (OHQOL). Using data from 3 veteran samples: male VA patients in the Veterans Health Study (VHS), female VA patients in the VA Women s Health Project (WHP), and male community dwelling veterans who do not use VA care (Normative Aging Study (NAS)), we examined whether individuals who screened positive for depression (scoring above the standard cutpoint) had worse oral QOL than those who were not, controlling for sociodemographics (age, education, marital status), and self-reported oral health. Results: In bivariate analyses, being depressed was associated with worse OHQOL scores in both the VHS and WHP veteran patient samples, as well as in the NAS. Depressed individuals had worse GOHAI scores in the VHS and WHP, but not in the NAS. After controlling for self-reported oral health, age, income, marital status and education, depression remained significantly associated with Initiatives

11 Veterans Health both oral quality of life measures in all samples, and the independent and control variables together explained between 15 and 30% of the variance. Conclusions: These results suggest that there is a strong association between depression and oral quality of life, suggesting further negative health impacts of depression in addition to those already quantified with regard to physical health. However, these cross-sectional data cannot prove causality. Future research should further explore the mechanisms of the association of depression and oral quality of life through the use of longitudinal data. Impact: The understanding of psychosocial and other factors which influence patients ratings of quality of life is crucial to the accurate interpretation of findings by researchers, clinicians, and policy makers. Recognizing that depression is a significant correlate of oral health outcomes improves the measurement of oral quality of life and provides a potential avenue for interventions to improve oral health outcomes. Prostate Cancer Quality of Life and Outcomes Research among Patients with Low Socioeconomic Status: An Overview of the VA Cancer of the Prostate Outcomes Study (VA CaPOS) Contact: Simon Kim, MPH. Chicago, VA Hospital. SJ Knight, E Moran, CN Robertson, and JE Smith. Charles Bennett, MD, PhD. Chicago VA Hospital, Chicago, IL. Objectives: Outcomes assessment for prostate cancer are important, because of debates over the benefits and costs of alternate treatments and outcomes. Because of a lack of evidence of survival benefits with specific therapies, quality of life (QOL) evaluations have taken on increased importance. QOL is rarely assessed among racial/ethnic minorities and men of lower socioeconomic status, who make up a disproportionately large part of the prostate cancer burden. We have initiated the first multi-center QOL outcomes study of lower socioeconomic status men, the VA Cancer of the Prostate Outcomes Study (VA CaPOS). Methods: VA CaPOS QOL information is collected from prostate cancer patients, spouses, and physicians at six VA medical centers. Because of low rates of literacy, interviewers assess QOL, involvement in care, and the relative importance of likely outcomes following alternative treatments. Spouses provide proxy ratings of patient QOL. Physicians provide information on patients performance status and the patients perceived preferences for alternate outcomes. Medical records and electronic databases are reviewed for sociodemographic characteristics and relevant clinical characteristics. Results: Currently, 601 men with prostate cancer are included in the VA CaPOS, over half of whom are African American. The mean time since diagnosis was 1.4 month for newly diagnosed patients and over 4 years for the rest. QOL responses were most favorable for newly diagnosed, intermediate for stable metastatic disease, and poorest for progressive metastatic disease patients, most of whom had been followed for several years. Spouse emotional well-being assessments were significantly worse than those of individual patients. While patients were not able to provide reliable estimates of their own preferences for future QOL states, they were able to respond reliably to questions phrased as a comparison of the preferences of two hypothetical patients. While African American prostate cancer patients were more likely to have advanced stage disease at the time of diagnosis, after adjustment for differences in health literacy, race was no longer a significant predictor of having advanced prostate cancer. Conclusions: The VA CaPOS provides useful information on health status, QOL, and low literacy for VA prostate cancer patients. Our results indicate that valid and reliable assessments in low literacy populations are feasible, but that long-term evaluations are needed to detect clinically meaningful information on QOL as the disease progresses. Alternative sources of QOL information, such as spouses, provided results that had poor concordance for emotional and social functioning, but were generally valid for other dimensions of health. The reliability of patient ratings Monitor, January 2000, volume 5, issue 1 6 of future QOL states was increased when questions were based on two hypothetical friends rather than consideration by the patients themselves of two potential, but different, future health states. Observational database efforts are potential sources of important information for lower socioeconomic status patients who are faced with difficult therapeutic decisions, limited financial resources, and concerns over both quantity and quality of life outcomes with alternative therapies. Impact: VA CaPOS provides useful information on health status, QOL, and low literacy for VA prostate cancer patients. It also provides useful information about spouse proxy ratings of patient QOL. A Disease-Targeted Measure of Health-Related Quality of Life (HRQOL) for Patients with Chronic Liver Disease the LDQOL 1.0. Contact: Ian Gralnek,MD, MSHS. West Los Angeles VA Medical Center, Los Angeles, CA. RD Hays, HR Rosen, EB Keeffee, DM Jensen, and P Martin. Objectives: The development and validation of a patient-centered HRQOL outcomes measure is timely and needed for individuals with chronic liver disease. Disease-targeted measures can capture small, yet clinically meaningful changes in patients health status due to an intervention or disease progression that a generic instrument may fail to detect. Therefore, the objective of this study is to evaluate the psychometric properties (reliability and validity) of a newly developed disease-targeted HRQOL instrument (the LDQOL 1.0) for individuals with chronic liver disease. Methods: Disease-targeted items in the LDQOL 1.0 were developed from focus groups of patients with chronic liver disease awaiting liver transplantation, expert hepatology panel input, and an extensive review of the literature. Cognitive interviews were conducted to detect potential problems with instrument design or wording of items. The HRQOL instrument was then constructed consisting of 36 generic items (SF-36) Initiatives continue on page 15

12 Veterans Health Use of a Generic Cost-Effectiveness Measure in Veterans Administration Patients Jeffrey M. Pyne, MD 1 Robert M. Kaplan, PhD 2 1 Dr. Pyne is Assistant Professor and Staff Physician, Department of Psychiatry, Central Arkansas Veterans Healthcare System and University of Arkansas for Medical Sciences, Little Rock, AR. He is supported by a VA Career Development Award. 2 Dr. Kaplan is Professor and Chair, Department of Family and Preventive Medicine, University of California San Diego, La Jolla, CA. He is also one of the principal developers of the Quality of Well-Being Scale. Generic cost-effectiveness analysis requires that the same unit of effectiveness be used regardless of diagnosis. A complement to generic cost-effectiveness analysis is disease-specific cost-effectiveness analysis. The primary advantage of generic cost-effectiveness analysis is the ability to compare cost-effectiveness ratios across a variety of diagnostic groups. A Department of Health and Human Services working group has recommended the quality-adjusted life year (QALY) as the effectiveness units for cost-effectiveness studies. 1 The original interviewer-version Quality of Well-Being Scale (interviewer QWB) is a generic health-related quality of life (HRQL) measure that produces QALYs as an output and was designed for use in cost-effectiveness analyses. The interviewer QWB has been used in numerous costeffectiveness studies over the past 25 years. However, only recently has the interviewer QWB been used in the assessment of mental health outcomes. 2-5 One of the frequent criticisms of the interviewer QWB that has limited its widespread use in research and clinical settings is the requirement of a trained interviewer to collect the data, making it more expensive to administer than selfadministered measures. In response to this criticism, a self-administered version of the Quality of Well-Being Scale (QWB-SA) was developed. 6 The QWB-SA includes the same four subscales as the interviewer QWB (symptom/problem complex, mobility, physical activity, and social activity), can be printed on two sides of a single page, is available in a scannable form, and usually takes less than seven minutes to complete. The output from the QWB- SA is a preference-weighted index score between 0.0 (death) and 1.0 (perfect health), similar to the interviewer QWB. A comparison between the interviewer QWB and QWB-SA, using the same scoring algorithm for each instrument, found relative equivalence between the two types of administration and good test-retest reliability. 6 Content of the QWB-SA The QWB-SA items are based on those used in the interviewer QWB. The primary differences between the QWB-SA and interviewer QWB are the mode of administration, the number of symptoms and problems, and the format of the questions. 6, 7 The total number of items on the QWB- SA is 76 compared to 43 on the interviewer QWB. The QWB-SA includes 58 symptoms or problems compared to 26 in the interviewer QWB. The increased number of symptoms resulted from a disaggregation of some of the original symptoms/problem complexes into individual items and an increased number of mental health and self-rated health items. In addition, a group of physicians suggested the inclusion of some new symptoms to make the form more similar to a review of systems examination, which is typically part of a thorough physical exam. The review of systems examination will include questions about current symptoms from all major body systems (e.g., cardiovascular, respiratory, neurologic, etc). The format for the QWB-SA includes five parts. Part I asks about acute and chronic symptoms. First, respondents are asked to respond yes Please address all correspondence to: Jeffrey M. Pyne, MD, Central Arkansas Veterans Healthcare System, Department of Psychiatry 116F2/NLR, 2200 Fort Roots Drive, North Little Rock, AR 72114, phone: (501) , fax: (501) , pynejeffreym@exchange.uams.edu Clinical Practice Applications

13 Veterans Health or no if they have each of 19 chronic symptoms or problems. Examples of the symptoms include blindness or severely impaired vision in both eyes (a separate item asks about one eye), and speech problems. Part I also asks about 25 acute physical symptoms (e.g., headache, coughing or wheezing) and 11 mental health symptoms (e.g., spells of feeling upset, downhearted, and blue). The format for these items asks respondents to think back over the last three days and to indicate if the symptom was present yesterday, two days ago, and/or three days ago. Part II uses a similar format and asks about self-care. It includes two items and asks if the respondent had been a patient in a hospital or other healthcare facility and whether he or she had needed help caring for him or herself. Part III asks about mobility (e.g., use of public transportation or driving). Part IV asks about physical functioning (e.g., walking, confinement to a bed or chair). Part V asks about performance of usual activity (e.g., work, school, or housework). Initial studies have demonstrated that the QWB-SA possesses good psychometric properties. 6, 8 In addition, the QWB-SA was found to be sensitive to clinically significant changes in the HRQL of migraineurs, 9 differences between a clinic sample and patients with rheumatoid arthritis, 10 and pre to post-operative changes in a sample of patients who have undergone cataract surgery. 11 Others have shown that the QWB-SA is acceptable to older adults as a mail-out survey. 12 It should be noted that the weighting system for the QWB-SA is different from the original interviewer QWB. Although the measures are highly correlated, scores on the QWB-SA are typically systematically lower by approximately 0.11 units. It is not known how this systematic difference between cross-sectional QWB-SA and interviewer QWB scores will affect the comparison of HRQL change scores or QALYs associated with particular interventions using the QWB-SA or interviewer QWB. Because QALYs comprise the denominator of cost-effectiveness ratios, it is important to know the relationship between QALYs derived from the QWB-SA and interviewer QWB for comparison across samples. This issue is being explored in current studies utilizing both QWB measures. A separate scoring algorithm derived specifically for the QWB-SA is currently available and several ongoing investigations are documenting the instrument s reliability, validity, and sensitivity in a variety of populations. The QWB-SA scoring algorithm is based on preferences for QWB-SA health states derived from a primary care patient sample of over 1000 subjects. The preferences for QWB-SA health states were elicited using a categorical rating scale method similar to that used for the interviewer QWB. A multiattribute utility model was then used to assign the actual weights. VA experience with the QWB-SA As mentioned above, the QWB-SA is being used in a variety of ongoing studies to evaluate its feasibility, reliability, validity, and sensitivity to change. VA studies using the QWB-SA as a generic HRQL measure include samples of patients with major depression and posttraumatic stress disorder. Non-VA samples where the QWB-SA is being used include patients with migraine headache, rheumatoid arthritis, cataract, diabetes mellitus, chronic obstructive pulmonary disease, multiple sclerosis, back pain, and cancer. To date, the majority of our VA experience using the QWB-SA is limited to samples of patients with major depression. Data collection on a one year observational longitudinal QWB-SA study of VA inpatient and outpatient depressed subjects was recently completed (n=67). In earlier studies we found the interviewer QWB to be sensitive to cross-sectional and longitudinal depression severity over a period of six months. 2, 3 One specific aim of the QWB-SA study was to evaluate the sensitivity of the QWB-SA to acute depressive symptom change (over a course of four weeks) and changes with longitudinal observations. During a face to face research interview we collected data on depression severity using the Hamilton Rating Scale for Depression Monitor, January 2000, volume 5, issue 1 8 (HRSD-17) and the Beck Depression Inventory and completed the interviewer QWB and QWB-SA. The research subject completed the QWB-SA during the research interview, and different members of the research staff completed the HRSD-17 and interviewer QWB in order to maintain the independence of assessment. We conducted the research interviews weekly for four weeks or until the subject s depressive symptoms decreased by 50 percent according to the HRSD-17. We then repeated the interview quarterly for one year. We found that the VA patients in this study were willing and able to complete the QWB-SA easily with minimal to no assistance from the research staff. In general, it took patients 5 to 15 minutes to complete the questionnaire. The average time to complete the QWB-SA in non-mental health samples is approximately seven minutes. Because the QWB-SA was completed as part of an inperson interview, we had the opportunity to review the questionnaire prior to the patient leaving the interview site, and therefore, missing data was minimal. We are currently analyzing the data to verify the sensitivity of the QWB-SA to acute and long-term changes in depression severity. Overall we have been encouraged by the performance of the QWB-SA in measuring HRQL and calculating QALYs in VA patients with depression. The interviewer QWB and QWB-SA were designed for use in cost-effectiveness analyses and to inform broad health policy decisions through the use of a common measure of effectiveness. The QWB-SA appears to be acceptable to VA patients and feasible as a self-administered measure. Soon we will report on its sensitivity to acute and long-term sensitivity to change. If the QWB-SA is confirmed as a sensitive HRQL instrument in a wide variety of physical and mental illnesses, then cost-effectiveness ratios, using a common self-administered measure of effectiveness, could be determined for a variety of conditions and treatments within the VA healthcare system. These cost-effectiveness ratios would provide patient-generated data to inform the VA health policy decisions regarding the most efficient blend of Clinical Practice Applications continues on page 18

14 The New VA: Using Patient Outcomes to Drive Health System Performance Thomas L. Garthwaite, MD, MPH Dr. Thomas L. Garthwaite was appointed Acting Under Secretary for Health in the Department of Veterans Affairs on July 1, In this capacity, Dr. Garthwaite is the highest official in the Veterans Health Administration. Despite its unique missions, funding, and structure, the Veterans Health Administration (VHA) within the Department of Veterans Affairs (VA) shares the same fundamental challenge facing all healthcare systems: providing healthcare value. To meet this challenge, we began a transformation in 1995 that dramatically changed the way we do business. This article will examine the underlying principles for that transformation, the key operational strategies and their rationale, and will emphasize the critical role that performance measurement has played in our success. Underlying Principles and Challenges The first principle underlying the transformation is that the business of the Veterans Health Administration is healthcare, not hospitals. Traditionally, independent medical centers competed for programs and funding. Success was measured in number of beds, number of programs and total budget. We believe that collaboration and coordination must replace competition and that success is measured by improved patient outcomes. As a result, we have reorganized into geographic networks of owned facilities and contracted services. We changed the focus of the networks from symptomatic users to an enrolled population. A second principle is that healthcare has become primarily an outpatient activity. Arcane eligibility rules, unclear reimbursement models, and a comfortable practice pattern habit all had led VA to emphasize inappropriate inpatient care. Based on the belief that the site of care is important only if it affects outcome, we have opened over 250 new sites of care and have integrated 52 facilities into 25 collaborative systems. We have also closed more than 50 percent of all acute inpatient beds, decreased admissions by a third and increased outpatient visits from 25 to 35 million per year. A third principle is that healthcare is fundamentally a local activity. Goals for healthcare are similar everywhere in accordance to national care guidelines. But local flexibility in achieving the goals leads to creative improvement. VA had been centralized, hierarchical, and bureaucratic. Therefore, changes have been made so that we can delegate many decisions to our networks while emphasizing the need to measure compliance with national standards. A fourth principle is that the success of future healthcare systems will depend on their ability to integrate and manage information. VA s information system has been lauded for its clinical usefulness, yet it has difficulty sharing data with private or other federal systems. In cooperation with other federal partners, we are committed to making our future versions compatible with emerging national standards. We believe that information management is the key to improving the consistency and quality of care while controlling costs thereby providing value. We have evolved our information system to be able to aid in the provision of care and in analyzing the effectiveness of our system. A fifth principle is that healthcare must reorient itself to be population-directed, ommunitybased, and health-promotive. All of American healthcare had been guilty of focusing on large medical centers that delivered high-tech interventions for patients with advanced disease. We believe that the goal of the healthcare system must be to maximize the health potential of the population it serves. For example, VA has published studies that demonstrate that the systematic administration of pneumovax leads to favorable outcomes: fewer hospital days, fewer dollars expended and fewer deaths. A sixth principle is that healthcare must become more accountable and responsive to those who purchase it and those who use it. In the absence of resource distribution systems that clearly provide incentives for quality, we believe that we should keep the distribution system simple and measure the quality of care as precisely as possible so that we can provide credible and meaningful data that demonstrate the effectiveness and value of our care. We believe that we have implemented the most extensive quality measurement of any healthcare system (for example, our National Surgical Quality Improvement Program s database Research & Policy

15 contains high quality data on over 700,000 major operations). A seventh principle is that medical education and research are accountable public goods. For the VA, education and research are powerful assets that enhance our ability to provide quality care. In addition, they provide an opportunity for us to lead in developing the healthcare systems of tomorrow. Health system administrators often make important and expensive decisions based on too little data with too few statistics. Further, health service researchers often have important data with copious statistics that sit quietly in journals waiting to be discovered and acted upon by administrators so data should drive health system design. With these points in mind, key VA administrators and health service researchers now meet regularly to share information and to plan new studies of relevant issues. Key Strategies The most important strategy used to transform the VA healthcare system has been the deployment of a performance measurement system. This system has several key components that are critical to its success. The first component is the performance contract. The contract is written and signed by each Network Director and the Under Secretary for Health. The contracts have four parts: core executive competencies, comprehensive quality framework adherence, quantitative measures, and organizational emphasis, such as fair workforce treatment, occupational safety and national contributions. Quantitative measures are the cornerstone of the contract and are developed by a committee and approved by the Under Secretary. The measures represent areas that need improvement or monitoring or that are important new initiatives for the year. The contracts articulate a clear direction and emphasis for the coming year. They also make the assessment of success more objective than had been common previously. Since the majority of the measures relate to patient care and clinical outcomes, if Network Directors are to improve the Network s performance (and therefore their own performance) they must engage the clinical staff in the process of understanding the measures and then in changing staff behavior or system design. As long as the measures are clinical and important, this system encourages administrators and clinical staff to work together to improve outcomes. A second component of the system is the measurement development process. Each measurement and the method of collection are clearly defined at the beginning of each year. The measurements target key organizational goals: healthcare value, employer of choice, exceptional accountability, and organizational effectiveness. Our primary goal is to deliver healthcare value, which we define as quality of care per unit of price or cost. We have further defined four domains of quality: technical quality (accurate, consistent, and adherent to current national standards/guidelines), patient satisfaction (patients perception of quality), access (geographic, temporal, knowledge/specialist, and covered benefit), and functional status (health outcome). In addition to the measures used in the performance contracts, many performance measures are in the process of development, testing, and use throughout the year. A committee with representation from both headquarters and the field facilities selects the panel of measures for inclusion in the performance contracts and oversees the development of additional measures for development. The Office of Quality and Performance supports the committee and coordinates the collection and communication of the data. A third component of the system is the actual measurement process. Every effort is made to be certain that the performance measures are accurate and objective. The integrity of the process is enhanced by the use of an external contract for trained chart reviewers. If the needed data is available in our information systems, that data is used. Verification may also include audits by either the Medical Inspector or the Inspector General. A fourth component of the system is the publishing of the results. Data is Monitor, January 2000, volume 5, issue 1 10 compiled and published quarterly to allow every employee to see the network s progress toward its goals. The reports are released within 30 days of the end of each quarter and the yearly performance report is released within 30 days of the end of the year. The rapid reporting of the data is critical to allow employees to relate the measures to the actions they are taking to make improvements toward the network s goals. While the open reporting has created some pressure to compete, the use of approximately 20 measures per year has allowed each network to find both strengths and weaknesses. Educational efforts and the promotion of sharing of innovations have helped move the entire system toward improvement rather than focal excellence at the expense of other networks (and patients). Other important strategies used to transform the VA healthcare system have included universal assignment of patients to primary care providers or teams, facility integrations, optional use of service lines rather than traditional departments, case management, clinical practice guideline implementation, performance-based interviewing to select employees, enhanced computer systems, and many others. Each strategy is aimed to improve care. Most strategies should have an effect on one or more of the performance measures. While the system of care is often too complex to relate a specific action or strategy to a specific measure, the composite effect is evident in the improvements seen in the measurements. Future Directions Future efforts in performance measurement will be directed toward the solution of two fundamental issues. The first issue is measurement definition and accuracy. Many health outcomes that we would like to measure are difficult to assess and capture. For example, outcomes in mental health and chronic medical conditions are more difficult to capture than outcomes from surgery. Even in surgery, we have found that measures of morbidity and mortality are more easily monitored than are functional status changes that result from the Research and Policy continues on page 14

16 Veterans Health Quality Outcomes of the Performance Management Program in The New VA Jonathan Perlin, MD, PhD, MSHA Dr. Perlin became Chief Quality and Performance Officer for the Veterans Health Administration (VHA) of the Department of Veterans Affairs on November 1, In this capacity, he has responsibility for supporting quality improvement and the performance management program throughout VHA's 22 regional networks which operate over 170 medical centers, 650 other facilities including outpatient clinics, and 70 home-care programs. Editor s Note: The accompanying article by Dr. Thomas Garthwaite, The New VA: Using Patient Outcomes to Drive Health System Performance describes the principles and challenges which undergird the transformation process of the Veterans Health Administration (VHA) since 1995 and provide rationale for the comprehensive Performance Management Program. The successful application of measures of structure and, especially, process and outcome, has been critical to rapidly advancing quality in The New VA to levels that increasingly surpass performance in other healthcare settings. This article (1) provides an overview of the Performance Management Program and (2) reports rates of evidence-based preventive healthcare services (e.g., cancer screening and immunization) and secondary prevention and therapeutic interventions (e.g., care for patients with diabetes and ischemic heart disease), which establish national benchmarks. Overview of the Performance Management Program The Veterans Health Administration (VHA) of the Department of Veterans Affairs (VA), the nation s largest integrated health system, implemented the Performance Management Program in 1995 to support its strategic plan to meet challenges of inconsistent healthcare quality and economic inefficiency. The Performance Management Program centrally embraces quality management and operates with the presumption that desired clinical, economic, and other outcomes can and should be specified. VHA leadership is committed to accountability in performance expectations as specified under the Performance Management Program. 1 This Program constitutes the broadest application of performance measures in healthcare management. Performance measures address a broad range of relevant processes and outcomes supporting mission and strategy. Specific measures address both administrative and clinical activities. Administrative measures evaluate factors such as occupational safety, employee education, and labor relations necessary to support operational and clinical improvement. Clinical measures are designed to support systemization of the best research and best practices in provision of healthcare services. This approach is designed to achieve consistent, reliable, efficient, and satisfying care of the highest quality that objectively surpasses governmental goals and existing community benchmarks. Performance measures are also framed by what VHA defines as five Domains-of-Value. The value domains simultaneously constitute areas traditionally presenting challenges in healthcare and, as such, provide focal points for performance measurement and improvement. These domains include: access to care, quality of care, patient functional status, cost of care, and customer satisfaction. Performance measures have been developed around each of these domains. For instance, measures of geographic access and waiting time have been developed to support evaluation and improvement of access to care. Appropriate disease-specific interventions and outcomes (e.g. blood pressure control in hypertension, blood sugar control in diabetes) exemplify measures of care quality. The most comprehensive longitudinal and cross-sectional program for assessment of patient functional status ever performed has been implemented by VHA. Almost 2 million patients have now received a version of the SF-36 optimized and calibrated specifically for Veterans. 2 Importantly, VHA has made a commitment to serially tracking functional status, with the performance goal of improving functional status where possible and slowing the rate of decline where inevitable. Consideration of functional status has been operationalized through specific performance measures. Clinical Performance Measures are supported by development and promulgation of national Clinical Practice Guidelines. Specific guidelines and clinical performance measures are, in part, Health Systems Improvement

17 Veterans Health Table 1: VHA Prevention, Chronic Disease, and Palliative Care Indices VHA Index Prevention Chronic Disease Care (Inpatient) (Outpatient) Palliative Care Component Indicators Immunization: Influenza Immunization Pnuemococcal Vaccination Cancer Screening: Breast Cancer Screening Cervical Cancer Screening Colorectal Cancer Screening Prostate Cancer Screening Substance Use: Alcohol Use Tobacco Use Smoking Cessation Counseling Chronic Obstructive Pulmonary Disease (COPD): Inhaler Use Observation / Education Inhaler Use Observation / Education Diabetes Mellitus: Annual Pedal Pulse Evaluation Annual Sensory examination of Feet Annual Visual Foot Inspection Annual Hemoglobin A1c Annual Retinal Exam Hypertension: Exercise Counseling Nutrition Counseling Ischemic Heart Disease: Aspirin Use post-myocardial Infarction Beta-Blocker use post-myocardial Infarction Cholesterol Management post-myocardial Infarction Advance Directives: Discussion of Resuscitation Status Clinical Management: Hydration / Nutritional Assessment Depression Management Plan Dyspnea Management Plan Pain Management Plan Continuum-of-Care Coordination: VA Home-Based Primary Care VA Hospice Enrollment Community-based Hospice Enrollment Psychosocial Care: Psychosocial Support Caregiver Support determined by high-volume, high-risk conditions. Leveraging the size and scope of VHA and its multiple academic affiliations, multidisciplinary panels of national experts within the health system review and incorporate the best evidence derived from research into guidelines. Accountability is introduced into the process as clinical experts and administrative leaders track clinical performance measures indicating guideline adherence. Through documents such as performance contracts, managers are specifically accountable for achieving realistic but challenging performance targets in defined timeframes. Organizational transformation and the use of performance measures have compressed performance improvement, in a system as large as VHA, from years to months. The measurement system is itself grounded in scientific methods of data measurement, which requires methodologically rigorous data definition, collection, and validation. Statistically valid sampling and frequent feedback to accountable managers is provided to show progress toward goals. In the clinical area, some measures have been devised to facilitate external benchmarking. In other instances, developers of performance measures have turned heavily to health services research to devise new ways to measure performance as a fundamental instrument for change management. Examples include development of the Prevention Index, the Chronic Disease Index, and the Palliative Care Index shown in Table 1. Clinical Quality Outcomes The Prevention Index, the Chronic Disease Index, and the Palliative Care Index are each composed of a number of component indicators. In turn, each indicator is supported by a methodologically viable measure, generally a process measure. Each indicator encompasses the use of best evidence supporting particular healthcare interventions. The indices, serving as healthcare process summaries, characterize and encourage preventive, therapeutic, and palliative healthcare interventions in particular states of health and disease and in appropriate types of healthcare encounters. From its inception in 1996, the Prevention Index has improved by 239 percent, increasing from a value of 34 to 81 on a 100-point scale. Similarly, the Chronic Disease Index has improved 190 percent from 47 to 89. It is exciting that mounting evidence is beginning to link improvements in these process indices with improved health outcomes. Increases in one-year survival for a number of conditions represented in high volume in VA have been noted including: angina pectoris, chronic renal failure, congestive heart failure, chronic obstructive pulmonary disease, and pneumonia. This trend is also observed in mental health disorders: bipolar disorder, major depression, and schizophrenia. 3 Statistically robust improvements in intermediate outcomes including control of hypertension, improved hemoglobin A1c, and lipid profiles have been noted for patients with diabetes. 4 Preliminary evidence demonstrat- Monitor, January 2000, volume 5, issue 1 12 Health Systems Improvement continues on page 16

18 Veterans Health Feature cont d from page 2 in the literature. 15 Ninety percent of the reliable variance in the eight Veterans SF-36 scales are explained by the physical and mental dimensions of health. As in the MOS version of the SF- 36, the two component summary scales are each scored using weights derived from a national probability sample of the US population. They are standardized to the US population and normbased so that the scores have a direct interpretation in relation to the distribution of scores in the US population with a mean of 50 and a standard deviation of 10. Higher scores indicate better health. Each summary is expressed as a T score, which facilitates comparisons between the VA patients and the general US population. Conversion formulas have been developed and validated where comparisons of VA patients with established norms using the MOS SF-36 are possible. In this case the results using the Veterans SF-36 are rescored so that the summary scores for physical and mental are comparable to the original version of the MOS SF-36. Prior published work indicates that measures of the Veterans SF-36 are strongly correlated with sociodemographics and morbidities of the veterans.7 Younger veterans between 20 and 49 years of age are sicker in their mental health status than older veterans who are 50 years of age or older. Veterans who use ambulatory care in the VHS reported lower levels of health status reflecting more disease than a non-va civilian population. These measures of health are important indicators of the disease burden or case-mix of the patient and pertinent to health systems such as the VA for resource allocation decisions and as outcomes of care. Norms have been developed for the VA. Nationally based upon the 1998 National Survey of Veterans in Ambulatory Care. The VA national average for PCS is 35.2 and for MCS is The PCS is about 1.5 standard deviations below the US population and the MCS is 0.7 of one standard deviation below. Application of the Veterans SF-36 in the VA Since 1996 close to 2 million Veterans SF-36 questionnaires have been administered nationally in six national surveys. The VHA has adopted the Veterans SF- 36 as one of the outcome measures in setting its performance measurement system. Several reports documenting these findings have been published and disseminated widely in the VA. These measures are now being used by VHA for multiple purposes. The first is as a measure of disease burden, or disease complexity of the patient. The VHA is composed of 22 regions or Veterans Integrated Service Networks, also termed VISNs. These VISNs are geographically divided among the VA hospitals. VA is keenly interested in determining how to allocate its resources among the VISNs. Based upon the utilization and cost data available in the VHA, it has been determined that veterans 10 points lower (worse health) than other veterans on the physical summary require $1482 per patient per year more than the others. Those veterans who score 10 points lower on the mental health summary require $864 per patient per year in higher costs. The Veterans SF-36 provides a useful indication of greater case-mix, or disease burden, among certain veteran groups. Data has also indicated that the Veterans SF-36 scores vary by more than 50 percent of one standard deviation amongst the VISNs. Those VISNs in the northeastern and western regions of the country have significantly higher physical summary scores (better health) than those VISNs in the southeastern regions. This discrepancy is an indication of differences in the case-mix of patients among these VISNs and has important implications for future resource allocation decisions in the VA. Second, as an outcome measure, the Veterans SF-36 is being administered over time to provide measures of patient outcomes. To this end, the VHA has instituted a system of goals to measure system performance using functional status. These goals were issued by the VA to the United States Congress in One of these goals states: Veteran health scores for physical, mental and social functioning will improve by 1.5% per annum when compared to 1999 baseline norms. The population of veterans served by the VHA will improve by 9% through the year (sic) The goal is to examine the changes in functional status overall for the system. The Veterans SF-36 is being measured nationally on an annual basis through the year A cohort of over 90,000 veterans to be followed annually using the Veterans SF-36 scores has been established to meet this goal and monitor the outcomes of care. The third purpose is for the monitoring of health status for individual patients using summary information from the Veterans SF-36. This information will be made available to clinicians as part of the electronic medical record during the clinic visit. A nationally based clinical trial is now being planned to evaluate the efficacy of this information. The difference in this study with previous published work is that the information is made available to the clinician in real-time during the clinic visit. Patients are administered the health status questionnaire using a computer-based telephone system at home prior to the visit. The profiling of the information will alert the doctor to the specific items that are important for that visit. Summary In the next few years, the VHA is quickly becoming a flagship for developing a system based on outcomes management using patient-centered measures of health status. The distribution of illness in the VA patient population differs markedly from that in the general population and from that of most other healthcare organizations. The Veterans SF-36 is an important assessment tool for systemwide monitoring of case-mix and in the future for assessing outcomes of care. Monitor, January 2000, volume 5, issue 1 13

19 Veterans Health References 1. The VA Responsibility in Tomorrow s Healthcare System: Strategy Paralyzed Veterans of America, Randall M, Kilpatrick KE, Pendergast JF, Jones KR, Vogel WB. Differences in patient characteristics between veterans administration and community hospitals: implications for VA planning. Med Care. 1987;25: Wolinsky FD, Coe RM, Mosely RR, Homan SM: Veterans and nonveterans use of health services: a comparative analysis. Med Care. 1985;23: Kizer K. Functional Status: Fifth Domain of Value. Official VA Memorandum, March 12, Kazis LE, Miller DR, Clark J, Skinner K, Lee A, Rogers W, Spiro III. A, Payne S, Fincke G, Selim A, Linzer M. Health-related quality of life in patients served by the Department of Veterans Affairs: Results from the Veterans Health Study. Arch Intern Med. 1998;158: Kazis L, Miller DR, Clark J, Skinner K, Lee A, Rogers W. et al. Health-related quality of life in veterans: The Veterans Health Study: Technical Report. White Paper to the Department of Veterans Affairs, Health Services Research and Development Service, Washington D.C., Kazis LE, Ren XS, Lee A, Skinner K, Rogers W, Clark J, Miller DR. Health status in VA patients: results from the Veterans Health Study. Am J Med Qual. 1999;14: Brook RH, Ware JE, Rogers WH, et al. Does free care improve adults health? New Engl J Med. 1983;309: Tarlov AR, Ware JE, Greenfield S, Nelson EC, Perrin E, Zubkoff M. The Medical Outcomes Study: an application of methods for monitoring the results of medical care. JAMA. 1989;262: Ware JE Jr, Sherbourne CD. The MOS 36-item short form health survey (SF-36) I. conceptual framework and item selection. Med Care. 1992;30: Ware JE, Kosinski M, Bayliss MS, McHorney CA, Rogers WH, and Raczek A. Comparison of methods for scoring and statistical analysis of SF-36 health profile and summary measures: Summary of results from the Medical Outcomes Study. Med Care. 1995;33(4):AS264-AS Kazis LE, Miller D, Clark J, Skinner K, Lee A, Ren XS, Spiro III A, Rogers W, Ware JE. Improving the response choices on the SF-36 role functioning scales: results from the Veterans Health Study (Medical Care Supplement, forthcoming). 13. Kazis LE, Lee A, Ren XS, Skinner K, Roger W. Health Status and Outcomes of Veterans: Physical and Mental Component Summary Scores (Veterans SF-12): 1998 National Survey of Hospitalized Patients. Office of Quality and Performance, and Health Assessment Project, Health Services Research and Development Service. Washington D.C., and Bedford Massachusetts, March Ware JE with Kosinski M, Keller SD. SF-36 Physical and Mental Health Summary Scales:A User s Manual. The Health Institute, New England Medical Center, Boston, Kazis LE., Wilson N. Health Status and Outcomes of Veterans: Physical and Mental Component Summary Scores (SF-36V): 1998 National Survey of Ambulatory Care Patients, Mid-Year Executive Report. Office of Quality and Performance, and Health Assessment Project, Health Services Research and Development Service. Washington D.C. and Bedford, Massachusetts, July v Research and Policy cont d from page 4 operation. We plan to pursue research on health outcome assessment as part of our health services and quality enhancement research agendas. The second issue is measurement capture. While we believe that our performance measurement system has been valuable, we invest heavily in the collection and analysis of data. Ideally, the collection of relevant performance data will occur automatically as we care for patients and the data will be automated such that the aggregation and analysis of it will be highly efficient. As we evolve our computerized patient record, we believe that it is critical to build in outcome measurement as a key functionality of the system. Summary The transformation of the Veterans Health Administration over the past five years has relied on performance measurement, primarily of health outcomes, as its main driving force. These measurements have defined the direction, set priorities, and demonstrated the progress. Our focus on health outcomes has helped to direct the administrative and clinical energy where it belongs-- improving patient care. v Monitor, January 2000, volume 5, issue 1 14

20 Veterans Health Initiatives cont d from page 6 supplemented with 77 disease-targeted items. A multicenter, cross-sectional field test was conducted. Results: 221 consecutive ambulatory adult patients being evaluated for liver transplantation participated in this field test (64.1% male; median age = 51 years (range years); 68.9% white, 6.8% Asian/Pacific Islander, 3.9% frican- American, 1.9% Native American, 18.5% other or multiracial). The LDQOL 1.0 is a self-report measure that includes 21 multiitem scales (number of items): physical functioning (10), role limitations-physical (4), pain (2), liver disease-related symptoms (19), emotional well-being (7), role imitations-emotional (3), energy (4), cognitive function (6), memory (4), concentration 3), hopelessness (7), loneliness (6), stigma of liver disease (8), social function (2), quality of social interaction (5), sexual function (3), sleep 6), general health perceptions (7), health distress (4), effects of liver disease 9), and impact of liver disease (4). Internal consistency reliabilities (Cronbach s alpha) ranged from 0.67 to 0.95 (median=0.86); 20/21 reliability estimates were excellent, alpha >= All 21 scales were significantly (p<0.05) associated with self-reported severity of symptoms and 4 scales with duration of liver disease (better HRQOL related to less severity and shorter duration of liver disease). Role limitations-physical was most strongly related to severity of symptoms (p<0.01); sleep was most strongly associated with duration of disease (p<0.05). Worse physical functioning (p<0.01), worse sexual functioning among males (p<0.01), role limitations-physical (p<0.01), more liver disease-related symptoms (p<0.05), and greater negative effects of liver disease (p<0.05) were all significantly associated with higher Child-Pugh class. Conclusions: This multicenter study demonstrates the high degree of reliability and construct validity of the LDQOL 1.0 for individuals with chronic liver disease. This HRQOL outcomes instrument is able to measure significant impairment of daily functioning not detected by more traditional clinician-rated methods (e.g., Child-Pugh classification). The LDQOL 1.0 is now ready for implementation into prospective, longitudinal studies. Impact: 1. The LDQOL 1.0 will allow for a better understanding of HRQOL in patients with Monitor, January 2000, volume 5, issue 1 chronic liver disease. 2. The LDQOL 1.0 fits well with the VHA s initiative to build a system of data collection that will integrate both generic and disease-targeted functional status instruments into a routine process of HRQOL data collection. 3. HRQOL data collection in chronic liver disease will allow for case mix comparisons, evaluation of changes in patient functional status over time and its potential relationship to processes of care, and generation of patient summary information for the clinician in their care of patients with chronic liver disease. 4. The LDQOL 1.0 may provide important information on resource utilization within the VA health care system such as response to therapeutic interventions (e.g., therapies for chronic viral hepatitis and liver transplantation). v 15 Research Summary cont d from page 4 status of geriatric psychiatric patients, a forward-looking, retrospective study of 106 consecutive admissions to a geriatric psychiatric unit at the Houston Veterans Affairs Medical Center Hospital was done. It was found that psychopathology and cognitive status, but not medical burden, contributed to the variance in functional status of geriatric psychiatric inpatients for both admission scores and for changes in scores during hospitalization. Improvements in cognitive state and psychopathology were associated with improvements in functional status during hospitalization. v

21 Veterans Health Health Systems Improvement cont d from page Table 2: Benchmark Clinical Quality Outcomes of the Performance Management Program Prevention Index VHA US PHS NCQA HEDIS VA Improvement Component Indicators 1999 HP * (%) from 1996 (%) Goals (%) baseline (%) Immunization: Influenza Immunization N/A 271 Pnuemococcal Vaccination N/A 275 Cancer Screening: Breast Cancer Screening Cervical Cancer Screening Colorectal Cancer Screening N/A 218 Prostate Cancer Screening 66 N/A N/A >500 Substance Use: Alcohol Use N/A >500 Tobacco Use N/A 194 Smoking Cessation Counseling *ftp:// US - United States PHS - Public Health Service HP 2000 Goals - Healthy People 2000 NCQA - National Committee for Quality AssuranceHEDIS - Health Plan Employer Data & Information Set (%) - Percent successfully meeting goal ing lower rates of vascular complications of diabetes is now accruing. Improvements in component indicators of the three indices have been observed that exceed U.S. Government Healthy People 2000 goals and private sector performance. For example, current VHA rates of pneumococcal vaccination (77 percent) and influenza immunization (76 percent) in patients with chronic disease or over 65 years of age exceed the U.S. Public Health Service Healthy People 2000 goal of 60 percent by 17 percent and 16 percent, respectively. VA success rates also exceed community rates published by the Centers for Disease Control (CDC) as well as more recent population-based rates of vaccination in older persons or those with chronic disease of 51 percent for the pneumococcal vaccine and 68 percent for influenza immunization. 5, 6 The evidence supporting preventive healthcare interventions is predicated on an observed correlation between those interventions and improved definitive outcomes. The clinical benefit predicted by robust support for the preventive health services embraced by the Prevention Index is significant. With respect to immunization, for example, a recent publication describing clinical outcomes of VA patients with chronic lung disease reports a 43 percent reduction in the number of hospitalizations for pneumonia and influenza, and a 29 percent reduction in the risk of death from all causes, with administration of the pneumococcal vaccine alone. During the two year outcome period, a 72 percent reduction in hospitalizations for influenza and pneumonia and an 82 percent reduction in deaths from all causes was observed for patients receiving both immunizations appropriately. Moreover, this was associated with a cost savings of $294 per vaccine recipient over the two-year period. 7 Table 2 summarizes accomplishments in preventive healthcare supported by the Prevention Index. Another significant intervention that has been driven by the use of performance measures is the administration of beta-blocker medications after heart attack. Beta-blocker therapy after heart attacks reduces risk of death in the subsequent two years by 43 percent and reduces the risk for rehospitalization by 22 percent. 8 The 1999 VA rate of betablocker administration after heart attack is 94 percent, exceeding 1999 non-va administration rates of 82 percent recorded by the National Committee for Quality Assurance (NCQA) and far Monitor, January 2000, volume 5, issue 1 16 exceeding rates of 49 percent recorded previously across a broad cross-section of non-governmental hospitals. 9, 10 While ultimate outcomes data, including survival, is still accruing, VA performance on these process measures predicts a substantial number of averted deaths and hospitalizations relative to published levels of performance in other healthcare environments. From an operational perspective, it is important to recognize that these improvements in clinical quality outcomes occurred while providing care to a larger number of veterans on a neutral budget. During this time, the overall percentage of satisfied patients (customer satisfaction domain) recorded an increase from 86 percent in 1995 to 91 percent in Functional Health Status Outcomes The large scale cross-sectional assessment of functional status and the longitudinal Veterans Health Study have provided significant insight into the health-related quality of life of VHA patients. Results of the Veterans Health Study demonstrate correlation between functional status and patient satisfaction, 11 treatment decisions in mental health, 12 leg

22 Veterans Health pain, 13, 14 and chronic lung disease. 15 Specific performance measures are attached to these clinical conditions as well as patient satisfaction. The Veterans Health Study provides insight into the relative disease burden of the VHA population. VHA averages for the physical and mental component summary scores are significantly below the national means for non-va population. 2 Additional insight into regional variations in disease burden throughout the VHA system carries important implications for resource allocation. On the basis of disease, age, and locality, cohort -specific variation provides insight into opportunities for clinical performance improvement. For example, these studies reveal that younger veterans are disproportionately afflicted with poorer mental health than older veterans. 2 Another finding suggests that providing patients with the opportunity to become more involved in their own healthcare decision-making increases patient satisfaction. 11 In summary, VHA has committed to tracking and managing functional status outcomes. Further investigation into mechanisms to most successfully utilize functional status data for improving healthrelated quality of life and other outcomes is necessary. Conclusion The Performance Management Program has achieved improvements in clinical quality that exceed U.S. Public Health Service Healthy People 2000 goals, private sector standards (e.g., NCQA), and published levels of performance. The Performance Management Program unifies managers and clinicians in purpose and provides the first demonstration of a health system consistently using the best scientific evidence in clinical practice to reliably and efficiently achieve the highest quality health outcomes. VHA accomplishments using the Performance Management Program are unmatched by any other health system in scope, magnitude, or speed; it serves as a model for improvement in other healthcare settings. Critical innovations of the Performance Management Program include systematized commitment to the best scientific evidence for clinical practice, explicit accountability for measured performance, incorporation of functional status measurement and management, and precise alignment of the program as a strategy for fulfilling the Department of Veterans Affairs mission to provide the highest quality healthcare for America s veterans. References 1. Kizer KW, Vision for Change. 1995; Washington, DC: Department of Veterans Affairs. 2. Kazis LE, Ren XS, Lee A, Skinner K, Rogers W, Clark J, Miller DR. Health status in VA patients: results from the veterans Health Study. Am J Med Qual. 1999; 14(1): Kizer KW, The New VA : A national laboratory for health care quality management. Am J Medical Qual. 1999;14(1): Sawin CT, Walder DJ, Bross DS, Pogach LM, Diabetes process and outcome measures in the VHA, in press, Centers for Disease Control and Prevention, Pneumococcal and influenza vaccination levels among adults > 65 years United States MMWR. 1997;46: Petersen RL, Saag K, Wallace RB, Doebbeling BN, Influenza and pneumococcal vaccine receipt in older persons with chronic disease: a population-based study. Med Care. 1999;37(5): Nichol KL, Baken L, Wuorenma J, Nelson. The health and economic benefits associated with pneumococcal vaccination of elderly persons with chronic lung disease. Archiv Intern Med. 1999; 159(20): Soumerai SB, McLaughlin TJ, Spiegelman D, Hertzmank E, Thibault G, Goldman L. Adverse outcomes of underuse of beta-blockers in elderly survivors of acute myocardial infarction. JAMA. 1997; 277(2): ftp:// benchmk.doc 10. Krumholz HM, Radford MJ, Wang Y, Chen J, Marciniak TA. Early betablocker therapy for actue myocardial infarction in elderly patients. Ann Intern Med. 1999; 131(9): Harvey RM, Kazis L, Lee AF. Decision-making preference and opportunity in VA ambulatory care patients: association with patient satisfaction. Res Nurs Health. 1999; 22 (1): Hankin CS, Spiro A 3rd, Miller, Kazis L. Mental disorders and mental health treatment among US Department of Veterans Affairs outpatients: The Veterans Health Study. Am J Psychiatry. 1999; 156(12): Ren XS, Selim AJ, Fincke G, Deyo RA, Linzer M, Lee A, Kazis L. Assessment of functional status, low back disability, and use of diagnostic imaging in patients with low back pain and radiating leg pain. J Clin Epidemiol. 1999; 52 (11): Selim AJ, Ren XS, Ficke G, Deyo RA, Rogers W, Miller D, Linzer M, Kazis L. The importance of radiating leg pain in assessing health outcomes among patients with low back pain. Results from the Veterans Health Study. Spine. 1998; 23(4): Selim AJ, Ren XS, Fincke G, Rogers W, Lee A, Kazis L. A symptom-based measure of the severity of chronic lung disease: results from the Veterans Health Study. Chest. 1997; 111(6): v Monitor, January 2000, volume 5, issue 1 17

23 Veterans Health Clinical Practice Applications cont d from page 8 clinical services. We may find that current clinical priorities need to be modified to better reflect the most efficient use of VA healthcare resources from a cost-effectiveness point of view. As a self-administered measure, the QWB-SA may be a useful and practical tool for use in research and clinical settings to inform healthcare resource allocation decisions both within and across service sectors within the Veteran Health Administration. References 1. Gold M, Siegel J, Russell L, Weinstein M. Cost-effectiveness in health and medicine. New York: Oxford University Press, Inc.; 1996: Pyne JM, Patterson TL, Kaplan RM, Gillin JC, Koch WL, Grant I. Assessment of the quality of life of patients with major depression. Psychiatr Serv. 1997;48: Pyne JM, Patterson TL, Kaplan RM, et al. Preliminary longitudinal assessment of quality of life in patients with major depression. Psychopharmacol Bull. 1997;33: Patterson T, Kaplan R, Grant I, et al. Quality of well-being in late life psychosis. Psych Res. 1996;63: Patterson T, Shaw W, Semple S, et al. Health-related quality of life in older patients with schizophrenia and other psychoses: relationships among psychosocial and psychiatric factors. Int l J of Geriatric Psych. 1997;12: Kaplan R, Ganiats T, Rosen P, Sieber W, Anderson J. Development of a selfadministered Qualityof Well-Being scale (QWB-SA): initial studies. Qual Life Res. 1995;4: Sieber WJ, Kaplan RM, Ganiats TG. The Quality of Well-Being scale - Self- Administered (QWB-SA): initial scoring and validation. San Diego: University of California San Diego; unpublished manuscript. 9. Sieber WJ, David KM, Adams J, Ganiats TG, Kaplan RM. Documenting the burden of illness in migraineurs with the Quality of Well-Being scale - Self- Administered (QWB-SA). San Diego: University of California San Diego; unpublished manuscript. 10. Sieber WJ, Frosch D, Wiesman M, Kaplan R. Validity of the Quality of Well- Being generic measure for patients with arthritis. National Meeting of American College of Rheumatology. San Diego, CA; Kaplan R, Rosen P, David K, Sieber W. Measuring outcomes of cataract surgery using the Qualityof Well-Being scale and the VF-14. Qual Life Res. 1999;8: Andresen E, Rothenberg B, Kaplan R. Performance of a self-administered mailed version of the Quality of Well-Being (QWB-SA) questionnaire among older adults. Med Care. 1998;36: v President Alvin R. Tarlov, MD Executive Director Leslie Lipkind, MBA Director of Marketing & Memberships Services Keri Collette, MHP Director of Publications and Special Projects Jennifer T. Le, MPH Program Assistant Chanel Thomas BOARD OF TRUSTEES Wade M. Aubry, MD, Chair BlueCross BlueShield Association Helen Darling Group Benefits and Health Care Watson Wyatt Worldwide Jerome H. Grossman, MD Lion Gate Management, Inc. George J. Isham, MD HealthPartners William Jacott, MD American Medical Association Kathleen N. Lohr, PhD Research Triangle Institute Dennis O'Leary, MD Joint Commission on Accreditation of Health Care Organizations Alvin R. Tarlov, MD Rice University, James A. Baker Institute of Public Policy John E. Ware, Jr., PhD QualityMetric, Inc. Walter J. McNerney Founding Chairman and Distinguished Life Trustee SCIENTIFIC ADVISORY COMMITTEE Kathleen N. Lohr, PhD, Chair Research Triangle Institute Neil K. Aaronson, PhD The Netherlands Cancer Institute Jordi Alonso, MD, PhD Institut Municipal d'investigacio Medica, Barcelona Audrey Burnam, PhD The RAND Corporation Edward B. Perrin, PhD. University of Washington 6. Kaplan RM, Sieber WJ, Ganiats TG. The Quality of Well-Being Scale: comparison of the interviewer-administered version with the self-administered questionnaire. Psych and Health. 1997;12: Monitor, January 2000, volume 5, issue 1 The Monitor is a web-exclusive publication for members of the Medical Outcomes Trust. Medical Outcomes Trust Monitor. Copyright 2000 by Medical Outcomes Trust. Editor: Jennifer T. Le, MPH ISSN Number: Donald L. Patrick, PhD, MSPH University of Washington Ruth E. K. Stein, MD Albert Einstein College of Medicine MEDICAL OUTCOMES TRUST 198 Tremont St. PMB # 503 Boston, MA (617) main (617) fax motrust@worldnet.att.net

24 GENERAL PRACTICE Validating the SF-36 health survey questionnaire: new outcome measure for primary care J E Brazier, R Harper, N M B Jones, A O'Cathain, K J Thomas, T Usherwood, L Westlake Medical Care Research Unit and Department of General Practice, University of Sheffield Medical School, Sheffield S10 2RX J E Brazier, lecturer in health economics R Harper, research associate N M B Jones, statistician A O'Cathain, research associate K J Thomas, senior research associate T Usherwood, senior lecturer in general practice L Westlake, statistician Correspondence to: Mr Brazier. BMJ 1992;305:160-4 Abstract Objectives-To test the acceptability, validity, and reliability of the short form 36 health survey questionnaire (SF-36) and to compare it with the Nottingham health profile. Design-Postal survey using a questionnaire booklet together with a letter from the general practitioner. Non-respondents received two reminders at two week intervals. The SF-36 questionnaire was retested on a subsample of respondents two weeks after the first mailing. Setting-Two general practices in Sheffield. Patients patients aged years randomly selected from the two practice lists. Main outcome measures -Scores for each health dimension on the SF-36 questionnaire and the Nottingham health profile. Response to questions on recent use of health services and sociodemographic characteristics. Results-The response rate for the SF-36 questionnaire was high (83%) and the rate of completion for each dimension was over 95%. Considerable evidence was found for the reliability of the SF-36 (Cronbach's a >0-85, reliability coefficient >0 75 for all dimensions except social functioning) and for construct validity in terms of distinguishing between groups with expected health differences. The SF-36 was able to detect low levels of ill health in patients who had scored 0 (good health) on the Nottingham health profile. Conclusions-The SF-36 is a promising new instrument for measuring health perception in a general population. It is easy to use, acceptable to patients, and fulfils stringent criteria of reliability and validity. Its use in other contexts and with different disease groups requires further research. Introduction It is important to be able to measure the perception of health of the population to assess the benefit of health care interventions and to target services. However, existing measures of mortality and morbidity in the NHS are too narrow, particularly in general practice, to measure the benefit of interventions aimed at improving a wide range of dimensions including mobility, functioning, mental health, and overall well being. Researchers have developed measures to assess the health of people with specific diseases or disabilities,"2 but these are of limited application when studying people with more than one condition or comparing perceived health across different groups. What is required is a measure which is comprehensive and sensitive to the full range of illness. To be of practical use the measure must also be brief and easy to use Ȯne measure which is sensitive to health differences in a general population has been developed out of the Rand Corporation's health insurance experiment, a comprehensive evaluation of alternative methods of financing health care in the United States.3 The original general health measure was lengthy, containing 108 items. In an attempt "to develop a general health survey that is comprehensive and psychometrically sound, yet short enough to be practical for use in large scale studies of patients in practice settings,"4 the authors experimented with several shortened versions. The short form 20 has already been fielded with some success in the medical outcomes study surveys in the United States' and in Scotland.6 However, the substantially revised short form 36 health survey questionnaire (SF-36) has yet to be independently validated in Britain. We examined the reliability and validity of the SF-36 in a British population, and compared it with the Nottingham health profile,7 which is widely used in Britain. Methods The SF-36 questionnaire is a self administered questionnaire containing 36 items which takes about five minutes to complete. It measures health on eight multi-item dimensions, covering functional status, well being, and overall evaluation of health (table I). TABLE I-Dimensions of the SF-36 health survey questionnaire Area Dimension No of questions Functional status Physical functioning 10 Social functioning 2 Role limitations (physical problems) 4 Role limitations (emotional problems) 3 Wellbeing Mental health 5 Vitality 4 Pain 2 Overall evaluation of health General health perception 5 Health change* I Total 36 *This item is not included in the eight dimensions nor is it scored. Five of these dimensions are similar to those in the Nottingham health profile, but items in the SF-36 questionnaire are claimed to detect positive as well as negative states of health.4 In six of the eight dimensions patients are asked to rate their responses on three or six point scales (box) rather than simply responding yes or no as in the Nottingham questionnaire. For each dimension, item scores are coded, summed, and transformed on to a scale from 0 (worst health) to 100 (best health). We conducted face to face interviews using the original American version of the SF-36 in a general practice surgery and among colleagues to examine its acceptability. As a result the wording of six questions was altered slightly. This anglicised version of the 160 BMJ VOLUME JULY 1992

25 Samples of questions from the SF-36 The following questions are about activities you might do during a typical day. Does your health limit you in these activities? If so, how much? Yes, limited Yes, limited No, not limited a lot a little at all Climbing several flights of stairs Bending, kneeling, or stooping Walking half a mile These questions are about how you feel, how things have been with you during the past month. How much time during the past month: A good All of Most of bit of Some of A little of None of the time the time the time the time the time the time Did you feel full of life? Q Have you felt downhearted and low? Has your health limited your social activities (like visiting friends or close relatives)? SF-36 was incorporated into a booklet, together with the Nottingham health profile and questions on sociodemographic characteristics and recent use of health services. We conducted a pilot postal survey of 120 patients from a general practice list to test the acceptability of mailing the booklet. We obtained a response rate of 40% without reminders, with a good completion rate. The questionnaire booklet was sent to 1980 people aged years randomly selected from two general practice lists in Sheffield. It was accompanied by a letter from the general practitioner, endorsing the aims of the study. Two reminder letters and further booklets were sent to non-respondents at intervals of two weeks. To examine the retest reliability a copy of the SF-36 questionnaire was sent to 250 randomly selected respondents after two weeks. STATISTICAL ANALYSIS The responses to the questionnaire were subjected to recommended tests of reliability and validity.89 These are discussed in detail below. Internal consistency is the extent to which items within a dimension are correlated with each other. It can be examined by several methods: item to own dimension correlations calculated after correction for overlap; Cronbach's a, a widely used method based on correlations between items; and reliability coefficients for each dimension calculated by two way analysis of variance.'0 We used non-parametric versions of these tests to avoid any distributional assumption. Test-retest reliability-a correlation coefficient measures the degree of association between the test and retest scores but does not indicate the direction of this association. For example, if everyone consistently scored lower on the retest, the correlation coefficient would be highly positive. To overcome this, Bland and Altman recommended a technique which examines the distribution of differences in scores." The differences are plotted, an overall mean and variance of differences calculated, and 95% confidence intervals constructed around the mean by assuming a normal distribution. The test and retest scores are assumed to be from the same distribution when the differences have a mean of zero and 95% of the differences lie within the 95% confidence limits. Validity-The validity of a health measure is conceptually difficult to prove without a standard. One method is to examine construct validity, where hypotheses or constructs concerning the expected distribution of health between groups are examined by the measure being validated.89 For example, women, older people, and people in social classes IV and V might be expected to perceive relatively poorer health; people making use of health services might also be expected to have poorer perceived health than nonusers. We used Kruskal-Wallis one way analysis of variance to test whether the SF-36 scores differed significantly among these groups. The convergent and discriminant validity of SF-36 was examined by the multitrait multimethod matrix. 12 For convergent validity, the correlation between comparable dimensions on SF-36 and Nottingham health profile-for example, between physical functioning and physical mobility-should be higher than the correlations between less comparable dimensions-for example, physical functioning and social isolation. We tested discriminant validity by comparing item to own scale correlation with item to other scale correlation. The item to own scale correlation should be higher if the categories within the SF-36 questionnaire are valid. Discriminatory power-the ability of an instrument to discriminate between different levels of ill health is strictly a form of validity testing. We considered it separately because it is a key criterion for any measure of general health in a population. Discriminatory power is indicated by the frequency distributions of scores obtained from the measures, with a less skewed distribution indicating greater discriminatory power. A highly skewed distribution of scores requires use of a binary outcome whereas a wider range of scores enables detection of intermediate health states. However, it should be confirmed that greater discriminatory power is genuine and correctly identifies ill health. Results We received completed questionnaires from 1582 of the 1980 patients surveyed, of whom 77 could not be contacted, thus giving a response rate of 83%. Of the 250 patients sent a repeat test, 187 (75%) responded. The proportions of missing data from each dimension were lower (0 5%-4%) for the SF-36 questionnaire than for the Nottingham health profile (4-7%). Because so few data were missing for the SF-36 dimensions and the study sample was large, we did not substitute for missing data. The extent of missing data was significantly associated (p<0-001) with increasing age in three of the eight SF-36 dimensions (pain, role limitations due to physical problems, and role limitations due to emotional problems). CHARACTERISTICS OF SAMPLE The sociodemographic characteristics and use of health services of the respondents did not differ from those found in the general household survey (1988) for the same age range, except for socioeconomic class, where the study sample included fewer people in class II but more in class III and more employed women. Too few patients from ethnic minorities were available to permit separate analyses. Non-respondents in the main survey (n=297) were significantly more likely to be male and younger in age and less likely to have visited their general practitioner recently (p<0 005). INTERNAL CONSISTENCY Internal consistency was acceptable. The item to own dimension correlations, after correction for overlap, exceeded 0 5 for all except three of the 33 items. Cronbach's a exceeded the recommended BMJ VOLUME JULY

26 minimum of 0859 and the reliability coefficients were greater than 0 75 for all dimensions except social functioning (a=0-73, reliability=0 74) (table II). The results for social functioning partly reflect the low number of items (two) in that dimension. TEST-RETEST RELIABILITY AT TWO WEEKS The re-test scores were highly correlated with those from the main survey (table II). In the analysis recommended by Bland and Altman" the mean of the differences was significantly different from zero for six dimensions but did not exceed one point on the 100 point scale, making it clinically insignificant (table II). For all dimensions 91-98% of cases lay within the 95% confidence interval constructed for a normal distribution. TABLE ii-reliability ofsf-36 questionnaire in general practice population Internal consistency Test-retest reliability (2 week interval) % Of cases lying Reliability Mean within 95% Dimension Cronbach's a coefficients Correlation difference confidence interval Physical functioning Social functioning Role limitations (physical problems) * 98 Role limitations (emotional problems) * 97 Pain * 95 Mental health * 91 Vitality * 9 General health perception * 9 *Significantly different from zero at 5% level. VALIDITY Table III shows the distributions of SF-36 scores by sex, age, social class, and use of health services and for patients with chronic disease. The distribution of scores conformed to what might be expected, thus providing evidence of construct validity. Men perceived themselves to be significantly healthier than women (p<0001), except on the general health dimension. Significant age gradients were found for physical functioning and pain (p<0 001), but little or no gradient was found for mental health (p=0 585). Health decreased with lower social class across all dimensions (p<005) except for general health perception. Those patients who had consulted a general practitioner in the previous two weeks had poorer perceived health than those who had not consulted recently. Seventy seven patients for whom the general practitioner had diagnosed one or more chronic physical problems perceived their health as worse on all dimensions (p<0001), except mental health, than a sample of patients without chronic physical problems matched for age, sex, and general practice (p<005). The expected relations for convergent and discriminant validity were mostly satisfied (table IV). Correlation coefficients for four comparable dimensions of the SF-36 questionnaire and Nottingham health profile were higher than correlations between non-comparable dimensions. This was not found for the correlation of social functioning with social TABLE III-Mean scores on dimensions ofsf-36 questionnaire in relation to sociodemographic variables and use ofhealth services Role Role Physical Social limitation limitation Mental General health Variable n* functioning functioning (physical) (emotional) Pain health Vitality perception Age (years): Sex: Male Female Socioeconomic class: I II III non-manual III manual IV V Students Chronic physical problems: Yes No General practitioner consultation in previous 2 weeks: Yes No Outpatient attendance in previous 3 months: Yes No *n Is the minimum number of respondents completing one dimension. The number of respondents varied for each dimension. TABLE IV-Multitrait multimethod matrix of correlation coefficients for SF-36 questionnaire and Nottingham health profile SF-36 Nottingham health profile Physical Social Mental Physical Social Emotional functioning functioning Pain health Vitality mobility isolation Pain reactions Energy SF-36: Physical functioning 0.93* Social functioning * Pain * Mental health * Vitality * Nottingham health profile: Physical morbidity -0-52t * Socialisolation t * Pain St * Emotional reactions t * Energy t * *Reliability coefficient. i-correlation coefficients are negative because the two scales run in the opposite direction. 162 BMJ VOLUME JULY 1992

27 isolation, where the constituent questions seemed to address different aspects of social well being. DISCRIMINATORY POWER Comparison of the frequency distribution of SF-36 scores and scores on the comparable dimensions of the Nottingham questionnaire (figs 1 and 2) showed that the SF-36 scores were less skewed. The median scores for all Nottingham health profile dimensions were zero (good health) but were less than 100 (poorer health) on five of the eight dimensions of the SF-36. Table V shows the patients who scored zero on the Nottingham questionnaire (good health) divided according to those who scored 100 (good health) and those who scored less than 100 (poorer health) on the SF-36 questionnaire (table V). The poorer health group had a higher proportion of women, had an older 40- Physical functioning I 00 - Physical mobility Soilfntoig Soilislto = Sol functioning s 00 Social isolation 60 (physica ~~~~~ EEI ~ O Role limitations 60 -(physical) Ojm - -mm FIG 1 -Frequency distrtbution ofscores on SF-36 dimensions (left side) and comparable dimensions on the Nottingham health profile (right side): functional status 50- Vitality 80- Energy Mental health ~~50-I Emotional reactions 0- ii ' General health 20 - perception S FIG 2-Frequency distribution ofscores on SF-36 dimensions (left side) and comparable dimensions on the Nottingham health profile (right side): well being and overall health mean age, and contained a higher percentage of patients not in full time employment than the good health group. Patients in the poorer health group were more likely to have consulted a general practitioner or used outpatient services. These results were significant for physical functioning, social functioning, and pain (p<005). The numbers of patients scoring 100 in the remaining two comparable dimensions (mental health and vitality) were too few for significance to be shown. Discussion In attempting to be comprehensive, existing general health questionnaires such as the sickness impact profile may be too long or require interviews, or both.' In primary care or community settings the contact time with patients is often short, and thus to be practical and TABLE v -Analysis of results for patients scoring zero on Nottingham health profile: comparison ofthose in good health (SF-36= 100) with those scoring in poorer health (SF-36< 100) in relation to sociodemographic characteristics and use ofhealth services % Visiting general % Attending No of Mean age Sex (% % Not full time practitioner in outpatients in % Inpatients in Dimension score patients (years) female) employed previous 2 weeks previous 3 months past year Physical functioning: < *** 57-1* 51 7*** 21* 13* 10 Social functioning: < *** 54.5** 29*** 19*** 13 Pain: < * 58.3** 49.2** 23*** 14*** 10 Mhental health: < Vitality: < * 50.1* *p<0.05, **p<0.01, ***p<0.001, by yx test except for age (by Mann-Whitney U test) BMJ VOLUME JULY

28 acceptable to the population the questionnaire must be brief, easy to use, and preferably self administered. These features are also important for researchers, who may want to add a generic health measure to a disease specific questionnaire. The SF-36 questionnaire seemed to meet these criteria, taking just five minutes to complete. We achieved a response rate of 83%, and despite its presentation being more complex than that of the Nottingham questionnaire there were fewer missing data. This quantitative evidence, and the favourable impression for face to face interviews, suggests that the SF-36 questionnaire is an acceptable measure of the health of a general population. Our findings supported the developers' claims of internal consistency for the SF-36 questionnaire.4 The test-retest reliability of the SF-36 questionnaire has not been examined before, and since an instrument with a high discriminatory power may be unreliable' it was reassuring to find that test-retest reliability was excellent. The maximum mean difference in dimension scores was 0-80, which implies that a person with a test score of 70 might score 71 on retesting. This difference is of no practical significance. The evidence for the construct validity of the SF-36 was substantial. The expected distribution of scores was observed by sociodemographic characteristics, general practitioner consultation, use of hospital services, and a group of patients with chronic physical problems. COMPARISON WITH NOTTINGHAM QUESTIONNAIRE In Britain many researchers,' and more recently the NHS,'3 have used the Nottingham health profile to study aspects of health including rheumatoid arthritis,'4 migraine,14 hypertension,'5 heart transplantation,'6 renal lithotripsy,'7 and cholecystectomy."' It has also been successfully applied in other countries.'920 The questionnaire takes just a few minutes to complete and is acceptable to the general population.7 However, it has been criticised for tapping the extreme end of ill health and therefore being unsuitable for examining improvements in health in a general population.' ' Our results strongly support this criticism-most of the general population sampled registered a zero score on the Nottingham dimensions, producing highly skewed distributions. The distributions of SF-36 scores were less skewed and showed a substantially higher prevalence of perceived health problems, particularly with regard to mental health and vitality. By dividing patients who scored zero (good health) on the Nottingham profile into those who scored 100 (good health) or less than 100 (poorer health) on the SF-36 questionnaire we were able to identify people with perceived health problems who were missed by the Nottingham profile. The SF-36 questionnaire therefore seems preferable to the Nottingham profile for measuring the health of a population with relatively minor conditions, such as in general practice or the community. APPLICABILITY The King's Fund is supporting several validation studies looking at different patient groups to determine whether the questionnaire is suitable for studying specific groups as well as the general population. Indications from unpublished work in the United States suggest that the SF-36 questionnaire could be used to study a wide range of serious conditions. However, the higher level of missing data for the year old age group in our study suggests that further research is required before it is widely applied to elderly patients. Measures such as the SF-36, which produce a profile of scores, can be criticised as unsuitable for comparisons between treatments that may improve the dimension scores differentially. For this purpose a single index of health is preferable and it is not yet known whether SF-36 scores can be used to generate a valid single index. Existing measures which purport to provide single indices, such as the York quality of life measure, have also yet to be validated.22 We thank our colleagues in the Department of General Practice, Dr John Poyser, and Dr Helen Joesbury. The study was supported by a grant from the Medical Research Council. The Medical Care Research Unit is funded by the Department of Health and Trent Regional Health Authority. The opinions in this article are those of the authors. I Wilkin D, Hallam L, Doggett MA. Measures of need and outcome for primarv health care. Oxford: Oxford Medical Press, Bowling A. Measuring health: a reviezv of quality of life measurement scales. Milton Keynes: Open University Press, Ware JE, Brook RH, Williams KN, Stewart AL, Davies-Avery A. Conceptualisation and measurement of health for adults in the health insurance study. Vol 1. Model of health and methodology. Santa Monica, California: Rand Corporation, (Publication No R-1987/1-HEW.) 4 Ware John E, Sherbourne CD. The SF-36 short-form health status survey. 1. Conceptual framework and item selection. Med Care (in press). 5 Stewart AL, Hays RD, Ware JE. The MOS short form general health survey. Med Care, 1988;26: Anderson JStC, Sullivan F, Usherwood TP. The medical outcomes study instrument (MOSI)-use of a new health status measure in Britain. Fam Pract 1990;7: Hunt S, McKenna SP, McEwen J. The Nottingham health profile user's manual. Manchester: Galen Research and Consultancy, Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. Oxford: Oxford University Press, McDowell I, Newell C. Measuring health: a guide to rating scales and questionnaires. New York: Oxford University Press, Kerlinger FN. Foundations ofbehavioural research. New York: Holt, Rinehart, and Winston, Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;i: Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull 1959;56: Final report: the CASPElFreeman outcome study. London: CASPE Research, Jenkinson C, Fitzpatrick R. The Nottingham health profile: an analysis of its sensitivity in differentiating illness groups. Soc Sci Med 1988;27: De Lame PA, Droussin AM, Thomson M, Verhaest L, Wallace S. The effects of endopril on hypertension and quality of life. A large multi-center study in Belgium. Acta Cardiologica, 1989;44: O'Brien BJ, Banner NR, Gibson S, Yacoub M. The Nottingham health profile as a measure of quality of life following combined heart and lung transplantation. J Epideomiol Community Health 1988;42: May N, Petruckevitch A, Snowdon C. Patients quality of life following extracorporeal shock wave lithotripsy and percutaneous nepholithotomy for renal calculi. Inlz Technol Assess Health Care 1990;6: Milner PC, Nicholl JP, Westlake L, Williams BT, Birch S, Ross B, et al. TFhe evaluation of lithotripsy as a treatment for gallstones: a randomised controlled trial approach in England. J3ournal oflithotripsy and Stone Disease 1989;1: Wiklund I, Romanus B, Hunt SM. Reliability of the Swedish version of the Nottingham health profile. Int Disabil Stud 1988;10: Baum FE, Cooke RD. Community-health needs assessment: use of the Nottingham health profile in an Australian study. Med J Aust 1989;150: Kind P, Carr-Hill R. The Nottingham health profile: a useful tool for epidemiologists? Soc Sci Med 1987;25: Carr-Hill R, Morris J. Current practice in obtaining the "Q" in QALYs: a cautionary note. BM3 1991;303: (Accepted 163June 1992) 164 BMJ VOLUME JULY 1992