Hospital admissions account for a large proportion of

Transcription

1 A Systematic Review of the Probability of Repeated Admission Score in Community-Dwelling Adults Emma Wallace, MB,* Tim Hinchey, MB,* Borislav D. Dimitrov, PhD,* Kathleen Bennett, PhD,* Tom Fahey, MD,* and Susan M. Smith, MD* OBJECTIVES: To perform a systematic review of the Probability of Repeated Admission (Pra) score in community-dwelling adults to assess its performance in a range of validation studies in the community setting. DESIGN: Systematic review and meta-analysis. SETTING: Primary and community care. PARTICIPANTS: Community-dwelling older people. MEASUREMENTS: The primary outcome was hospital admission; secondary outcomes were mortality, hospital days, functional decline, other health service use, and costs. RESULTS: Nine validation studies describing 11 cohorts of individuals aged 65 and older were identified. A metaanalysis of the Pra score in five cohorts (8,843 individuals) with comparable and available data revealed good discrimination performance (summary area under the receiver operating characteristic curve 69.7% (standard error 2.8%)). Pooled specificity was high (96%, 95% confidence interval (CI)= %), indicating that a Pra score of 0.5 or greater effectively rules in the likelihood of admission, but pooled sensitivity was low (12%, 95% CI= %). Calibration performance was good, with an overall risk ratio of 1.12 (95% CI= ), indicating that the Pra score reliably predicted hospital admissions. CONCLUSION: The Pra score performs well in predicting hospital admission in community-dwelling adults categorized as high risk according to the score. This tool has clinical and healthcare policy utility in terms of targeting elderly people at highest risk of hospital admission, but the low pooled sensitivity of the score From the *Department of General Practice, HRB Centre for Primary Care Research, Royal College of Surgeons in Ireland Medical School, Dublin, Ireland; Academic Unit of Primary Care and Population Sciences, University of Southampton, Southampton, UK; and Department of Pharmacology and Therapeutics, Trinity Centre for Health Sciences, St James Hospital, Trinity College Dublin, Dublin, Ireland. Address correspondence to Dr. Emma Wallace, HRB Centre for Primary Care Research, Department of General Practice, Royal College of Surgeons in Ireland Medical School, 123 St Stephens Green, Dublin 2, Ireland. emmawallace@rcsi.ie DOI: /jgs indicates that it is not a reliable way of excluding hospital admission in those stratified as low risk. J Am Geriatr Soc 61: , Key words: Pra; risk stratification; risk score; hospital admission; questionnaire Hospital admissions account for a large proportion of healthcare expenditure and have high personal costs for the affected individuals. 1,2 An expanding elderly population will place capability and financial pressure on health systems. The importance of identifying older people at risk of admission is recognized, and a number of clinical prediction rules or risk scores exist for this purpose. 3 5 There is a widely accepted methodology for the development such risk scores. 6 Derivation is followed by internal and external validation before the effect of use on clinical outcomes is finally tested. These steps require cumulative levels of evidence to answer the relevant research and clinical questions. 7,8 One commonly used risk score for identifying older people at high risk of hospital admission is the Probability of Repeated Admission (Pra) score. 9 The Pra score was originally derived in 1993 in an elderly U.S. community-based population using a self-report postal questionnaire. Screening questions ask about age, sex, presence of diabetes mellitus and ischemic heart disease, admission to hospital in the previous year, previous physician visits, self-rated health, and the availability of an informal caregiver. The score is calculated using a logistic equation that calculates a value of between 0 and 1; a score of 0.5 indicates a 50% likelihood of two or more admissions within the following 4-year period. 9 Since its derivation, the Pra score has subsequently been validated to predict one or more hospital admissions over a 1-year follow-up period and to predict other JAGS 61: , , Copyright the Authors Journal compilation 2013, The American Geriatrics Society /13/$15.00

2 358 WALLACE ET AL. MARCH 2013 VOL. 61, NO. 3 JAGS outcomes such as healthcare utilization and healthcare costs. 10,11 This risk score has been identified as the primary risk indicator for health planners seeking to identify vulnerable older adults at risk of hospital admission who may benefit from targeted interventions. 12 The aim of this systematic review of the validation studies of the Pra risk score was to assess its performance in predicting hospital admission in community-dwelling adults. METHODS Data Sources and Search Strategy The PRISMA guidelines for the conduct and reporting of systematic reviews and meta-analyses were adhered to in the conduct of this systematic review. 13 A computerized literature search was performed including PubMed, Embase, Cinahl, and the Cochrane library databases from January 1990 to April 2012 and limited to publications in English. The databases were searched using a combination of terms, including probability of repeated admission, Pra, Pra score, Pra instrument, and Pra questionnaire. Hand searches of the references of retrieved full-text articles supplemented this search. Study Selection and Data Extraction Criteria for inclusion were prospective or retrospective cohort studies; participants aged 18 and older (although the Pra score was designed for use in older people, this age cutoff was chosen to ensure that all validation studies were captured in the search); Pra score calculated; setting of care, primary care and community-based care; primary outcome of hospital admissions; secondary outcomes of mortality, hospital days, functional decline, other service use, and costs; and published in English. All articles that the electronic search identified were screened according to title and abstract, and those potentially eligible for inclusion were identified. Two researchers (EW, SS) then independently screened full texts of potentially eligible studies. Disagreements were managed according to consensus. Three authors (EW, SS, TH) abstracted data from included studies using a standardized data collection form. All retrieved studies that met the inclusion criteria were included in the narrative systematic review. Meta-analysis was conducted on included studies that used a Pra score of 0.5 or greater to denote high risk (according to the original derivation study), reported hospital admissions at 1-year of follow-up, and had relevant data available. Methodological Quality Assessment Two researchers (TH and SS) independently assessed quality of studies included in the meta-analysis following the modified methodological standards of McGinn for validation studies of clinical prediction rules. 6 The McGinn criteria examine the internal and external validity of studies in terms of blinded assessment of predictor variables and outcome (hospital admission vs no hospital admission), loss to follow-up, method of participant selection, and spectrum of participants included. Data Synthesis and Analysis for Studies Included in the Meta-Analysis Discrimination First, the discrimination performance of the Pra score was assessed by performing a pooled receiver operating characteristic (ROC) analysis, fitting a symmetrical, summary ROC curve using the DerSimonian-Laird random-effects model and estimating the area under the curve (AUC) and its standard error and 95% confidence interval (CI). 14 This model was used to derive summary estimates of sensitivity and specificity and their corresponding 95% CIs. To further confirm these results, a bivariate random-effects model was used to construct and analyze a hierarchical summary ROC (HSROC) curve with its 95% confidence and prediction regions. 15 This approach preserves the two-dimensional nature of the original data and takes into account study size and heterogeneity. 16 The bivariate model also estimates and incorporates the negative correlation that may arise between the sensitivity and specificity as a result of differences in reference standards used in different studies. Individual and summary estimates of sensitivity and specificity are plotted in a HSROC graph, plotting sensitivity (true positive) on the y-axis against 1 specificity (false negative) on the x-axis. The 95% confidence and prediction regions around the pooled estimates are also plotted to illustrate the precision with which the pooled values were estimated (confidence ellipse around the mean value) and to illustrate the amount of betweenstudy variation (prediction ellipse). Heterogeneity was assessed visually using the summary ROC plots and statistically using the variance of logit-transformed sensitivity and specificity, with smaller values indicating less heterogeneity among studies. Calibration Second, a pooled analysis of the calibration performance of the Pra score was performed by deriving risk ratios (RRs) of predicted versus observed (p/o) hospital admissions in each of the five eligible cohorts, constructing a forest plot of RRs and deriving a pooled estimate (RR p/ o) with 95% CIs, and determining heterogeneity (Rev- Man 5, Cochrane Collaboration). The predicted admissions were obtained using the admission frequency distribution from the original derivation cohort. 9 The proportionate admissions estimate from the original derivation study of the Pra score was applied according to the two risk strata: low risk (<0.5) and high risk ( 0.5). RR p/o less than 1 indicates underprediction of admissions by the Pra score (the observed number of admission is greater than the predicted number) and RR p/o greater than 1 indicates an overprediction (the observed number of admissions is less than the predicted number). A RR p/o of 1 indicates a perfect calibration between observed and predicted admissions. The RRs p/o with 95% CIs were calculated using the random-effects Mantel Haenszel model, and heterogeneity was assessed in percentage using the I 2 statistic. (I 2 < 50% was assumed to be low heterogeneity.)

3 JAGS MARCH 2013 VOL. 61, NO. 3 SYSTEMATIC REVIEW OF THE PRA SCORE 359 RESULTS Included Studies A flow diagram of the search strategy is presented in Figure 1. The literature search retrieved 4,484 titles, and 18 potentially eligible articles were reviewed in full, leading to the identification of nine validation studies describing 11 cohorts of participants. 9 12,17 23 Meta-analysis was performed on the cohort of participants from studies that used a score of 0.5 or greater to indicate a high risk of subsequent hospital admission, predicting hospital admissions at 1-year follow-up and for whom relevant data was available. Five cohorts of participants from three studies (n = 8,843) met these inclusion criteria and were included in the meta-analysis. Study Description Details of included studies are presented in Table 1. Nine studies were conducted in North America and one in South America, and one examined three European cohorts. 22 The study cohorts ranged in size from 306 to 17,469 participants and included a total of 36,140 participants. Participant characteristics varied across all cohorts and are described in Table 2. As expected, the vast majority of studies focused on elderly adults, with all studies except one using aged 65 and older or aged 70 and older as inclusion criteria. There was variation in the reporting of the proportion of high risk participants in different studies (4 25%). This variation was in part due to the different Pra cutoff scores used to define high-risk groups, with some studies using cutoffs of 0.3 or 0.4. Of 11 validation cohorts, seven used the original cut-off point of 0.5. There was also significant variation in other participant characteristics such as previous admissions and self-rated health. All studies considered hospital admissions as their primary outcome, although this varied in how it was reported. The original derivation study reported the risk of two or more admissions over a 4-year follow-up period, but subsequent studies have used 6-month and 1-year follow-up periods. Secondary outcomes included mortality, hospital days, functional decline, other service use, and costs. Across all studies, participants defined as high risk had a higher rate of subsequent hospital admission, as predicted. Two of the validation studies examined the use of administrative databases to calculate Pra score where possible; the response rates for studies using postal questionnaires to determine Pra score ranged from 40 70%. 12,20 Study Methodological Quality Overall, external validity was good, with any potential bias in participant selection resulting from the response rates to the original mailed surveys; these are described in Table 1. The main shortcomings in relation to internal validity related to blinding, and no study specifically reported whether the outcome assessors were blinded to Records identified through database searching (n = 6,192) Records after duplicates removed (n = 4,484) Duplicates = 1,708 Records screened (n = 4,484) Records excluded (n = 4,466) References of references (n =1) Full-text articles assessed for eligibility (n = 18) Studies included in narrative synthesis (n: 8+1= 9) Studies included in metaanalysis (n = 3) (n = 5 cohorts*) *1 study presented data for 3 cohorts Full-text articles excluded, with reasons (n = 10): Hospitalized/Residential care: 5 No original data: 3 Pra to risk stratify (intervention): 1 Cohort already included: 1 Excluded from meta-analysis, with reasons: (n = 6) Different Pra score cutoff used: 3 Adequate data not available: 2 Follow up of 6 months: 1 Figure 1. PRISMA Flow diagram records identified through database searching (n = 6,192). Pra = probability of repeated admission.

4 360 WALLACE ET AL. MARCH 2013 VOL. 61, NO. 3 JAGS Table 1. Characteristics of Included Studies Study Reference Country Participants and Design Outcome Secondary Outcomes Notes 9 United States 5,876 from multistage probability sample of all noninstitutionalized U.S. citizens aged 70 Split sample with second half used to validate score derived from first half 17 a United States 306 of all local low-income individuals (Medicaid enrollees) in three counties in Minnesota; 61% response rate Number with 2 admissions in 4 years Admissions in 1 year (mean number per person-year survived) Mortality, hospital days, hospital costs Mortality, hospital days, pharmacy payments, total payments Derivation study Data from Longitudinal Study on Ageing Pilot study to assess ability of Pra score to identify individuals suitable for RCT intervention 10 a United States Same population; additional secondary outcomes reported 12 United States 2,174 of 5,240 randomly selected enrollees aged 65 of a HMO in Washington state 18 United States 386 enrollees aged 65 in Medicare managed-risk health plan at Pennsylvania State University 19 a United States 1,783 of 4,506 new enrollees in United Heath Group s Secure Horizons Medicare Advantage Plan in Alabama, Florida, or Ohio; response rate 45% 11 United States 6,802 enrollees aged 65 in Medicare TEFRA risk plan (PacifiCare s Secure Horizons), California 20 United States 17,469 of 24,947 enrollees in Medicare Plan in Pennsylvania State University; mean age 71.2; 70% response rate 21 United States 296 of 558 enrollees aged 65 in Medicare HMO in Philadelphia, also Medicaid eligible; response rate 54% 22 a Three cohorts in Germany, United Kingdom, and Switzerland 6,924 community-dwelling people aged 65 from initial sample of 18,932 potentially eligible individuals identified by general practitioners; only those with follow-up data included 26 Brazil 515 of 551 enrollees non-institutionalized adults 60 years Number with 2 admissions in 4 years Functional decline at 4 years Comparison of self-report with administrative data Admissions in 1 year Compares Pra score with Level II Nutrition screen Pra > 0.3 considered high risk Admissions in 1 year Compares Pra score with Hierarchical Condition Model Admissions in 1 year Claims, emergency department visits, nursing home admissions, home visits Admissions in 1 year (mean) Admissions in 6 months (sensitivity and specificity for Pra cut-points of 0.5, 0.45, and 0.4) Mean number of hospital days, claims Total claims over 6-month period Admissions in 1 year 6 physician visits in 1 year Reported variation in usage between different primary care sites within the plan Compared Pra score with database-derived score used as administrative proxy Data from PRevention in Older people Assessment in GEneralists practices Trial Admissions in 6 months Pra >0.5 high risk a Cohorts included in meta-analysis, Figure 3. Pra = probability of repeated admission; HMO = health maintenance organization; RCT = randomized controlled trial.

5 JAGS MARCH 2013 VOL. 61, NO. 3 SYSTEMATIC REVIEW OF THE PRA SCORE 361 Table 2. Comparison of Study Samples Study High Risk of Being Readmitted, n (%) Probability of Repeated Admission Score Male (%) Age Self-Rated Health Fair or Poor (%) 1 Admissions in Previous Year > 6 Physician Visits in Previous Year 9, derivation 206 (8) > % > , (21) > % > NR (14) > % > (>3 visits) 18 NR 47 3% > (5) >.3 NR NR (Medicare) NR NR NR 11 1,716 (25) > % > Described as younger healthier cohort overall (5) >.5 48 Mean age (25) >.4 23 Mean age 71 NR , German 132 (7) > % > , UK 94 (4) > % > , Swiss 137 (5) > % > (7) > % NR = not reported. the original Pra score of participants, although the outcomes were generally collected from automated data sets such as Medicare claims databases, so the quality of outcome assessment depended in part on the availability of claims data. Discrimination A meta-analysis of the Pra score conducted for three studies that included five cohorts (8,843 participants) indicated that discrimination performance was high, with the Pra score clearly distinguishing between those who had been admitted to the hospital from those who had not (summary AUC 69.7%, standard error 2.8%) (Figure 2). The pooled sensitivity was low (12%, 95% confidence interval (CI) = %), but the pooled specificity was high (96%, 95% CI = %). The variance of logit transformed was 0.25 (95% CI = ) for sensitivity and 0.33 (95% CI = ) for specificity, and the 95% prediction regions (amount of variation between studies) on the HRSOC graph were wide, indicating some heterogeneity between studies (Appendix 1). Calibration Further analysis undertaken to assess the calibration performance of the Pra score by estimating the pooled risk ratio (RR p/o ) indicated that the Pra score reliably predicted admissions (RR p/o =1.12 (95% CI = ) (Figure 3). Secondary Outcomes Secondary outcomes were difficult to compare across studies but indicated that the Pra score can also be used to predict which individuals will have higher future healthcare utilization and healthcare costs. Details of secondary outcome measures are outlined in Table 2. In two of the studies included for the narrative synthesis, the main focus was a comparison of results derived from the traditional postal questionnaire with those derived using administrative or chart data alone. One of these studies validated the Pra score excluding the availability of an informal caregiver question and substituting the self-rated health status question for a chronic disease score, because this information was not available from administrative data. The other study also omitted the availability of a formal caregiver question and substituted the Deyo-Charlson comorbidity index for the self-rated health question. These studies indicated that administrative data were as reliable as postal questionnaires in predicting future hospital admissions and service use, with the advantage of better coverage of potentially eligible participants. DISCUSSION Statement of Principal Findings Participants stratified as high risk according to Pra score have a high probability of being admitted to the hospital in the following year (pooled specificity = 96%, 95% CI = %), indicating that a Pra score of 0.5 or greater effectively rules in the likelihood of admission and that this risk score is a reliable predictor of future hospital admission. Current Context and Future Research Implications Recent systematic reviews have indicated that risk prediction models to predict 30-day rehospitalization perform poorly and that no single intervention has been regularly found to effectively reduce the risk of such readmissions. 24,25 It could be argued that a shift in focus to targeting community-dwelling elderly people at high risk of hospital admission or healthcare utilization who could then benefit from a community-based intervention may be more effective. The Pra score has been used in this context to recruit elderly people within a randomized controlled trial (RCT) of interventions for those at high risk of functional decline. 26 Participants who

6 362 WALLACE ET AL. MARCH 2013 VOL. 61, NO. 3 JAGS Sensitivity=True positive rate 1-Specificity=False positive rate Summary ROC=Area under the Curve (SE) = (0.0280)% Figure 2. Summary receiver operating characteristic (SROC) curve. Sensitivity = true positive rate, 1 specificity = false positive rate. SROC area under the curve (AUC) % (standard error (SE) %). Study or Subgroup Boult 95 Mosley Wagner Ger Wagner Swiss Wagner UK Predicted Observed Risk Ratio Risk Ratio Events Total Events Total Weight M-H, Random, 95% CI M-H, Random, 95% CI ,783 1,820 2,663 2, ,783 1,820 2,663 2, % 21.1% 21.5% 21.9% 21.5% 0.70 [0.47, 1.02] 1.45 [1.27, 1.66] 0.94 [0.84, 1.06] 1.03 [0.93, 1.13] 1.55 [1.38, 1.75] Total (95% CI) 8,843 8, % Total events 2,117 1,800 Heterogeneity: Tau² = 0.06; Chi² = 60.96, df = 4 (P < ); I² = 93% Test for overall effect: Z = 0.98 (P = 0.33) 1.12 [0.89, 1.42] Underprediction Overprediction Figure 3. Meta-analysis of predicted risk versus observed hospital admissions. completed the Pra questionnaire were eligible to participate in the study, which involved a complex intervention including training of health professionals, administration of the Health Risk Appraisal for Older persons questionnaire, group education sessions for participants, additional personal reinforcement, two home visits by a specially trained nurse with detailed feedback and discussion with the geriatric team, and written feedback to the participant s general practitioner. This intervention had consistently moderately favorable effects on primary outcomes of preventative care use (uptake of vaccinations) and health behavior (physical activity and diet). 27 The Pra has also been used in a similar context to identify high-risk individuals for inclusion in a RCT to measure the effects of an outpatient geriatric evaluation and management plan. 28 This review indicates that it may be reasonable to use administrative or chart data to gather the information required for calculating the Pra score, which could offer advantages in terms of practicality and have higher response rates and less response bias than traditional questionnaires, thus augmenting the potential of the score in healthcare planning. Nevertheless, administrative or chart data may not be complete or available, so a combination of self-reporting with the use of administrative data may produce the most-complete data. 12 Another interesting development is the increasing recognition of the effect of multiple chronic medical conditions, or multimorbidity, and polypharmacy on high healthcare utilization and hospital admissions Two validation studies of the Pra score using administrative databases rather than the traditional self-report question-

7 JAGS MARCH 2013 VOL. 61, NO. 3 SYSTEMATIC REVIEW OF THE PRA SCORE 363 naire have substituted self-rated health status for a chronic disease score and the Deyo-Charlson comorbidity index. 9,10 Future studies using this approach will need to consider how best to define, measure, and account for multimorbidity and polypharmacy. Since its derivation in 1993, there have been few large scale studies validating the Pra score and only two studies outside North America. This is surprising considering the simplicity of administering this questionnaire and the real potential this risk score offers in terms of targeting high-risk elderly adults for community-based interventions and management. Other risk scores developed for predicting the risk of admissions exist but are at earlier stages of development. Examples include the Predicting Emergency Admissions Over the Next Year (PEONY) score, based on a U.K. population-derived algorithm but directed to younger populations (aged 40 65), and the Emergency Admission Risk Likelihood Index (EARLI), which, similar to the Pra score, uses a short postal questionnaire, although the questions focus more on current physical functioning and memory loss. 4,5 EARLI has been developed in people aged 75 and older and has yet to be validated in younger individuals or in other settings. Furthermore, an algorithm for identifying elderly people at high risk of hospital admission in the following year for use in U.K. primary care has recently been developed with the aim of identifying individuals who would benefit from an intervention to reduce their risk of subsequent hospital admission. 3 Limitations There are also some limitations to this review. There have been a limited number of validation studies performed on the Pra score, and many of these were performed more than 10 years ago. This has made it difficult to obtain additional data from authors and limited the number of studies that could be included in the meta-analysis. In addition, authors have used different cutoff points for the Pra score to define high risk groups and different follow-up periods. There is variation across the included studies in terms of participant characteristics, but this is to be expected, and it could be argued that such variation adds to the external validity of the findings. CONCLUSION This systematic review suggests that the Pra score is a risk score that is useful when trying to identify older people living in the community who are at high risk of hospital admission. The high pooled specificity of the score indicates that those whom the score stratifies as high risk have a high likelihood of being admitted to the hospital in the following year, although the low pooled sensitivity indicates that admission cannot be confidently excluded for participants stratified as low risk. Despite the potential of the Pra score, there are a limited number of validation studies. Further studies are warranted in populations of individuals with broad risk of readmission to better determine the validity of the Pra risk score in different populations and enhance its generalizability. Authors of future validation studies will also need to decide whether to use the traditional approach of collecting data using a postal questionnaire, administrative data, or a combination of both approaches. ACKNOWLEDGMENTS We would like to acknowledge the contribution of Mr. Paul Murphy, information specialist, who assisted with the search strategy for the review. This work is supported by Health Research Board of Ireland through the HRB Centre for Primary Care Research under Grant HRC/2007/1. Conflict of Interest: The authors declare no financial or personal conflicts of interest. Author Contributions: Emma Wallace and Susan M. Smith: protocol development, literature search, data extraction, data analysis, manuscript preparation. Tim Hinchey: protocol development, literature search, data extraction, manuscript preparation. Borislav Dimitrov: protocol development, data extraction, data analysis, manuscript preparation. Kathleen Bennett and Tom Fahey: protocol development, data analysis, manuscript preparation. Sponsor s Role: None. REFERENCES 1. Goldfield N. Strategies to decrease the rate of preventable readmission to hospital. Can Med Assoc J 2010;182: Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med 2009;360: Billings J, Dixon J, Mijanovich T et al. Case finding for patients at risk of readmission to hospital: Development of algorithm to identify high risk patients. BMJ 2006;333: Donnan PT, Dorward DWT, Mutch B et al. Development and validation of a model for Predicting Emergency Admissions Over the Next Year (PEONY): A UK historical cohort study. Arch Intern Med 2008;168: Lyon D, Lancaster GA, Taylor S et al. Predicting the likelihood of emergency admission to hospital of older people: Development and validation of the Emergency Admission Risk Likelihood Index (EARLI). Fam Pract 2007;24: McGinn TG, Guyatt GH, Wyer PC et al. Users guides to the medical literature: XXII: How to use articles about clinical decision rules Evidence- Based Medicine Working Group. JAMA 2000;284: Falk G, Fahey T. Clinical prediction rules. BMJ 2009;339:b Wallace E, Smith SM, Perera-Salazar R et al. Framework for the impact analysis and implementation of Clinical Prediction Rules (CPRs). BMC Med Inform Decis Mak 2011;11: Boult C, Dowd B, McCaffrey D et al. Screening elders for risk of hospital admission. J Am Geriatr Soc 1993;41: Pacala JT, Boult C, Boult L. Predictive validity of a questionnaire that identifies older persons at risk for hospital admission. J Am Geriatr Soc 1995;43: Pacala JT, Boult C, Reed RL et al. Predictive validity of the Pra instrument among older recipients of managed care. J Am Geriatr Soc 1997;45: Coleman EA, Wagner EH, Grothaus LC et al. Predicting hospitalization and functional decline in older health plan enrollees: Are administrative data as accurate as self-report? J Am Geriatr Soc 1998;46: Moher D, Liberati A, Tetzlaff J et al. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. BMJ 2009;339: b Zamora J, Abraira V, Muriel A et al. Meta-Disc: A software for meta-analysis of test accuracy data. BMC Med Res Methodol 2006;12: Harbord RM, Whiting P. Metandi: Meta-analysis of diagnostic accuracy using hierarchical logistic regression. Stata J 2009;9: Reitsma JB, Glas AS, Rutjes AW et al. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005;58: Boult C, Pacala JT, Boult LB. Targeting elders for geriatric evaluation and management: Reliability, validity, and practicality of a questionnaire. Aging (Milano) 1995;7:

8 364 WALLACE ET AL. MARCH 2013 VOL. 61, NO. 3 JAGS 18. Jensen GL, Friedmann JM, Coleman CD et al. Screening for hospitalization and nutritional risks among community-dwelling older persons. Am J Clin Nutr 2001;74: Mosley DG, Peterson E, Martin DC. Do hierarchical condition category model scores predict hospitalization risk in newly enrolled Medicare Advantage participants as well as probability of repeated admission scores? J Am Geriatr Soc 2009;57: Sidorov J, Shull R. My patients are sicker: Using the Pra risk survey for case finding and examining primary care site utilization patterns in a Medicare-risk MCO. Am J Manag Care 2002;8: Vojta CL, Vojta DD, Tenhave TR et al. Risk screening in a Medicare/Medicaid population: Administrative data versus self report. J Gen Intern Med 2001;16: Wagner JT, Bachmann LM, Boult C et al. Predicting the risk of hospital admission in older persons validation of a brief self-administered questionnaire in three European countries. J Am Geriatr Soc 2006;54: Dutra MM, Moriguchi EH, Lampert MA et al. Predictive validity of a questionnaire to identify older adults at risk for hospitalization. Rev Saude Publica 2011;45: Kansagara D, Englander H, Salanitro A et al. Risk prediction models for hospital readmission: A systematic review. JAMA 2011;306: Hansen LO, Young RS, Hinami K et al. Interventions to reduce 30-day rehospitalization: A systematic review. Ann Intern Med 2011;155: Stuck A, Kharicha K, Dapp U et al. The PRO-AGE study: An international randomised controlled study of health risk appraisal for older persons based in general practice. BMC Med Res Methodol 2007;7: Dapp U, Anders JA, von Renteln-Kruse W et al. A randomized trial of effects of health risk appraisal combined with group sessions or home visits on preventive behaviors in older adults. J Gerontol A Biol Sci Med Sci 2011;66A: Boult C, Boult LB, Morishita L et al. A randomized clinical trial of outpatient geriatric evaluation and management. J Am Geriatr Soc 2001;49: Marengoni A, Angleman S, Melis R et al. Aging with multimorbidity: A systematic review of the literature. Ageing Res Rev 2011;10: Glynn LG, Valderas JM, Healy P et al. The prevalence of multimorbidity in primary care and its effect on health care utilization and cost. Fam Pract 2011;28: Hajjar ER, Cafiero AC, Hanlon JT. Polypharmacy in elderly patients. Am J Geriatr Pharmacother 2007;5: APPENDIX 1: HIERARCHICAL SUMMARY RECEIVER OPERATING CHARACTERISTIC CURVE (HRSOC) WITH 95% CONFIDENCE REGION AND 95% PREDICTION REGION FOR STUDIES INCLUDED IN META-ANALYSIS Sensitivity Study estimate HSROC curve 95% prediction region.6.4 Specificity.2 0 Summary point 95% confidence region