A European consensus on outcome measures for psychosocial intervention research in dementia care

Transcription

1 Aging & Mental Health, January 2008; 12(1): A European consensus on outcome measures for psychosocial intervention research in dementia care E. MONIZ-COOK 1, M. VERNOOIJ-DASSEN 2, R. WOODS 3, F. VERHEY 4, R. CHATTAT 5, M. DE VUGT 4, G. MOUNTAIN 6, M. O CONNELL 7, J. HARRISON 1, E. VASSE 2, R. M. DRÖES 8, & M. ORRELL 9 FOR THE INTERDEM* GROUP 1 Institute of Rehabilitation, University of Hull & Humber Mental Health Teaching NHS Trust, Hull, UK, 2 Centre for Quality of Care Research, Radboud University Medical Centre Nijmegen, Nijmegen, The Netherlands, 3 Dementia Services Development Centre, University of Wales, Bangor, UK, 4 Institute of Brain & Behaviour, University of Maastricht, Maastricht, The Netherlands, 5 Department of Psychology, Bologna University, Bologna, Italy, 6 Centre for Health & Social Care, Faculty of Health and Wellbeing, Sheffield Hallam University, Sheffield, UK, 7 Alzheimer Europe, Dublin, Ireland, 8 Department of Psychiatry, University of Amsterdam Institute for Research in Extramural Medicine, VU Medical Centre, Amsterdam, The Netherlands, and 9 Centre for Ageing & Mental Health Sciences, Department of Mental Health Sciences, University College London, London, UK (Received 27 May 2007; accepted 27 July 2007) Abstract Psychosocial intervention makes a vital contribution to dementia care. However, the lack of consensus about which outcome measures to use to evaluate effectiveness prevents meaningful comparisons between different studies and interventions. This study used an iterative collaborative, evidence-based approach to identify the best of currently available outcome measures for European psychosocial intervention research. This included consensus workshops, a web-based pan-european consultation and a systematic literature review and a rigorous evaluation against agreed criteria looking at utility across Europe, feasibility and psychometric properties. For people with dementia the measures covered the domains of quality of life, mood, global function, behaviour and daily living skills. Family carer domains included mood and burden, which incorporated coping with behaviour and quality of life. The only specific staff domain identified was morale, but this included satisfaction and coping with behaviour. In conclusion twenty-two measures across nine domains were recommended in order to improve the comparability of intervention studies in Europe. Areas were identified where improved outcome measures for psychosocial intervention research studies are required. Introduction There is increasing recognition that psychosocial interventions make an essential contribution to dementia care (Bird, Llewellyn, Jones, Korten, & Smithers, in press; Dröes, Meiland, Schmitz, & van Tilburg, 2006a; Graff et al., 2006; Livingston, Johnston, Katona, Paton, & Lyketsos, 2005; Mittelman, William, Haley, Clay, & Roth, 2006; Pinquart & Sorenson, 2006; Sorensen et al., 2002; Spector et al., 2003; Teri, McCurry, Logsdon, & Gibbons, 2005; Woods, Spector, Jones, Orrell, & Davies, 2005) and may be cost effective (Knapp et al., 2006). Psychosocial interventions may target the person with dementia, their family carer or staff and may cover a range of domains such as mood, well-being and behaviour. Additionally, the re-definition of traditional approaches to rehabilitation has recently been applied to dementia care (Marshall, 2005; Mountain, 2005). These developments have led to new demands for instruments to effectively measure these domains. However, there is little consensus on which domains are relevant (and should be measured) or which measures provide the best fit for a given domain and considerable inconsistency in both study designs and the outcome measures that are employed. A survey of the European Alzheimer s Disease Consortium (EADC) highlighted the need for consensus in the use of assessment tools across Europe (Diaz et al., 2005). A European consensus has been achieved for measures of cognition such as the Alzheimer s Disease Assessment Scale (ADAS-Cog), Nurses Observation Scale for Geriatric Patients (NOSGER), Cambridge Cognitive Correspondence: Esme Moniz-Cook, Professor of Clinical Psychology & Ageing, Coltman Street Day Hospital, Coltman Street, Hull, HU3 2SG, UK. Tel: þ /19. Fax: þ E.D.Moniz-Cook@hull.ac.uk ISSN print/issn online/08/ ß 2008 Taylor & Francis DOI: /

2 European consensus on outcome measures 15 Examination-revised (CAMCOG-R) and the Mini Mental State Examination (MMSE) (see Verhey et al., 2003; 2004). Some, such as the ADAS-Cog, are commonly used in clinical trials to evaluate the effects of the anti-dementia drugs. However, even in this well-established domain of cognition-related outcomes, there is debate on whether these harmonised and commonly used tools are meaningful in the clinical situation. Drug studies are perceived to have a natural theoretical framework with measurement of cognition being the primary outcome (Birks, 2006). However, researchers also measure other domains, such as activities of daily living and carer burden, using these as proxy indicators of overall improvement. In contrast, psychosocial interventions are derived from wide-ranging theories and concepts and thus have disparate targets which conceptually address: a person s anxieties about loss of autonomy; relative s or informal carer s personal experience of the consequences of the disease; and those that are concerned with enabling formal carers to provide care to people and their families. As psychosocial interventions become better designed and more effective there is a need for more coherent application and evaluation internationally, thus building a broader and more robust evidence base in dementia care. For this to happen, two conditions must be met. First, clarity is needed on the underlying domains that are addressed by a given intervention (i.e. who and what the intervention is directed at). Second, a European consensus on measures and domains is required in view of the wide range of instruments currently used to measure similar psychosocial domains. INTERDEM is a cross-national collaboration concerned with improving psychosocial intervention studies across Europe. In order to provide guidance for future psychosocial intervention research across Europe we undertook this empirically-based consensus study to: (1) harmonise, using a critical conceptual evaluation, the key domains that are the focus of psychosocial intervention in dementia care; and (2) identify through consultation and searches of the literature, suitable measures for examining the effects of psychosocial intervention on these domains. Methods Design and participants This study used an iterative collaborative, evidencebased approach to identify the best of currently available outcome measures for European psychosocial intervention research. This included consensus workshops, a web-based pan-european consultation and a systematic literature review of peer reviewed publications between June 2005 and February 2007 and a rigorous evaluation against agreed criteria that focussed on utility across Europe, feasibility and psychometric properties. Experts from eight European countries (The Netherlands, Germany, Belgium, Ireland, Italy, France, Spain and the UK) and from the disciplines of gerontology, clinical psychology, old age psychiatry, medical sociology, physiotherapy, occupational therapy and nursing, contributed to the workshops and scientific literature searches. The web-based consultation enabled the involvement of professionals and user groups across a wider number of countries. The iterative collaborative consensus process involved four steps, which are described below. Procedure Step 1. Experts (the authors and see acknowledgements) from Europe attended a study day in June 2005 to critically evaluate the conceptual domains for key psychosocial interventions and to establish a preliminary list of potential outcome measures for use across Europe. Two parallel consensus workshops, each with equivalent discipline and country participant representation, were conducted to synthesise findings and reach consensus. These involved detailed discussion, first of the domains to be included and then of the most relevant measures to be included in each domain. Each group achieved a consensus, voting as necessary, followed by feedback to the whole workshop, where again consensus was achieved. Data were derived using cross-disciplinary knowledge and resources from European nations, including the findings from the EADC memory clinic study (Diaz et al., 2005), a manual of common dementia care rating scales (Burns, Lawlor & Craig, 2004) and data from DIADEM, a previous pan-european study of psychosocial intervention across Europe, which included unpublished literature searches (Vernooij-Dassen et al., 2003). For this review, we excluded measures of cognition since this has been covered by others (Diaz et al., 2005). For similar reasons, at Step 2 we excluded evaluation of the costs of interventions, as overviews of the associated issues have been outlined in other traditional-style consensus statements (Knapp, 2007; Wimo, 2007). Ninety-two measures covering ten domains, (where some measures are used across more than one domain) were initially selected and scrutinised during the workshops (see Appendix 1, process ratings 1 & 2). As a result of this process, fifty-seven measures across the ten domains were identified for the web-based consultation (see Appendix 1, process rating 2). Measures were excluded at this stage on the general basis that other measures were available for the domain that appeared to have greater potential as outcome measures for psychosocial intervention studies in Europe.

3 16 E. Moniz-Cook et al. Step 2. The web-based consultation was conducted using Alzheimer s Europe and the INTERDEM website ( between July 2005 and March Participants were asked to: indicate if the measure was known and used in their country; provide study details of use of the measure; add measures that were not listed if these had demonstrated value for psychosocial intervention outcome measurement; and provide evidence (contacts or references) for these. The webbased consultation included 136 people from 13 countries (i.e. the original eight countries and Switzerland, Greece, Finland, Sweden and Poland) and a wide range of disciplines including neurologists, psychiatrists, geriatricians, physicians, psychogeriatricians, psychologists, nurses, occupational therapists, physiotherapists, social workers and staff working for the Alzheimers Societies. It also included five family carers. Findings from this were used in a second workshop in March 2006 to redefine the list of domains and measures to be studied and to develop quality criteria for the evaluation of measures. For the latter, common limitations of outcome measures noted 10 years previously (Ramsay, Winget, & Higginson, 1995), together with the Centre for Outcomes, Research and Effectiveness (CORE) guidance on outcome measurement in routine clinical practice for older people (Sperlinger, Clare, Bradbury, & Culverwell, 2004), were examined to derive criteria and an evaluation rating scale. Domain leaders (BW, MV-D/EV, FV, RC two domains, Mde-V, GM, MO/R-MD and EM-C, JH) piloted a search strategy and evaluation of the measures (Appendix 1, process rating 3). Step 3. A third consensus workshop (June 2006) was arranged where INTERDEM participants critically appraised pilot study results. The search strategy and operational definitions for the qualitative review were further refined to improve their reliability. Domain leaders conducted a second literature search and evaluation between July and October The search engines used were: PubMed, Web of Science, PsychInfo and EMBASE with terms for domain combined with the term dementia or Alzheimer ;s results of this first step were then combined with the name of the scale. Domain leaders used their literature review and knowledge base to complete a qualitative evaluation of each measure within their domain, employed a pre-determined rating scale for consistency and then prepared a summary of their findings with justification for the final choice of measures. Qualitative evaluation of measures for psychosocial intervention in dementia care was divided into three areas, i.e. European Utility, Feasibility and Properties (Table I). Each of these was weighted towards one important aspect of pan-european psychosocial intervention outcome measurement and, overall, a total of 24 operationally defined evaluation criteria were used for:. European utility. Translations and use of these in dementia research were of key importance. The following five criteria were used: number of published studies in peer reviewed journals; number of peer reviewed studies from individual European nations; number of translations available; number of translations where the measure had been validated; and number of translations where the measure had been used in dementia research.. Feasibility. Key considerations were ease of administration and availability. The following seven criteria were used: level of training required; profession-specific expertise required; length taken to administer the instrument; used in routine clinical practice; used in intervention research; cost of measure; and costs of translations.. Properties. The key emphasis was on use in psychosocial intervention studies where the measure had shown change following an intervention. The following 12 criteria were used: psychometric properties (validity and reliability); scale developed with dementia-specific populations; face validity with dementia-specific populations; ability to measure positive capacity in participant (person or carer, dependant on measure); ability to directly access the person (i.e. extent to which a measure relies on proxy or informant ratings); validity of self-report; ability to span the full career in dementia (for example, also captures those who are not burdened or distressed or those who have not lost skills i.e. positive capacity, as well as the range of stages and settings, i.e. in-home and care home and that the participant may encounter); demonstrates change in longitudinal observational studies; demonstrates change in published psychosocial or mixed intervention research; demonstrates change in pharmacological studies; demonstrates change in published psychosocial intervention research in individual European nations; and demonstrates change in published and unpublished psychosocial intervention research using European translations. Each of the nine domains was evaluated in detail by the domain leaders. Measures within domains were quality rated using agreed criteria and operational definitions either as a Yes/No or a rating for evidence as 1 3 i.e. poor/minimal, moderate and adequate/acceptable. For example, support for measures on translation was operationally defined as: (1) Minimal: 52 European translations; (2) Moderate: >2 but 56; (3) Adequate/ Acceptable/Good: >6 translations.

4 Table I. Psychosocial outcome measures with potential for use across Europe and their quality ratings 1. European/international utility Feasibility Properties Sensitivity Scale Research published in peer reviewed journals Translation - breadth of languages Useful in routine clinical practice Used in intervention research studies Easily available/no cost incurred No training Required Valid and reliable Scale Dementia applicable (aspects of face validity) Measures positive capacity in person or carer Measures self-report views person/ carer for carer scales 2 Measures across stages in dementia career Longitudinal\ Studies Intervention \Studies Patient Mood CSDD GDS Patient quality of life QOL-AD P C & proxy 3 P C & proxy 2 3 DQOL Author request P C EQ-5D P& proxy C 73proxy C 3 1 Patient ADL/IADL Lawton -PSMS & IADL P C& SC AMPS P observed by SC ADFACS C Patient behaviour RMBPC P3 C NPI e P 3C mostly drug studies Carer mood HADS Cost C 7 3C 3C 1 2 GHQ Cost for some C research CES-D Carer burden ZBI C SCQ C (Continued)

5 Table I. Continued. European/international utility Feasibility Properties Sensitivity Scale Research published in peer reviewed journals Translation - breadth of languages Useful in routine clinical practice Used in intervention research studies Easily available/no cost incurred No training Required Valid and reliable Scale Dementia applicable (aspects of face validity) Measures positive capacity in person or carer Measures self-report views person/ carer for carer scales 2 Measures across stages in dementia career Longitudinal\ Studies Intervention \Studies SSCQ-7 7 Carer quality of life SF items SF 12 available online WHOQOL-Bref Available on request from authors, registration requested, no costs C 1 2 No carer studies C 2 2 No carer studies EQ-5D C 3 C 3 1 No carer studies Global measures CIBIC plus Interviews P&C separately GBS Interviews P&C separately Staff carer morale GHQ- 12 or Cost for some research SC MBI Cost for scale SC Quality Ratings were Yes/No; or evidence-based using a predetermined operational definition: 1 ¼ poor/minimal; 2 ¼ moderate; and 3 ¼ acceptable/adequate/good. 2 P ¼ person or patient with dementia; C ¼ family carer; SC ¼ staff carer.

6 European consensus on outcome measures 19 Step 4. Results were reviewed at a fourth consensus workshop (November 2006), by following which the final scales were selected and the domain leader justification for their selection was validated by two raters (RMD and EM-C) using the pre-determined rating scale criteria used for evaluation by domain reviewers. The last meeting, to finalise the results, occurred in February Results Step 1 generated 92 measures covering 10 domains, which were scrutinised during the first workshops (see Appendix 1, process ratings 1 & 2,). These were reduced to 57 measures across 10 domains for use in the web-based consultation (see Appendix 1, process rating 2). At Step 2, economic and needs assessment measures were excluded, since the former had been reviewed elsewhere (Knapp, 2007; Wimo, 2007) and, for the latter, the web-based consultation confirmed our preliminary observations that pan- European interventions using needs assessment were largely absent. An additional global functioning patient domain was added since this emerged as important after the web-based consultation. The web-based consultation also resulted in three measures that had been initially rejected at Step 1 being re-included for consideration at Step 3 and four new measures being added. Fourteen measures were excluded following Step 2, either because they were in the excluded domains (4) or because little evidence of their use across Europe had emerged (10). At Step 3, nine non-cognitive domains and 50 measures were included in the final study, on clinical and theoretical grounds. These were: Patient Mood (5); Patient Quality of Life (QOL) (10); Patient Activities and Instrumental activities of daily living (ADL/IADL) (7); Patient Behaviour (6); and Patient Global Function (2). Carer measures included: Family Carer Mood (5); Family Carer Burden (to include stress, strain and competence) (4); Family Carer QOL (5); and Staff Carer Morale (to include coping with patient behaviour, stress, burnout, competence and satisfaction (6) see Appendix 1, process rating 3). Quality ratings for each of the 50 measures are available from the authors. In each domain, we aimed to select a maximum of two or three measures, based on these quality ratings. Table I illustrates the 22 measures across nine domains which were finally selected as having potential (based on the study objectives) for pan- European psychosocial intervention research, i.e. Patient Mood (2); Patient QOL (3); Patient ADL/ IADL (3); Patient Behaviour (2) Patient Global Function (2); Family Carer Mood (3); Family Carer Burden (2); Family Carer QOL (3); and Staff Carer Morale (2). Patient mood The Cornell Scale for Depression in Dementia (CSDD) (Alexopoulos, Abrams, Young, & Shamoian, 1988) and the Geriatric Depression Scale (GDS) (Yesavage et al., 1983) are the most widely used, but in the EADC (Diaz et al., 2005) memory clinic survey their reported use was in only 13 and 22% of the memory clinics participating, respectively. The Hospital Anxiety and Depression Scale (HADS) (Zigmond & Snaith, 1983) is rarely used; the FACE mood rating from the COOP/ WONCA (Volicer, Hurley, & Canberg, 1999) requires a trained observer and is limited to studies of severe dementia in Belgium and the Netherlands. The Rating Anxiety in Dementia (RAID) (Shankar, Walker, Frost, & Orrell, 1999), as an anxiety analogue of the CSDD, has promise, but has mainly been used in the UK and lacks evidence of sensitivity to change in response to treatment. The validity of the GDS (and its versions) in dementia is controversial since it relies on recall of feelings over the past week (Montorio & Izal, 1996), has a positive response set-bias with low re-test score correlations (Bedard et al., 2003) and internal consistency reduces with dementia-severity (Muller-Thomsen, Arit, Mann, Mass, & Ganzer, 2005). Whilst the GDS-15 may not perform well in moderate dementia, it can be used in pre-clinical and early dementia (McCabe et al., 2006) for patients with awareness of their disabilities,since a person s ability to accurately report their limitations and abilities accounts for much of the discrepancy between GDS-self-report and informant-report in people with and without dementia (Snow et al., 2005). Caregivers can act as proxy reporters (Logsdon & Teri, 1995) but the advantage of CSDD is its reliance on the perspective of both the person with dementia and the caregiver. The CSDD emerges well from all comparative studies with the GDS and other depression measures (Mayer et al., 2006; Muller-Thomsen et al., 2005; Vida, DesRosiers, Carrier, & Gauthier, 1994) and demonstrates sensitivity to change in treatment studies. It is the measure of choice for patient mood. For studies requiring a short self-report nondiagnostic measure to monitor change, the GDS- 12R may be considered since, in significantly cognitively impaired samples, it drops three items to improve on the validity of the GSD-15 (Sutcliffe et al., 2000). Family carer mood The GDS (Yesavage et al., 1983) is limited in its use for young carers and studies using the Symptom Check List (SCL) (Derogatis, Lipman, & Covi, 1973)-mood subscales with dementia carers are few. The Center for Epidemiological Studies, Depression Scale (CES-D) (Radloff & Teri, 1986) has been

7 20 E. Moniz-Cook et al. used in North American caregiver interventions (Ostwald, Hepburn, Caron, Burns, & Mantell, 1999; and the REACH Study: Burns, Nichols, Martindale-Adams, Graney, & Lummus, 2003) but has not been developed for pan-european research. Both the self-administered General Health Questionnaire (GHQ) (12 and 28 versions; Goldberg & Williams, 1988) and the HADS can be used clinically and for research, but the latter has greater potential (Bjelland, Dahl, Haug, & Neckelmann, 2002; Mykletum, Stordal, & Dahl, 2001). It measures, and can also screen for, both anxiety and depression, has cut-points for the degree of disturbance and age-norms are available. Both the HADS and GHQ have validated published translations in more than six languages. However, carer intervention studies with the HADS are rare. In contrast, the GHQ demonstrates sensitivity to psychosocial intervention (Brodaty, Gresham & Luscombe, 1997; Woods, Wills, Higgistonn, Hobbins, & Whithy, 2003), distinguishes with a case cut-point healthy caregivers from those who are distressed, and has sub-scale scores for monitoring treatment effects on both mood and behaviour (Werneke, Goldberg, Yalcin, & Ustun, 2000). If depression is the primary intervention target, the CES-D may have potential, but it will require further validation across Europe before it can be used for psychosocial intervention research. The HADS has potential for use since it measures both anxiety and depression. As a measure of general distress, the GHQ remains the measure of choice. Patient QOL The conceptual basis for QOL measurement has recently grown, with over a dozen rating scales available for people with dementia (Ettema, Dröes, Lange, de Mellenbergh, & Ribbe, 2005a; 2005b; Ready, Ott & Grace, 2004; Schölzel-Dorenbos et al., in press). Since QOL is a highly subjective construct, the perception of an individual is preferable to proxy ratings (Thorgrimsen et al., 2003), but in severe dementia proxy ratings are useful (Selai, Vaughan, Harvey, & Logsdon, 2001). Proxy-measures that fared well in our quality ratings include the QUALIDEM (Ettema, Dröes, de Lange, Mellenbergh, & Ribbe, 2007) and the Discomfort Scale for Dementia in Alzheimer s-type - DS-DAT (Hurley, Volicer, Hanrahan, Houde, & Volicer, 1992). Overall, our analysis suggests that the Quality of life in Alzheimer s disease (QOL-AD) (Logsdon, Gibbons, McCurry, & Teri, 1999) is the measure of choice. It is brief (only 13 items), has demonstrated sensitivity to psychosocial intervention (Spector et al., 2003), correlates with health-utility measures, is widely translated and used internationally and can be used by people with MMSE scores as low as three (Hoe, Katona, Roch, & Livingston, 2005; Thorgrimsen et al., 2003). The Dementia Quality of Life Instrument (DQOL) (Brod, Stewart, Sands, & Walton, 1999) may have an advantage over the QOL-AD when more detail of quality of life is required, but for patients it can appear repetitive and the self-rated version is probably limited to people with mild to moderate dementia. If a generic health utility measure that is short, easy to administer and independent of the underlying disease is required, the EuroQOL (EQ5D) (Brooks, 1996) has strong potential. It can be used with a mixed population with varying co-morbidities but it lacks an in-depth validity for people with dementia and studies using the EQ5D in dementia are few. Family carer QOL Two conceptually distinct types of measures of QOL exist: i.e. health-related such as the Short-Form Health Survey (SF) scales (Ware & Sherbourne, 1992), the EuroQOL and the World Health Organization Quality of Life Assessment instrument (WHOQOL) (WHOQOL Group, 1998) and disease-specific tools. Measures within these groups may target various dimensions (Lim & Zebrack, 2004). The SF-scales (12 and 36) have good psychometric properties and are widely translated but there are few longitudinal or intervention studies in dementia (Argimon, Limon, Vila, & Cabezas, 2005; Bond, Clark, & Davies, 2003) or studies of caregiver intervention (Markowitz, Gutterman, Sadik, & Papadopoulos, 2003) in which the SF-scales are used. The multi-dimensional WHOQOL has cross-cultural international validity and applicability with data (albeit not from intervention research) available for dementia caregivers (Banerjee et al., 2003; Takashaki, Tanaka, & Miyaoka, 2005). Two disease-specific scales for caregivers are found in the PIXEL study in France (Thomas et al., 2006) and a recent study from Russia describing a new QOL for Caregivers measure (Glozman, 2004). Wider translation and validation of these caregiver disease-specific QOL measures is needed before they can be used for outcome research. The SF-scales are easy to administer but not to score. Given the high correlations between the 12 and 36 versions, the shorter scale may be a good starting point for future research. If a generic, widely-used, easy to administer measure is required, the EQ5D is a potential alternative (Brazier, Jones, & Kind, 1993), although, as in the case of patient QOL studies, studies with family caregivers are also absent. The WHOQOL-Bref takes slightly longer than the SF-scales to administer but has the advantage of an easier scoring system and both the SF-scales and the WHOQOL-Bref (WHOQOL Group, 1998) measures have been translated and validated in intercultural studies.

8 European consensus on outcome measures 21 Overall, caregiver QOL measurement for psychosocial intervention studies is in its infancy. Family carer burden The 22-item version of the Zarit Burden Interview (ZBI) (Zarit, Reever, & Bach-Peterson, 1980) is the most widely used measure in caregiver research. The 12-item version suits clinical settings (Bédard et al., 2003; O Rourke & Tuokko, 2003a). Other versions (i.e. 18-item, 14-item, 12-item, 8-item and 4-item) have been used for both dementia care and research (Arai, Tamia, & Yano, 2003; O Rourke & Tuokko, 2003b). Despite its long history of use in carer intervention studies, few intervention studies using the ZBI were identified where significant changes were demonstrated (see Dröes et al., 2004). A recent meta-analysis (n ¼ 5930) of interventions for carers found significant, but small, positive effects on burden, but there is no information on what measures were used (Pinquart & Sorenson, 2006). The COPE-Index (McKee et al., 2003) measures carers subjective perceptions of positive and negative aspects of caregiving with translations into Italian, Greek, Swedish and Polish. No studies of outcome were identified. The Relative Stress Scale (RSS) (Greene, Smith, Gardiner, & Timbury, 1982) is a dementiaspecific caregiver stress measure with psychometric studies from the UK and Norway. Pharmacological studies using the RSS note significant change (e.g. Moretti et al., 2004) but for psychosocial intervention only one dementia-caregiver intervention study with non-significant outcomes was found (Conlin, Caranasos, & Davidson, 1992). The Sense of Competence Questionnaire (SCQ) (Vernooij- Dassen, Persoon, & Felling, 1996) is adapted from the ZBI using the family crisis model of Bengtson and Kuypers (1985). It has 27- and 7-item versions, with the latter suitable for clinical practice. Versions of the SCQ have been used in dementia care studies in the Netherlands, Belgium, France and England, with new studies ongoing in Germany and Portugal. In three out of four intervention studies, significant changes were found on this measure (Dröes et al., 2004; Graff et al., 2006; Moniz-Cook et al., 2001a; Vernooij-Dassen, Huygen, Felling, & Persoon, 1995). The SCQ has potential for use in future pan-european outcome caregiver research. Despite the paucity of intervention outcome data on the widely used ZBI, this measure will probably stand on its own in future well-designed intervention studies and also for those where base comparator samples are required. Patient ADL/IADL The Physical Self-Maintenance/Instrumental Activities of Daily Living (PSMS/IADL) (Lawton & Brody, 1969) measure ADL and IADL respectively, but are usually applied separately with the former usually replaced by the KATZ (Katz, Ford, Moskowitz, Jackson, & Jaffe, 1963). The KATZ and PSMS/IADL have been used widely in antidementia drug studies and in some psychosocial intervention studies in North America and Europe (Burns, Nichols, Martindale-Adams, Graney, & Lummus, 2003; Gitlin, Corcoran, Winter, Boyce, & Hauck, 2001). They are frequently applied only in part, using selected items (e.g. Barberger- Gateau et al., 2004). Some now consider their hierarchical order of complexity questionable and, although they have been used as a gold standard against which the psychometric properties of new instruments might be assessed, they were not developed for dementia populations. Measures developed with dementia-populations include: the Blessed (Blessed, Tomlinson, & Roth, 1968), which does not measure positive capacity and is therefore limited in its value for psychosocial intervention research; the Disability Assessment for Dementia (DAD) (Gelinas, Gauthier, McIntyre, & Gauthier, 1999) and the Interview for Deterioration in Daily Living Activities in Dementia (IDDD) (Teunisse & Derix, 1991), which were both developed for application with carers of people with early stage dementia. Although the DAD was developed primarily for psychosocial use, its reported application has been limited to anti-dementia drug studies, whilst the IDDD has been used in both pharmacological and psychosocial studies (e.g. Graff et al., 2006). The Alzheimer s disease Functional Assessment and Change Scale (ADFACS) (Galasko et al., 1997) was specifically developed for dementia-drug research. It allows researchers to select the appropriate level of ADL across the participant severity range. No psychosocial studies using the ADFACS were found. The ADFACS may be worth considering, if only to explore its conceptual relevance and value as an outcome measure in psychosocial research studies. The Assessment of Motor and Process skills (AMPS) (Fisher, 1997) is the only dementia-specific measure that was developed for outcome research with demonstrated sensitivity to psychosocial intervention in the Netherlands and UK. However, it requires specialist occupational therapist training, takes one hour to conduct and users of the measure note that that direct observation of IADL is important to provide additional information regarding process and motor skills that can subtly change over the course of dementia (Doble, Fisk, MacPherson, Fisher, & Rockwood, 1997). Both the Lawton PSMS-IADL and KATZ scales remain widely used in psychosocial intervention research and are easy to complete. The PSMS-IADL is not as popular as the KATZ but has an option for patient self-report. In the absence of an adequate

9 22 E. Moniz-Cook et al. dementia-specific measure that is short, easy to complete, reliable and accesses positive capacity, the Lawton PSMS-IADL has strong potential for outcome measurement of psychosocial intervention in dementia. Patient behaviour The Revised Memory and Behavior Problems Checklist (RMBPC) (Teri et al., 1992) has been widely used in many important caregiver studies with demonstrated sensitivity to interventions. This short caregiver self-administered measure requires no training; detects differences in mild, moderate and severe dementia; yields a global summary score and three sub-scale scores (including mood); rates the frequency of problems and caregiver coping; and is useful in routine clinical practice. Its nursing home version, the RMBPC-NH (Allen, 2003), is not widely used. The BEHAVE-AD (Reisberg et al., 1987) evaluates behavioural symptoms in Alzheimer s disease (AD); is responsive to psychological interventions (Monteiro, Boksay, Auer, Torossian, Ferris, & Reisberg, 2001); and can detect symptom variability over the disease course (Reisberg et al., 1987). Both the RMBPC and BEHAVE-AD have been used in several European countries but few validated translations exist. The Neuropsychiatric Inventory (NPI) (Cummings et al., 1994) is a de facto standard for measuring neuropsychiatric symptoms in clinical trials. In the EADC (2005) memory clinic study it was used with 27% of patients. The interviewer rates frequency, severity and level of caregiver distress. The instrument demonstrates stage-specific symptom trends in AD (Mega, Masterman, O Connor, Barclay, & Cummings, 1999), but few psychosocial intervention studies using this outcome measure were found. The NPI-Nursing Home (NPI-NH) (Wood et al., 2000) and NPI-Questionnaire (NPI-Q) (Kaufer et al., 2000) versions are yet to be fully developed (Lange, Hopp & Kang, 2004). As compared with the BEHAVE-AD, the NPI is the measure of choice as it has a wider range of behaviours (such as apathy); is sensitive to behavioural changes, such as disinhibition in Fronto-Temporal Dementia (Malloy & Grace, 2005); and taps the emotional impact of symptoms on the caregiver. The RMBPC also assesses caregivers reaction to problem behaviour and can be used as a brief and practical alternative. However, there is greater potential for caregiver report-bias with self-administration than with a structured interview. Adequate measures for psychosocial intervention in care homes are still outstanding. Studies to improve the psychometric properties of the NPI-NH are an avenue for future development. Global patient measures The Clinicians Global Impressions of Change (CIBIC-Plus) (Schneider et al., 1997) and Gottfries- Brane-Steen (GBS) (Gottfries, Brane & Steen, 1982) are comprehensive global measures of clinically detectable change that may have been missed when specifically focusing on one or more pre-defined domains. The CIBIC-plus interview with the person and the carer was developed from the Global Clinical Impression (Guy, 1976; Schneider et al., 1997) and systematically evaluates the degree of change on cognition (patient interview); function (caregiver interview); and behaviour (separate patient and caregiver interviews). Interviews are conducted by experienced clinicians separately with study patients and caregivers to assess overall change on the CIBIC-Plus. Interviewers must be blinded to adverse events and other study assessments. To ensure consistency, the same clinician completes all CIBIC-Plus interviews for each study patient and associated caregiver wherever possible. It has been used for the evaluation of effects of anti-dementia medication, and in this domain it maybe useful in routine clinical practice. The CIBIC-Plus has potential for development in new psychosocial research, but its use in routine psychosocial practice is currently premature. The GBS scale has been shown to be robust and valid with high reliability when used by a variety of healthcare professionals. It is sensitive to change over time, thus enabling it to be used in evaluating the effects of treatment. It is also reliable across a number of countries and languages. Neither has been used in psychosocial intervention studies and both are worthy of future consideration as a global measure for the severity of dementia. The CIBIC- Plus is of particular interest as it combines the sensitivity of the clinical investigation with the standardised approach of the traditional rating scales, but the GBS also deserves closer attention as a global assessment for dementia. Staff carer morale The Minnesota Satisfaction Questionnaire (MSQ) (Weiss, Davis, England, & Lofquist, 1967) for job satisfaction has been used in long-term care facilities (e.g. Friedman, Daub, & Cresci, 1999; Waxman, Carner, & Berkenstock, 1984) but there are few psychosocial intervention studies demonstrating change. The NPI (reaction and distress components) has not been used in outcome studies, whilst the Challenging Behaviour Scale (CBS) (Moniz- Cook, Woods, Gardiner, Agar, & Silver, 2001b) has been used in the UK and Ireland. No dementia outcome studies using the Graham Job Satisfaction Scale (see Graham et al., 1996) were found. The Maslach Burnout Inventory (MBI) (Maslach,

10 European consensus on outcome measures 23 Jackson, & Leiter, 1996) is widely used across Europe in observational (cross-sectional and longitudinal), psychosocial intervention (Baldelli et al., 2004) and staff support (Berg, Welander-Hansson, & Hallberg, 1994; Hallberg & Norberg, 1995; Moniz-Cook, Win, Elston, & Millington 1995) outcome studies. However, it is conceptually weak as an outcome measure, since it is difficult to demonstrate a causal relationship between burnout and its deteriorating effects in follow-up studies (Prosser et al., 1996). There are various reasons for this. Burnout, by definition, is a chronic stress syndrome (Taris, Le Blanc, Schaufeli, & Schreurs, 2005) in that it develops gradually, persists over a long period and interventions need to be sustained to demonstrate a clinically significant effect. Therefore, measurement is problematic in the face of high staff turnover and studies tend to focus on the attitudinal concomitants of burnout, such as satisfaction, organisational commitments, turnover intentions and self-reported psychosomatic symptoms or health problems (Brenninkmeijer & Van Yperen, 2003) rather than burnout per se. The overlap between burnout and stress is also unclear, since when measured against psychosomatic symptoms, they appear to be unrelated (Bargellini et al., 2000) and the association between stress and sick-leave does not equate to burnout (Tillett, 2003). Use of individual MBI sub-scales is probably better than using full MBI scores, since in practice, outcome studies show change on subscales only (Berg et al., 1994). Developing a European perspective on job satisfaction outcomes using the MSQ is warranted, since to date most European studies appear to develop their own study-specific measures, which does not allow adequate comparison of outcome across nations. Many of these study-specific satisfaction scales also have psychometric properties of variable quality. For staff perception of behaviour management, comparative research is needed on the NPI-distress components and the CBS-challenge component. Although outcome studies outside the UK are absent, the GHQ-28 (Goldberg & Williams, 1988) is a useful measure that may identify those who are not stressed but may become so. It also allows both global measurement as well as individual subscale evaluation of aspects of psychological morbidity. However, care must be taken in data collection as staff may not be prepared to rate some items such as those about suicide ideation. Discussion This paper has highlighted the shortage of multicentre pan-european psychosocial intervention studies or studies that demonstrate change in more than one country. However, this review of the quality of the wide range of dementia care outcome measures has identified twenty-two measures across nine conceptual domains, which may be especially useful in psychosocial intervention research across Europe. Academics and practitioners from a range of disciplines and countries used their expertise and theoretical knowledge to examine the quality of measures. Our methodology was novel in that it extended beyond that of the traditional expert consensus statement, to a more rigorous multi-level study that included an iterative process of expert consensus, a web-based consultation and a systematic literature review. The first difficulty we encountered was that of epistemological variation in expert professional concepts in the context of outcome measurement. For example in the behaviour domain neuropsychiatric perspectives might focus on the measurement of clusters of behaviour such as affect, apathy and psychosis (Katona et al., 2007) or agitation, psychosis, mood and other disorders (Ballard, O Brien, James, & Swann, 2001), whereas psychosocial perspectives might focus on challenging behaviour, where an individual s behaviour is seen as a challenge only if others (informal and formal carers) perceive it so (Bird, 2005). The implications are that key variables, and how they are measured, will alter. For example, for neuropsychiatric measurement, a cluster-specific measurement of apathy or agitation is indicated, whilst contrasting psychosocial perspectives argue against cluster-driven measurement and for generic measures that are derived from the problems or concerns reported by caregivers themselves (Moniz-Cook & Bird, 2003; Moniz-Cook et al., 2001b). Similarly, measurement of ADL/IADL is complex, with a number of constructs guiding practice in this area and instrument development. For example, some studies might focus on the differentiation between functional impairment and disability, whilst others may be concerned with psychosocial intervention and the measurement of positive capacity as opposed to measurement of incapacity. We found studies where ADL/IADL had shown sensitivity to change on a disease-specific ADL/IADL measure or one that had not been developed for use with dementiapopulations. However, it remains arguable that the outcome achieved was attributable to the particular goal of the given psychosocial intervention. In this domain an emerging and highly promising scale, the AMPS, has to be treated with caution, since it requires specialised training and can only be administered by one professional discipline (Bouwens et al., in press). A second difficulty that emerged was that of cultural variation and associated criteria for evaluating measures. Whist it was relatively easy to operationally define criteria for language translations in terms of both numbers and whether the translation was validated with

11 24 E. Moniz-Cook et al. populations, cultural appropriateness or specificity of a measure was less easy to consider. Given variations in levels of stigma and diverse understandings of the meaning of dementia across Europe (Vernooij-Dassen et al., 2005), psychosocial interventions and associated outcome measures will also need to address the rich cultural diversity in European nations. A third difficulty was the existence of emerging measures for outcomes of specific psychosocial interventions, which are yet to be validated across Europe. Study limitations First, there was lack of involvement of people with dementia and carers, since only five families responded to the web consultation, thus leading to an absence of information from a user perspective. Second, representation on the web-consultation was strong from Western and some parts of Northern and Southern Europe but, apart from one participant from Poland, Eastern Europe was not represented. There was minimal response from the Scandinavian countries, probably due to poor representation in our expert study group. Third, under-representation of experts from some countries may have resulted in exclusion of unpublished literature, for example with the RSS for carer burden, where eight participants from three countries suggested positive potential in the webconsultation, but the systematic literature search did not identify published research to support this. Improving outcome measurement for psychosocial intervention research The EUROCARE study of fourteen nations (Schneider, Murray, Banerjee, & Mann, 1999) found high levels of carer burden and stress using the ZBI and GHQ respectively. Although the ZBI has a notable history in family carer research, with wide use in both cross-sectional and intervention studies, we found limited numbers of outcome studies where it demonstrated sensitivity to change following psychosocial intervention. Acton & Kang (2001) noted that the pooled effect of interventions evaluated by the ZBI was zero (0.00), whilst Sörensen and colleagues (2002) observed lower effect sizes than those of other measures. Knight et al. (1993) suggest that emotional outcome measures are more sensitive than the ZBI. Notwithstanding the ZBI as a measure of burden, Brodaty Green, & Koschera (2003) found only one out of twenty intervention studies with a statistically significant effect on burden. Interestingly, when the ZBI was adapted using Bengtson & Kuypers (1985) family crisis model to derive the SCQ (Vernooij- Dassen et al., 1999), we note an emerging potential of this new scale for aspects of psychosocial outcome family caregiver research. Given the continued worldwide societal costs of informal care (Wimo, Winblad, & Jonsson, 2007), pan-european measurement of family burden may require refining, using results of the factors associated with carer burden, such as family concerns (Richards, Duggan, Carr, Wang, & Moniz-Cook, 2003); attention to their subjective experience (Murray, Schneider, Bnerjee, & Mann, 1999); and difficulties with behaviour (Schneider et al., 1999). For the latter, the RMBPC emerges with strong potential for development across Europe as an outcome measure that targets the day-to-day difficulties faced by family carers, since it has demonstrated sensitivity to change in their coping with, and management of, behaviour following psychosocial intervention in the US and Canada (Hebert et al., 2003; Mittelman, Roth, Haley, & Zarit, 2004; Teri et al., 2005) as well as in the UK (Moniz-Cook et al., 2001a). An important gap was the relative absence of intervention studies on what might be important to people with dementia, including their subjective needs (e.g. Dröes et al., 2006b; Van der Roest et al., 2007). The Camberwell Assessment of Need for the Elderly (CANE) (Orrell & Hancock, 2004; Reynolds et al., 2000) has been used for psychosocial intervention research in the UK (Ashaye, Livingston, & Orrell, 2003), is widely translated across Europe ( has been used in observational research in the UK and the Netherlands and measures needs of people with dementia as well as those of informal and paid carers. It therefore has strong potential for development as outcome measure in future pan-european psychosocial intervention research. A multiplicity of psychosocial interventions has arisen from an equally wide-ranging theoretical and knowledge base. Their goals need to be associated with the underlying concepts they address and the associated measures need to be as close to the goal or goals addressed. Thus, broadly defined goals of slowing down deterioration or reducing carer burden may not be enough for psychosocial interventions that aspire to address the needs of a person with dementia or the family. For example, ADL/ IADL can fluctuate and carer burden will usually vary across the range of family relationships and living arrangements (Murray et al., 1999). Similarly, if rehabilitative goals are set to build on strengths and enhance positive capacity in the person with dementia, or self- efficacy in the family or staff caregiver, then measures of positive capacity and positive aspects of care are needed. In the staff support literature, for example, a new self-efficacy scale (Mackenzie & Peragine, 2003) may be a starting point for future pan-european research, since intervention studies showing change in job satisfaction with a widely used and validated measure remain rare. Overall, there was a paucity of measures of positive capacity or satisfaction, apart from patient-rated QOL measures.