The validity of questionnaire measures for assessing depression after stroke

Transcription

1 Clinical Rehabilitation 2003; 17: The validity of questionnaire measures for assessing depression after stroke NB Lincoln, CR Nicholl, T Flannaghan School of Psychology, University of Nottingham, M Leonard Department of Psychiatry, Queens Medical Centre and E Van der Gucht School of Psychology, University of Nottingham, Nottingham, UK Received 2nd December 2002; returned for revisions 4th February 2003; revised manuscript accepted 27th March Objective: Depression has been reported to occur frequently after stroke. The aim of the study was to assess the validity of questionnaire measures for screening for depression after stroke. Design: Cross-sectional correlational study between questionnaire measures of mood and psychiatric interview. Setting: Hospital and community. Participants: Stroke patients were recruited from hospital wards and from a randomized controlled trial of cognitive behavioural therapy. Main measures: Beck Depression Inventory, Wakefield Depression Inventory, General Health Questionnaire 28 and Schedules for Clinical Assessment in Neuropsychiatry. Results: Poor agreement was found between psychiatric diagnosis and questionnaire measures of mood. The sensitivity of the questionnaire measures was high, but specificity was low. No cut-off points with satisfactory sensitivity and specificity could be identified from ROC curves. Conclusions: Although questionnaire assessments of depression provide a satisfactory screening method, specificity values are too low to provide a basis for the diagnosis of depression. Measures need to be developed with higher specificity to facilitate screening for depression after stroke. Introduction Depression has been reported to be a serious problem after stroke with estimates of prevalence ranging from 25% to 79%. 1 Depression has a negative impact on long-term recovery and presents an obstacle to rehabilitation. 2 The study of depression after stroke is made more complex by Address for correspondence: Professor NB Lincoln, School of Psychology, University of Nottingham, University Park, Nottingham NG7 2RD, UK. Nadina.Lincoln@nottingham.ac.uk uncertainty as to the nature of the depression. It could be an endogenous form of depression, a psychological reaction to a traumatic event or an artefact resulting from the use of inappropriate classification criteria. 3 Various explanations have been proposed for the wide variation in prevalence rates. These include case-mix, time since onset and methods of assessing depression. However, there are difficulties in assessing depression in patients with accompanying physical illness. Most assessments are based on the assumption that patients are aware of their situation and able to give an accu- Arnold / cr687oa

2 Questionnaire measures for assessing depression 843 rate report of their mood. This may not be the case for stroke patients, who have been found to minimize or exaggerate changes in physical, cognitive and affective functioning after stroke. 4,5 Snowdon 6 also reported that older adults tend to deny or somatize symptoms of depression. A further problem is that traditional psychiatric criteria, such as Diagnostic and Statistical Manual-III-R (DSM-III-R), 7 and psychological measures, such as the Beck Depression Inventory (BDI) 8 and Wakefield Depression Inventory (WDI), 9 rely heavily on somatic symptoms in combination with depressed mood. Others include items that may occur due to the effects of stroke, such as restless disturbed nights and not being able to enjoy a good book or TV programme. These symptoms may reflect behavioural or neurological impairments resulting from the stroke, the effect of age or the environmental effects of hospitalization. The aim of this study was to assess the validity of questionnaire measures and to determine cut-off scores for identifying post-stroke depression, in relation to diagnosis by standardized psychiatric interview. The BDI 8 was chosen as it has been reported to be sensitive to depressive symptoms in medical patients 10 and has been used in several studies of post-stroke depression However, while BDI scores may provide reasonable approximations of severity of depression they may be a relatively inaccurate way of defining cases. 11 House et al. 14 examined the BDI in relation to diagnosis by psychiatric interview in stroke patients. They found that with a cut-off set to achieve a sensitivity of 90% the BDI had a false positive rate of 25 30% depending on the time since stroke. Others have suggested using higher cut-off values for identifying depression, using the BDI in neurological patients. 15 The General Health Questionnaire 28 (GHQ-28) 16 was also used as it has been used to assess rehabilitation outcome. 17 It is also a measure of distress rather than specifically depression and therefore may be more sensitive to the problems of stroke patients. Cut-off scores have been recommended for identifying depression on the GHQ-28 and GHQ-30 in neurological patients. 18,19 Psychiatric diagnosis of depression has typically been seen as the gold standard against which alternative measures are compared. The aim of this study was to examine the correspondence between scores above the threshold for depression on questionnaire measures of mood and diagnosis according to DSM-III-R and International Classification of Mental and Behavioural Disorders 10 (ICD-10) classification of depression obtained using the Schedule for Clinical Assessment in Neuropsychiatry (SCAN). 20 Methods An opportunity sample of 20 stroke patients who gave informed consent was recruited from two hospitals. In addition, the data from 123 stroke patients recruited for a randomized controlled trial of cognitive behavioural therapy 21 were included. Patients were included if they had a stroke, were able to give informed consent, were not blind or deaf, did not have dementia documented in the medical notes prior to the stroke, could speak English, were not in residential care before the stroke, lived within a defined geographical area and had communication abilities sufficient to complete questionnaires and an interview. All patients completed questionnaire measures, the BDI, 8 the WDI 9 and GHQ The 20 inpatients were assessed on questionnaires by a research assistant. The patients from the randomized controlled trial were sent questionnaire measures by post or, if still in hospital, the questionnaires were administered by a research assistant. Those who scored more than 10 on the BDI or more than 18 on the WDI were included in the trial. All patients were also assessed by a standard psychiatric interview, the Schedules for Clinical Assessment in Neuropsychiatry (SCAN), 20 and assigned DSM-III-R and ICD-10 diagnoses, using the CATEGO5 computer program. The 20 inpatients were assessed by a specialist registrar in psychiatry and the patients from the randomized trial by a community psychiatric nurse. Both interviewers had been formally trained in the administration of the SCAN. In each case the person administering the questionnaire and the interviewer were blind to each other s assessment scores, except that the interviewer for patients in the randomized trial knew

3 844 NB Lincoln et al. sion, 60 patients (41%) were diagnosed as depressed by ICD-10 criteria and 21 (15%) by DSM-III-R criteria. Agreement between the two classifications was poor (kappa = 0.31). Results are shown in Table 2. Exclusion of the 32 patients with mild depression according to ICD-10 criteria still resulted in poor agreement between classifications (kappa = 0.39). Three further patients had a secondary ICD-10 diagnosis of depression, primary diagnoses for these patients were dementia of Alzheimer s type and alcohol abuse. All questionnaire measures were highly significantly correlated (r s = , all p 0.001) with each other. The distribution of scores on questionnaires is shown in Table 3. Patients diagnosed as depressed on either ICD-10 and DSMthat patients had scored within the depressed range on questionnaires. Results Of the 143 patients assessed, 69 (48%) were women and 74 (52%) were men. The mean age was 66 (SD 13.5) years. One hundred and five patients (73%) were living in their own home, 27 (19%) were in hospital and 9 (7%) were in a residential home. Classification of patients by ICD- 10 and DSM-III-R criteria are shown in Table 1. Most patients (49% on DSMIII-R and 34% on ICD-10) had no psychiatric condition. Comparing patients with a current diagnosis of depres- Table 1 Distribution of diagnostic categories on the SCAN DSM-III-R diagnosis Frequency % a Major depression, single episode unspecified (296.20) Major depression, single episode in full remission (296.26) 2 1 Residual schizophrenia, unspecified (295.60) 2 1 Schizoaffective disorder depressive type (295.72) 2 1 Panic disorder without agoraphobia (severe) ( ) 1 1 Alcohol dependence (303.9) 2 1 Primary insomnia (307.42) Sleep/wake schedule (307.45) 2 1 Dream anxiety disorder (307.47) 1 1 Primary hypersomnia (780.54) 2 1 No diagnosis found (ndf) Missing 10 7 ICD-10 classification Depressive episode (f32) 12 8 Mild depressive episode (f32.0) 1 1 Depressive episode of mild severity without somatic symptoms (f32.00) Depressive episode of mild severity with somatic symptoms (f32.01) 3 2 Depressive episode of moderate severity without somatic symptoms (f32.10) 12 8 Depressive episode of moderate severity with somatic symptoms (f32.11) 1 1 Severe depressive episode without psychotic symptoms (f32.2) 2 1 Recurrent depressive episode of mild severity without depressive symptoms (f33.0) 1 1 Recurrent depressive disorder currently in remission (f33.00) 1 1 Dementia in Alzheimer s disease with early onset (f00.0) 1 1 Alcohol abuse, harmful use (f10.10) 3 2 Alcohol dependence, currently abstinent (f10.20) 2 1 Non-organic sleep disorder (f51) 2 1 Non-organic insomnia (f51.0) 12 8 Non-organic hypersomnia (f51.1) 1 1 Non-organic disorder of sleep wake schedule (f51.2) 1 1 Sleep terrors (f51.3) 1 1 Nondependent abuse of laxatives (f55.1) 1 1 No diagnosis found (ndf) Missing cases 9 6 a Total percentage is less than 100 due to rounding down scores.

4 Questionnaire measures for assessing depression 845 III-R scored significantly higher on the BDI, WDI and GHQ-28 total scores (p 0.001) than patients who were not diagnosed as depressed. Questionnaire scores were also used to classify patients as depressed or not depressed. Patients were classified as depressed if scoring >10 on the BDI, >18 on the WDI or >4 on the GHQ28. Cutoffs were chosen at levels consistent with the treatment trial 21 and previous research with stroke patients. 14,18,19,22,23 Comparisons of those classified as depressed using the two psychiatric criteria and the questionnaire measures are shown in Table 2. Agreement between psychiatric diagnoses and questionnaire measures was poor (kappa = ). ROC curves were computed using ICD-10 and DSM-III-R classification of depression to identify the optimum cut-off points for diagnosing depression on the BDI, WDI and GHQ-28. Cut-off points and corresponding values of sensitivity and specificity are shown in Table 4. None of the three measures produced ROC curves indicating clear cut-off points with both high sensitivity and specificity. The area under the ROC curves ranged from 0.69 to The highest values were obtained from the GHQ-28. Table 2 Comparison of depression diagnosis according to interview and questionnaire methods ICD-10 diagnosis DSM-III-R diagnosis a Measure Not Depressed Not Depressed depressed depressed 74 (52%) 60 (42%) Kappa 112 (77%) 21 (15%) Kappa BDI Not depressed Depressed WDI Not depressed Depressed GHQ Not depressed Depressed DSM-III-R Not depressed Depressed a Frequencies differ from ICD-10 and DSM-III-R frequencies due to missing data. DSM-III-R, Diagnostic and Statistical Manual-III-R; ICD-10, International Classification of Mental and Behavioural Disorders 10; BDI, Beck Depression Inventory; WDI, Wakefield Depression Inventory; GHQ-28, General Health Questionnaire Table 3 Distribution of scores on questionnaires according to diagnosis Depressed Not depressed n M IQR n M IQR ICD-10 BDI WDI GHQ-28 a DSM-III-R BDI WDI GHQ-28 a a Based on 0,0,1,1 scoring. M, Median; IQR, interquartile range; BDI, Beck Depression Inventory; WDI, Wakefield Depression Inventory; GHQ-28, General Health Questionnaire 28.

5 846 NB Lincoln et al. Table 4 Sensitivity and specificity of cut-off scores for questionnaire measures against depression diagnosis a ICD-10 diagnosis DSM-III-R diagnosis Sensitivity Specificity Sensitivity Specificity BDI cut-off ( ) WDI cut off ( ) GHQ cut-off ( ) ,18, 19,22,23 All recommended cut-off scores gave high sensitivity (93 98%) but low specificity (24 36%). Optimum cut-off scores were obtained by calculating the sum of the sensitivity and the specificity, with the constraint that sensitivity should be at least 80%. The optimum cut-off on the BDI in relation to DSM-III-R criteria was 15/16, which had a sensitivity of 91% and a specificity of 56%, and in relation to ICD-10 was 12/13, which had a sensitivity of 83% and specificity 44%. The optimum values on the WDI were 20/21 in relation to DSM-III-R (sensitivity 86%, specificity 50%) and 18/19 in relation to ICD-10 (sensitivity 92%, specificity 46%). On the GHQ-28 the optimum cut-off in relation to DSM-III-R was 11/12 (sensitivity 81%, specificity 68%). and in relation to ICD-10 criteria was 7/8 (sensitivity 85%, specificity 61%).

6 Questionnaire measures for assessing depression 847 Discussion Classifying patients as depressed or not depressed according to psychiatric interview and questionnaires were found to correspond poorly. The lack of consistency between the two psychiatric classification systems was not reported by Pohjasvaara et al. 13 who also used the SCAN to assess post-stroke depression or Almeida and Almeida 24 in a study of depression in people aged over 60 (kappa agreement 0.75). However, others have found rates of agreement to be poor Using DSM-III-R criteria, patients are classified as suffering a major depressive episode, while ICD-10 depression classification covers a range of severity of depression, mild, moderate and severe, with separate coding for depression with somatic and psychotic symptoms. ICD-10 criteria pay more regard to symptoms of reduced energy and increased fatigability. The inclusion of mild depression in ICD-10 and the difference in key symptoms between the classification systems inevitably result in discrepancies. This is exacerbated in physically ill patients, who experience less severe depression than psychiatric patients 10 and are more likely to be tired or lacking in energy due to their physical condition. This needs to be considered when evaluating studies using psychiatric interview as a gold standard for the identification of depression, as the two systems of depression classification are clearly not interchangeable. High rates of depression have been found using questionnaire measures compared to psychiatric interview in previous studies. 14,22,23,28 However, as Madianos et al. 28 pointed out, although not all people identified as depressed by questionnaire methods fit the criteria for clinical depression, they are a group of people experi- Clinical messages Questionnaires measures of depression do not correspond to diagnosis by psychiatric interview in people with stroke. Psychiatric diagnostic classifications do not show consistency in a sample of stroke patients. Questionnaires measures of mood are sensitive to depression after stroke but not specific. encing a high rate of depressive symptoms. It is therefore important to identify such people as they may have an increased risk of developing depression in the future. Good diagnostic measures are expected to have sensitivity greater than 80% and specificity greater than 60%. Measures used for screening, as opposed to diagnosis, may need to be more sensitive (90%), if it is important that no one with the problem is missed. House et al. 14 also found that using cut-offs on the BDI, which were high enough to detect most of those with problems (sensitivity 90%), meant that a high proportion would need further evaluation to identify whether they had a mood disorder. Raising the sensitivity to 90% in the present study resulted in low specificity. A screening measure with low specificity would be little better than conducting a full assessment on everyone. Greater specificity in the classification would therefore be desirable. Patients categorized as depressed on ICD-10 and DSM-III-R criteria scored significantly higher on questionnaires than patients who were not depressed. The questionnaire measures were all highly significantly correlated with each other. However, the correlations only accounted for between 23% and 34% of the variance in scores. The inclusion of participants from a treatment trial 21 meant that few patients had low scores on the questionnaire measures. However, the advantage of including these patients was that there were similar numbers of patients who were depressed and not depressed according to psychiatric interview. The results highlight the limitations of standard assessment measures of depression after stroke. Psychiatric diagnoses are generally taken as the gold standard for assessing post-stroke depression 28,29 but there was low agreement between the diagnostic systems used. The results suggest that although questionnaire measures can be used to screen for depression due to their high sensitivity, further assessment is necessary before it is assumed that a person is diagnosed as suffering from depression. There was no clear advantage of any one questionnaire over the others. The GHQ-28 showed closest correspondence with interview and had high sensitivity, even though it is intended to detect psychological distress rather than depression. On all measures it was found that the optimum cut-off points to correspond to psychiatric diagnosis were higher than

7 848 NB Lincoln et al. those recommended in previous studies. Acknowledgements We thank the Stroke Association for financial support and Dr Lyn Sutcliffe and Ms Lucy Rother for assistance with data collection. References 1 Kneebone II, Dunmore E. Psychological management of post stroke depression. Br J Clin Psychol 2000; 39: Morris PLP, Raphael B, Robinson RG. Clinical depression is associated with impaired recovery from stroke. Med J Aust 1992; 157: Gainotti G, Azzoni A, Razzan, C, Lanzillotta M, Marra C, Gasparini F. The Post-stroke Depression Rating Scale: a test specifically designed to investigate affective disorders of stroke patients. J Clin Exp Neuropsychol 1997; 19: Hibbard MR, Gordon WA, Stein PN, Grober S, Sliwinski MJ. Unawareness in stroke patients: a problem of the mind and not the body. Arch Phys Med Rehabil 1990; 71: Gordon WA, Hibbard MR. Post stroke depression: An examination of the literature. Arch Phys Med Rehabil 1997; 78: Snowdon J. The epidemiology of affective disorders in old age. In: Chiu E and Ames D eds. Functional psychiatric disorders of the elderly. Cambridge: Cambridge University Press, 1994: American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, third edition revised (DSM III-R). Washington, DC: American Psychiatric Association, Beck AT, Steer RA, Brown GK. BDI-II manual. San Antonio, TX: The Psychological Corporation, Snaith RP, Ahmed SN, Mehta S, Hamilton M. Assessment of severity of primary depressive illness: Wakefield self-assessment depression inventory. Psychol Med 1971; 1: Clark DA, Cook A, Snow D. Depressive symptom differences in hospitalized, medically ill, depressed psychiatric inpatients and non-medical controls. J Abnormal Psychol 1998; 107: House A, Dennis M, Mogridge L, Warlow C, Hawton K, Jones L. Mood disorders in the first year after stroke. British Journal of Psychiatry 1991; 158: Kotila M, Numminen H, Waltimo O, Kate M. Depression after stroke: Results of the FINNSTROKE study. Stroke 1998; 29: Pohjasvaara, T, Leppavuori A, Siira I, Vataja R, Kaste M, Erkinjuntti T. Frequency and clinical determinants of post stroke depression. Stroke 1998; 29: House A, Dennis M, HawtonK, Warlow C. Methods of identifying mood disorders in stroke patients. Age Ageing 1989; 18: Lykouras L, Oulis P, Adrachta D, Daskalopoulou et al. Beck Depression Inventory in the detection of depression in neurological inpatients. Psychopathology 1998; 31: Goldberg D, Williams PA. User s guide to the General Health Questionnaire. Windsor: NFER- Nelson, Juby LC, Lincoln LB, Berman P. The effect of a Stroke Rehabilitation Unit on functional and psychological outcome: A randomised controlled trial. Cerebrovasc Dis 1996; 6: Lykouras L, Adrachta D, Kalfakis N et al. GHQ-28 as an aid to detect mental disorders in neurological inpatients. Acta Psychiatr Scand 1996; 93: O Rourke S, MacHale S, Signorini D, Dennis M. Detecting psychiatric morbidity after stroke: comparison of the GHQ and the HAD scale. Stroke 1998; 29: World Health Organization. Schedules for clinical assessment in neuropsychiatry. Geneva: WHO, Lincoln NB, Flannaghan T, Sutcliffe LM. Cognitive behavioural psychotherapy for depression after stroke: a randomised controlled trial. Stroke 2003; 34: Aspen I, Verhey F, Lousberg R, Lodder J, Honig A. Validity of the Beck Depression Inventory, Hospital Anxiety and Depression Scale, SCL-90 and Hamilton Rating Scale as screening instruments for depression in stroke patients. Psychosomatics 2002; 43: Johnson G, Burvill PW, Anderson CS, Jamrozik K, Stewart-Wynne EG, Chakera TMH. Screening instruments for depression and anxiety following stroke: experience in the Perth community stroke study. Acta Psychatr Scand 1995; 91: Almeida OP, Almeida SA. Short versions of the geriatric depression scale: a study of their validity for the diagnosis of a major depressive episode according to ICD-10 and DSM-IV. Int J Geriatr Psychiatry 1999; 14: Hendersen AS, Jorm AF, Mackinnon A et al. The prevalence of depressive disorders and the distribution of depressive symptoms in later life: a survey using Draft ICD-10 and DSM-III-R. Psychol Med 1993; 23: Phillips CJ, Hendersen AS. The prevalence of depression among Australian nursing home residents: results using draft ICD-10 and DSM-III-R criteria. Psychol Med 1991; 21: Silverstone PH. Concise Assessment for Depression (CAD): a brief screening approach to depression in the medically ill. J Psychosom Res 1996; 41: Madianos MG, Gournas G, Stefanis CN. Depressive symptoms and depression among elderly people in Athens. Acta Psychiatr Scand 1992; 86: