Cross-cultural assessment of the Contextual Memory Test (CMT)

Transcription

1 246 Occupational Therapy International, 7(4), , 2000 Whurr Publishers Ltd Cross-cultural assessment of the Contextual Memory Test (CMT) NAOMI JOSMAN Department of Occupational Therapy, Faculty of Social Welfare and Health Studies, University of Haifa ADINA HARTMAN-MAEIR School of Occupational Therapy, Hebrew University, Israel ABSTRACT: The Contextual Memory Test (CMT) measures aspects of memory and metamemory of people with cognitive disabilities. The assessment tool was originally developed and standardized in the United States. The objectives of this study were: (1) to evaluate the applicability of the CMT to an Israeli population; (2) to further investigate the construct validity of the CMT to discriminate among age groups; and (3) to analyse the 40 items on the CMT from a cultural point of view. The CMT was administered to 217 typical Israeli adults, grouped into three age categories, closely matching those in the US normative study (Toglia, 1993). Similar levels of performance were obtained for Israelis and Americans on the various test components. Statistically significant differences between American and Israeli subjects performance levels were evident in three memory components in the elderly groups (group 3) and in only two memory components in the young group (group 1). In addition, within-sample comparisons of the three Israeli age groups yielded significant age effects for recall, recognition, strategy use and general awareness. This study confirmed discriminant validity for the CMT. The tool seems to be highly appropriate for use by occupational therapists in assessing memory and metamemory with American and Israeli adult subjects. The relatively small size of the age groups and the lack of random selection of subjects are limitations of this study. Therefore, it is recommended that the study be replicated with a larger and randomized sample. The multifaceted nature of the assessment provides much more information than traditional recall scores, and the metamemory components enhance both differential diagnosis and appropriate planning of treatment. Key words: assessment development, Contextual Memory Test, memory, metamemory.

2 Cross-cultural assessment of the CMT 247 Memory is an underlying component in most activities of daily life and plays a central role in every learning experience. Memory is a multistage process and includes acquisition, storage and retrieval of information recorded by the various modalities. Memory functions include short-term and long-term memory, and explicit and procedural memory, which are often examined through immediate and delayed recall and recognition tasks of the auditory and visual channels (Bjork and Bjork, 1996). In the normal population, memory changes are observed in older adults, such as difficulties in recall (Craik and Jennings, 1992). Various types of memory impairment are associated with injuries or diseases involving the central nervous system, such as head injury, stroke and dementia. Memory deficits have a significant impact on functional performance and constitute one of the major challenges in cognitive rehabilitation (Raskin and Sohlberg, 1996). According to Ponds and Jolles (1996), there is a growing interest in the possible mediating role of self-knowledge and self-belief in personal memory functioning, termed metamemory. Metamemory is defined as the knowledge we have about ourselves, the tasks we encounter and the strategies we use (Flavell, 1985; Brown, 1987; Garner, 1987). Knowledge about ourselves implies awareness of personal memory capacities and limitations; knowledge of tasks implies a recognition of the memory demands of different tasks. The interactive knowledge of one s abilities in relation to task performance is termed self-efficacy (Willoughby et al., 1996). The strategies we use are ways of organizing information in order to enhance memory performance (Koriat and Goldsmith, 1996). For example, when confronted with the task of memorizing a telephone number, if you are aware of the limitations of your short-term memory, you might use strategies such as rehearsal or jotting down the number. The growing interest in metamemory underlies the importance of metamemory in education and in treatment of memory disorders (Alexander et al., 1995; Jarman et al., 1995; Metacalfe and Shimamura, 1994). Metamemory plays a significant role in the rehabilitation of people with memory deficits. The awareness of personal strengths and limitations and the knowledge of task requirements and available strategies are prerequisites for successful performance in everyday life (Boake et al., 1995). A variety of standardized tests are available for diagnosing memory disorders, such as the Wechsler Memory Scale (Wechsler, 1987). Most of these tests address specific memory functions and facilitate diagnoses as well as neuroanatomical localization (Lezak, 1995). These tests, however, do not address the implications of memory deficits on daily life, and do not include the assessment of self-awareness or memory strategies, factors that concern occupational therapists who assist clients in acquiring skills to enhance daily function.

3 248 Josman and Hartman-Maeir The Rivermead Behavioral Memory Test (RBMT Wilson et al., 1985) provides a profile of memory abilities and limitations in everyday tasks. It does not provide information pertaining to self-awareness or strategy use, which are both necessary components for the planning of treatment and rehabilitation. The Contextual Memory Test (CMT) (Toglia, 1993) was developed to measure different aspects of memory and metamemory to be used in conjunction with other memory tests. The test measures: (1) immediate recall, delayed recall and recognition; (2) awareness of memory ability; and (3) the use of strategies. The CMT is a standardized test that addresses everyday memory, metamemory and strategy use. The CMT directly assesses self-awareness of one s memory ability, self-efficacy and strategy use, in addition to the more traditional aspects of immediate and delayed recall, as well as recognition. Normative data on the CMT were collected on 375 adults in the New York area, ranging in age from 18 to 86 years. Previous reliability and validity studies have been conducted on the CMT with people with brain injury. Reliability estimates for the parallel form ranged from 0.73 to 0.81 for the different parts of the test. Test-retest reliability ranged from 0.74 to 0.87 for the control group and from 0.85 to 0.94 for the group with brain injuries. Concurrent validity with the RBMT was established, with correlations ranging from 0.80 to 0.84 (Toglia, 1993). The test consists of two picture cards, each containing 20 objects that are thematically related to either a restaurant theme or a morning theme. The pictures are not presented in the order in which the depicted functional activities usually occur. This permits the assessment of a patient s ability to spontaneously detect and use the theme that employs the context strategy as a memory aid. Standardized evaluations may not be valid when used to evaluate persons from a cultural group other than the group on which the evaluation was standardized (Teresi et al., 1989) because of cultural bias, language and translation differences (Cermak et al., 1995). Helms (1992) identified four kinds of cultural equivalence in testing: linguistic equivalence whether the translation is equivalent to the original language; functional equivalence whether test scores have the same functional meaning in different cultures; conceptual equivalence the amount of familiarity with the content of the test across cultures; and psychometric equivalence the extent to which the test measures the same things at the same levels across cultures. Conceptual equivalence of the CMT was the focus of this study, specifically the extent to which the CMT pictures are familiar to Israeli subjects, and their depiction of valid Israeli examples of the restaurant or morning contexts was examined. Assessments should be re-standardized in the new culture before being applied to a clinical situation. Recent research in occupational therapy has addressed the issue of cultural adaptation mainly in regard to functional assessments of Activities of Daily Living (ADL) (Fisher et al., 1992; Dickerson and Fisher, 1995; Magalhaes et al., 1996).

4 Cross-cultural assessment of the CMT 249 The purposes of this study were: (1) to evaluate the applicability of the CMT for the Israeli population on all components of the CMT; (2) to further investigate the construct validity of the CMT regarding its ability to discriminate among age groups; and (3) to analyse the 40 items on the CMT from a cultural point of view. Method Participants Two hundred and seventeen participants (129 women and 88 men) with no documented central nervous system dysfunction and ranging in age from 18 to 86 years were included in the study. Participants were divided into three age groups parallel to those of the US normative study: (1) 18 to 39 (mean age 23.12); (2) 40 to 58 (mean age 50.21); and (3) 59 to 86 (mean age 73.13) (see Table 1). Seventy-four participants were included in the younger group, 48 participants in the middle group and 95 participants in the older group. The sample was recruited from around the country (Northern Israel and Jerusalem) by using the snowball sampling method. The mean number of years of education for this group was 13.13, but their level of education ranged from elementary school to graduate school. Instrument The CMT was used to measure the four aspects of memory for all participants namely, recall, recognition, awareness of memory and strategy use. Recall Immediate and delayed memory performance was assessed using one of the picture cards (either the restaurant or the morning theme) recall was assessed either immediately after exposure or after a 15-minute delay. Recognition Recognition was assessed using 40 cards from the same theme, in which 20 are identical to the originals and the remaining 20 are similar pictures but not identical to the originals. Awareness of memory A questionnaire incorporated in the test assesses: (1) general memory awareness, comprising eight questions addressing personal memory capacity and observed changes in memory ability; (2) awareness of task performance,

5 250 Josman and Hartman-Maeir comprising one question about prediction of performance before testing and five questions about estimation of actual performance after testing. Strategy use Strategy use was probed by debriefing the subject after testing, asking questions such as, What did you do in order to remember the pictures? The complete test was administered according to instructions included in the CMT manual (Toglia, 1993). For the purpose of this study, the CMT questionnaire, instruction manual and scoring sheets, as well as the wording on several pictures, were translated into Hebrew (with permission from the author). Translation was done by the second author with back translation by another translator in order to check for any mistranslation. The study did not call for any further changes in the CMT. Procedure An individual appointment of about 20 minutes was scheduled with each subject. Before testing, demographic data were obtained, including age, place of residence, years of education and education level. The two picture cards were used interchangeably half of the participants were tested on the restaurant card and the remaining 108 participants on the morning card. The CMT was administered according to the procedure in the manual (Toglia, 1993). Administration of the CMT is very clear according to the test manual and does not require special training. Results Cross-cultural comparisons Demographics of participants are presented in Table 1. Two hundred and seventeen subjects were divided into three age groups, parallel to those of the US normative study. The educational level of American and Israeli participants differed. Whereas 24% of the American normative sample had elementaryschool and high-school diplomas, 51% of the Israeli sample had corresponding levels of education. Likewise, 15% of the American sample had graduate or doctor of medicine degrees, whereas only 7% of the Israeli sample had an equivalent level of education. It is apparent that the normative American participants included in the study had higher levels of education than their Israel counterparts (Toglia, 1993). The performance of Israeli participants on the CMT compared with the American normative sample is presented in Table 2. The comparisons of immediate recall scores are also shown in Figure 1. The performance of Israeli participants did not yield significant differences between the two test versions,

6 Cross-cultural assessment of the CMT 251 TABLE 1: Comparison of demographic data between American and Israeli subjects Israelis Americans Age group range n M SD n M SD Gender n(%) n(%) Female 129 (59) 156 (41.6) Male 88 (40.6) 210 (56) Educational level Grade school 21 (9.7) 23 (6) High-school diploma 90 (41.5) 68 (18) One or more years of college 53 (24.4) 90 (24) College degree 38 (17.5) 143 (38) Master s degree 11 (5.1) 34 (9) PhD or MD 4 (1.8) 15 (4) or between genders. The scores for the American normative sample are slightly elevated in comparison with scores for Israeli participants. Significant differences were obtained: (1) for the older age group (group 3) in three test components t(197)= 4.44, p=0.0002; t(197)= 4.09, p=0.0001; t(197)= 4.92, p= ; and (2) for group 1 in delayed and total recall t(235)= 2.36, p=0.02; t(235)= 2.75, p=0.01. The recognition part of the test was not administered in the American study, so cross-cultural comparisons are not available. An independent t-test was computed to compare the mean score performance of participants with that of normative sample on the different test components. The strategy component comparison yielded a significant difference between samples only for age group 1 t(235)=3.6, p= The general awareness component revealed no significant differences between normative and Israeli samples. The self-efficacy (estimation immediate recall) component revealed a significant difference only for age group 2 t(154)=4.57, p= Within Israeli age groups Comparisons between the three age groups of the Israeli sample were computed using one-way analysis of variance and post-hoc comparisons. Findings are presented in Table 3 and Figure 2. Results revealed a significant age effect for immediate recall F(2, 215)=38.27, p<0.0001; delayed recall F(2, 215)=52.55,

7 252 Josman and Hartman-Maeir TABLE 2: Performance score comparison between the three Israeli and American groups Group 1 Group 2 Group 3 Israeli American Israeli American Israeli American (n=74) (n= 163) (n=48) (n=108) (n=95) (n=104) t p t p t p Immediate recall Mean NS NS SD Delayed recall Mean NS SD Total recall Mean NS SD Recognition Mean SD Total strategy Mean NS NS SD General awareness Mean NS NS NS SD Self-efficacy Predicted immediate recall Mean NS NS NS SD Estimation immediate recall Mean NS NS SD

8 Cross-cultural assessment of the CMT 253 Recall score <40 40 to 59 >59 Age group Israelis Americans FIGURE 1: Comparison of Israeli and American immediate recall scores p<0.0001; recognition F(2, 215)=15.17, p<0.0001; general awareness F(2, 215)=5.3, p<0.005 and total strategy score F(2, 215)=5.6, p< Post-hoc analyses pointed to significant differences between age group 1 and age group 3 on all dependent variables. Comparisons between age group 2 and age group 3 on memory scores (recall and recognition) also showed significant differences. Investigation of Israeli responses to pictures To study the cultural relevance of the items in the two picture cards, the percentage of participants questioning the identity of a picture (Table 4) was calculated. If more than 5% questioned the identity of a specific picture, the TABLE 3: One-way analysis of variance, and Tukey s studentized range test post hoc analysis of differences between age groups Dependent variable SS df MS F p Post hoc comparisons Immediate recall ; 2 3 Delayed recall ; 2 3 Recognition ; 2 3 General awareness Total strategy score Score Immediate Delayed Recognition <40 40 to 59 >59 FIGURE 2: Comparison of Israeli age groups on memory scores

9 254 Josman and Hartman-Maeir picture was considered a culturally problematic item. The following three problematic pictures were identified in the restaurant card: Wallet (16.5%), Water and Tap (9.8%) and Sandwich (5.2%). Only one problematic picture was found in the morning card: the Sink (6.6%). In addition, the number of participants who failed to recall an item was calculated for each picture. If more than 30% of participants did not recall a specific picture, it was recorded. For the Restaurant card, the following pictures were found less memorable : Sugar packets (55.9%), Ashtray (55.3%), Wallet (48.7%), Water and Tap, Flowers (both 45.4%) and Table and Chair (37.5%). For the Morning card, less memorable pictures were: Cup of coffee (48.5%), Newspaper and Razor (both 47.1%), Clock (42.6%), Deodorant (40.4%), Blow dryer (38.9%) and Chest of drawers (36.7%). TABLE 4: Restaurant version (n=152) problematic items Picture % of subjects that % of subjects that did not questioned the identity of remember this picture this picture (>5%) (>30%) Sugar packets Sandwich Soda can Flowers Table and chair Coat Menu Water and tap Ashtray Wallet Napkins Cash register Morning version (n=136) problematic items Picture % of subjects that % of subjects that did not questioned the identity of remember this picture this picture (>3%) (>30%) Bed Razor Newspaper Blow dryer Coffee Towel Chest Toilet Deodorant Clock Shoes Sink

10 Cross-cultural assessment of the CMT 255 Supplementary analyses in the form of correlations between test results and demographic variables were also carried out. These analyses were conducted in order to control for intrinsic factors possibly influencing the finding of this study (Polit and Hungler, 1995). The samples included in the study were not randomized, as mentioned above. Therefore, correlations between test results and demographic characteristics, such as gender and level of education, were needed to control for extraneous variables in determining the true nature of the relationship between the independent and dependent variables under investigation (Polit and Hungler, 1995). The Pearson product moment correlation between scores on the CMT and demographic variables gender and level of education did not yield significant relationships. A statistically significant relationship between age and CMT performance was obtained, both for immediate and delayed recall (r= 0.50, p<0.0001; r= 0.57, p<0.0001). Discussion Applicability of the CMT for Israeli population The results of this study provide support for the use of the CMT as an appropriate and time-efficient tool for assessing visual memory, awareness of memory and use of strategy. But, as the CMT measures only contextual visual memory, it should be used in conjunction with additional tests for evaluating other aspects of memory abilities, such as auditory memory, prospective memory and so on. The mean recall performance of the older Israeli group was significantly poorer than that of the older normative sample (total recall: 23.52<26.82, p<0.0001), a smaller but significant difference between performance of the Israeli and American groups was evident only in delayed and total recall of the younger group (group 1) (30.30<31.85, p<0.01) a finding that is inconsistent with the findings for the other age group. The finding could be attributed to differences in the educational level of the two samples. To illustrate, a relevant study by Toglia (1993) found a significant correlation between CMT performance and level of education, immediate recall (r=0.32, p<0.001), and delayed recall (r=0.31, p<0.001). Individuals with more education tended to have higher recall scores than individuals with less education. We therefore recommend repeating this study with older and younger Israeli subjects. The finding of a significant difference on the total strategy score for only the youngest age group (the Israelis scored higher than their normative counterparts (11.58> 10.44; p<0.005) is surprising. This discrepancy may possibly reflect the different roles that Israelis assume at the young age of 18 by virtue of their compulsory three-year army service. In the army there are unusual demands for assuming high individual role responsibilities, whereas, by comparison, most Americans are engaged in college studies and adapt a somewhat more passive yet familiar student role from their high-school years. We

11 256 Josman and Hartman-Maeir recommend that additional studies comparing young Americans and Israelis be conducted in order to further explore the reasons for the obtained differences. The only self-efficacy comparison (estimation immediate recall) that had significance was in the second age group for both Israelis and Americans. The slight mean difference between actual memory performance and estimation of memory indicates an underestimation. This indication of underestimation is consistent with Toglia s finding (1993) where normal subjects tended to slightly underestimate their performance. Construct validity of the CMT The Israeli participants in this study obtained similar scores to the American normative sample, and discriminant validity was attained among age groups, thus supporting the use of the CMT in Israel. These findings are congruent with the research of Craik and Jennings (1992), who provide strong evidence for the decline in age-related memory performance. This is consistent with the recent research (Ponds and Jolles, 1996) indicating agerelated differences in memory, awareness and strategy use. These findings have been well substantiated in theory and clinical work, and provide support for the discriminative validity as part of the construct validity of the CMT. Cultural relevance The difficulty encountered in identifying three pictures suggests that these items may not be suitable for the Israeli population. The analysis of the percentage of pictures that were less remembered may also be the result of difficulty in item identification. Although additional factors may contribute to this outcome, the issue of cultural suitability should be considered. For example, the picture of sugar packets is not a typical presentation of sugar in an Israeli restaurant, where sugar packets are placed in a bowl or small plate, and not organized in a holder. Evaluators noted that many subjects mistook the sugar packets for napkins before reading the wording on the packets. This may explain why this item was neither questioned in the beginning nor remembered correctly. For Israeli clients, we recommend that the three pictures be replaced with pictures of similar items that are more culturally familiar. The less remembered pictures should be further examined in order to determine whether a cultural factor influenced memory performance. It is also recommended to conduct a content validity study, examining the clarity and contextual relevance of each picture to its respective theme.

12 Cross-cultural assessment of the CMT 257 Limitations The relatively small size of the age groups and the lack of random selection of subjects are limitations. It is recommended that the study be replicated with a wider sample. Conclusions The CMT constitutes a unique assessment tool, enhancing the occupational therapist s repertoire of memory function assessment tools. Occupational therapists evaluate and treat clients with memory dysfunction such as people suffering from dementia, schizophrenia and brain injury. In this study the CMT has been shown to be a culturally suitable and valid assessment instrument, and therefore its implementation in occupational therapy practice is strongly recommended. The multifaceted nature of the assessment provides much more information than mere traditional recall scores, and the metamemory components enhance both differential diagnosis and appropriate planning of treatment. Similar memory scores versus different awareness scores can support the differential diagnosis of dementia versus pseudodementia. In addition, dementia and Age Associated Memory Impairment (AAMI) can be distinguished from one another based on memory scores and self-efficacy scores. Treatment planning is enriched by the various scores of memory and metamemory, enabling more accurate clinical reasoning centred on the client s awareness of his or her memory deficits and disabilities. The CMT may hold much promise as an evaluation instrument for the purpose of documenting progress throughout the treatment process. However, further research is required with different client populations and before and after intervention, in order to establish the CMT as an outcome measure. References Alexander JM, Carr M, Schwanenflugel PJ (1995). Developmental of metacognition in gifted children: Directions for future research. Developmental Review 15: Bjork EL, Bjork RA (eds) (1996). Memory. San Diego, CA: Academic Press. Boake C, Freeland JC, Ringholz GM, Nance ML, Edwards KE (1995). Awareness of memory loss after severe closed-head injury. Brain Injury 9: Brown AL (1987). Metacognition, executive control, self-regulation, and other more mysterious mechanisms. In FE Weinert, RH Klowe (eds) Metacognition and Understanding. Hillsdale, NJ: Erlbaum, pp Cermak SA, Katz N, McGuire E, Greenbaum S, Peralta C, Maser-Flanagan V (1995). Performance of Americans and Israelis with cerebrovascular accident on the Loewenstein Occupational Therapy Cognitive Assessment (LOTCA). American Journal of Occupational Therapy 6:

13 258 Josman and Hartman-Maeir Craik FIM, Jennings JM (1992). Human memory. In FIM Craik, TA Salthouse (eds) Handbook of Aging and Cognition. Hillsdale, NJ: Erlbaum, pp Dickerson AE, Fisher AG (1995). Culture relevant functional performance assessment of the Hispanic elderly. Occupational Therapy Journal of Research 15: Fisher AG, Liu Y, Velozo CA, Pan AW (1992). Cross-cultural assessment of process skills. American Journal of Occupational Therapy 46: Flavell JH (1985). Cognitive Development. Englewood Cliffs, NJ: Prentice Hall. Garner R (1987). Metacognition and Reading Comprehension. Norwood, NJ: Ablex. Goto S, Fisher AG, Meyberry WL (1996). Assessment of motor and process skills applied cross culturally to the Japanese. American Journal of Occupational Therapy 50: Helms JE (1992). Why is there no study of cultural equivalence in standardized cognitive ability testing. American Psychologist 47: Jarman RF, Vavrik J, Walton P (1995). Metacognition and frontal lobe processes: At the interface of cognitive psychology and neuropsychology. Genetic, Social and General Psychology Monographs 2: Koriat A, Goldsmith M (1996). Monitoring and control processes in the strategic regulation of memory accuracy. Psychological Review 103(3): Lezak M (1995). Neuropsychological Assessment (3rd edn). Oxford: Oxford University Press. Magalhaes L, Fisher AG, Bernspang B, Linacre JM (1996). Cross cultural assessment of functional ability. Occupational Therapy Journal of Research 16: Metacalfe J, Shimamura AP (1994). Metacognition: Knowing about Knowing. Cambridge, MA: MIT Press. Polit DF, Hungler BP (1995). Nursing Research: Principles and Methods (5th edn). Philadelphia, PA: JB Lippincott. Ponds RWHM, Jolles J (1996). The abridged Dutch Metamemory in Adulthood (MIA) questionnaire: Structure and effects of age, sex and education. Psychology and Aging 11: Raskin SA, Sohlberg MM (1996). The efficacy of prospective memory training in two adults with brain injury. Journal of Head Trauma Rehabilitation 11: Teresi JA, Cross PS, Golden RR (1989). Some applications of latent trait analysis to the measurement of ADL. Journal of Gerontology 44: S196 S204. Toglia JP (1993). Contextual Memory Test (Manual). Tucson, AZ: Therapy Skill Builders. Wechsler DA (1987). Wechsler Adult Memory Scale Revised. New York: The Psychological Corporation. Wilson B, Cockburn J, Baddeley A (1985). The Rivermead Behavioral Memory Test (Manual). London: Thames Valley Test. Willoughby C, King G, Polatajko H (1996). A therapist s guide to children s self-esteem. American Journal of Occupational Therapy 50: Address correspondence to Naomi Josman, PhD, OTR, Department of Occupational Therapy, Faculty of Social Welfare and Health Studies, University of Haifa, Mount Carmel, Haifa 31905, Israel. naomij@construct.haifa.ac.il