Evaluation of a health literacy screening tool in primary care patients: evidence from Serbia

Transcription

1 Health Promotion International, Vol. 29 No. 4 doi: /heapro/dat011 Advance Access published 27 February, 2013 # The Author (2013). Published by Oxford University Press. All rights reserved. For Permissions, please journals.permissions@oup.com Evaluation of a health literacy screening tool in primary care patients: evidence from Serbia ALEKSANDRA JOVIĆ-VRANEŠ 1*, VESNA BJEGOVIĆ-MIKANOVIĆ 1, JELENA MARINKOVIĆ 2 and DEJANA VUKOVIĆ 1 1 Institute of Social Medicine and 2 Institute of Medical Statistics and Informatics, Medical School Belgrade University, Dr Subotica 15, Belgrade 1100, Serbia *Corresponding author. aljvranes@yahoo.co.uk SUMMARY Improving health literacy skills is important for patient comprehension of health-related topics and their ability to attend to their medical problems. Promoting health literacy is a pivotal policy for maintaining and promoting health. The objective of the present study was to translate the Test of Functional Health Literacy in Adults (TOFHLA; long and short versions) into Serbian and evaluate the translated and cross-culturally adapted questionnaires in Serbian primary care patients. The translated TOFHLA questionnaires were administered to 120 patients. Additionally, a self-completed questionnaire was used. Both descriptive and inferential statistics were measured. The mean score for the TOFHLA was (median, 78; SD ¼ 17.94; range, 0 100) and the mean Key words: health literacy; TOFHLA; primary health care; Serbia INTRODUCTION The sphere of health has expanded far beyond the confines of the healthcare system itself. Today, in almost every aspect of our lives, we are faced with questions and decisions about our health. As society changes, so do the necessary skills needed to function. Citizens have to continually assimilate new information and discard outdated information in order to guide their health decisions (Kickbusch et al., 2006). Health literacy is defined by the World Health Organization (World Health Organization,2009) as the cognitive and social skills, which determine score for the Short Test of Functional Health Literacy in Adults (STOFHLA) was (median, 32; SD ¼ 6.16; range, 0 36). Sex, age, education, self-perceived health and presence of any chronic disease were associated with health literacy scores. The internal consistency (Cronbach s alpha) was 0.73 for the TOFHLA numeracy subset, 0.95 for reading comprehension, 0.94 for the TOFHLA and 0.90 for the STOFHLA. The Pearson correlation between the TOFHLA and STOFHLA was The area under the curve of these two tests was 0.79 (95% CI, ). The Serbian translated versions of the TOHFLA questionnaires offer valid measures of functional health literacy. There were no differences between the reliability and validity of the short and long TOFHLA forms. the motivation and ability of individuals to gain access to, understand, and use information in ways to promote and maintain good health. Health literacy is a critical component of quality of life and should be treated as such in policy debates, not just in health, but across all sectors (Kawachi and Berkman, 1999). Access to good, reliable information is the cornerstone of health literacy. Yet, providing information, even good information, is often not enough. Somehow, we need to construct better communication channels to make sure that health information translates into healthy behaviors (Kickbusch, 2001; Kickbusch et al., 2006). 601

2 602 A. Jović-Vraneš et al. Health literacy may affect the health and ability of the healthcare system to provide effective, high-quality health care (AHRQ, 2004). Studies have shown that people with low health literacy understand health information less well, receive less preventive health care, incur higher medical costs and use inefficient mixes of services and expensive health services more frequently (Howard et al., 2005; Berkman et al., 2011). Instruments for measuring literacy in the health care setting have focused on the ability to read and, in some cases, numeracy. The Wide Range Achievement Test (WRAT) reading subset (Wide Range Inc., 1993), the Rapid Estimate of Adult Literacy in Medicine (REALM) (Davis et al., 1993) and the Test of Functional Health Literacy in Adults (TOFHLA) are widely used to measure health literacy (Parker et al., 1995). The WRAT and REALM are word recognition tests validated as instruments of reading ability; they are highly correlated with one another and with other traditional reading assessments (Parker et al., 1995). The aim of this study was to develop Serbian versions of questionnaires for measuring health literacy based on the TOFHLA questionnaires. METHODS Sample and setting The study was conducted in two randomly selected primary healthcare centers from a total of 16 primary health care centers in Belgrade, Serbia. Health care at the primary level in Serbia is provided by state-owned primary health care centers, which cover the territory of one or more municipalities or towns. The primary health centers provide at least preventive health care for all population categories, urgent care, general medicine, health care for women and children, health visitor services, as well as laboratory and other diagnostic services. Primary health care patients are outpatients. The sample size was calculated based on the number of total and first visits in each primary health care centers during the previous year. Assuming a standard error of 2%, the minimum sample size was 80 patients. To allow for nonrespondents (estimated to be 10%), at least 90 patients were enrolled. To ensure samples of equal size, we enrolled 60 participants from both centers. To diminish selection bias, patients were selected sequentially from the medical charts of patients waiting to be seen. One hundred and twenty patients were enrolled during the 4-week period. Exclusion criteria were age,18 years, lack of cooperation, insufficient visual activity to read the instruments being tested and being too ill to participate. Ethical approval was obtained from the Medical School University of Belgrade. A letter of introduction describing the study was given to all participants and written informed consent was obtained from all participants prior to beginning the testing. Instruments and procedures The original English version of the TOFHLA (long and short form) was translated into the Serbian language by a multidisciplinary team. The aim of cultural adaptation was to provide a version that was conceptually as close as possible to the original questionnaire, considering the patient s understanding. In the first step, the original questionnaires were translated by two independent bilingual translators with the intention to combine the two translated versions into consensus versions. In the next step, the questions were translated back into English. Qualified translators, public health experts and physicians discussed the problematic items (e.g. questions regarding health care insurance as adapted to the Serbian health care insurance system and the conversion of US dollars to Serbian dinars). In order to verify that the Serbian population understood and interpreted the Serbian version of the TOFHLA, it was administrated to 10 primary care patients. We found that the questionnaires were simple to administer and were readily accepted by the patients. The final Serbian translations of the short and long TOFHLA versions were tested on 120 patients. The long TOFHLA version consisted of two sections: a 50-item reading comprehension test and a 17-item numeracy test. The reading comprehension test was designed to determine a patient s ability to read passages using actual materials from a health care setting. The three passages are selected from: (A) instructions for preparation for an upper GI series, (B) patient s rights and responsibilities, and (C) a hospital informed consent agreement. Each passage had every fifth to seventh word deleted; for each blank, the respondent must select the word that best completed the sentence from a list of four words (a modified Cloze procedure) (Taylor,

3 Health literacy screening tool in primary care patients ). The three passages were ordered by increasing difficulty and it took 12 min to administer. The numeracy test assesses quantitative literacy needed in a healthcare setting (i.e. the ability to read and understand numerical information in the form of prescription bottles, taking medications, monitoring blood glucose, appointment slips and other health-related materials). The numeracy items took 10 min to administer. The 17 items were weighted to yield a numerical score of 50, which gives a total of 100 possible points for the TOFHLA when added to the 50 Cloze items. The short version of the Test of Functional Health Literacy in Adults (STOFHLA) consisted of 36 reading comprehension items ( passages A and B from the long TOFHLA version) and uses the modified Cloze procedure. It took 7 min to administer. We analyzed health literacy as scores and categories as follows: inadequate literacy (0 59 for TOFHLA and 0 16 for STOFHLA); marginal literacy (60 74 for TOFHLA and for STOFHLA); and adequate literacy ( for TOFHLA and for STOFHLA). An additional self-completed questionnaire containing 11 questions was used. Individual level variables covered sociodemographic characteristics, self-perception of material status, self-perception of health, health system utilization (i.e. self-reported doctor visits) and health status ( presence of chronic conditions including hypertension, diabetes mellitus, coronary artery disease, heart failure, chronic bronchitis or emphysema, asthma, arthritis and cancer). Selfperception of material status referred to personal assessment of financial status regardless of actual revenue. Education level was classified into three categories: low (8 years of school completed), medium (8 12 years of school completed) and high (.12 years of school completed). Self-perception of material and health status was measured with a 5-point Likert scale; however, for the final analysis, we summed the responses using a 3-point scale (poor, 1 2; average, 3; and good, 4 5). The study required two contacts with the participants, who were randomly selected into two groups of equal size from both centers. The first group completed the long TOFHLA version and an additional self-completed questionnaire during their first visits. Up to 1 week later, they completed the STOFHLA version. The second group completed the STOFHLA version and up to 1 week later, the long TOFHLA and an additional self-completed questionnaire. Statistical methods Descriptive statistics were reported in terms of the mean, standard deviation, standard error, median and quartiles for the continuous variables, and absolute frequencies and percentages for the categorical variables. For the purposes of comparison between the TOFHLA and STOFHLA, short versions of the questionnaires with scores 0 36 were translated to a score of (tstofhla). Statistical analysis using t-tests and analysis of variance (ANOVA) were applied to compare mean health literacy scores between socioeconomic variables in the TOFHLA and STOFHLA, and also between the TOFHLA and translated STOFHLA (tstofhla). Reliability was calculated using the Cronbach s alpha formulas (Cronbach, 1951; Bland and Altman, 1998). Convergent validity was assessed by the Pearson correlation between the TOFHLA and STOFHLA. The association of health literacy categories (inadequate, marginal and adequate) between questionnaires was tested using the Cohen s kappa test. For this analysis, the inadequate and marginal categories were combined into one category to denote limited functional health literacy. Area under the curve values of both tests were determined. Logistic regression was used to evaluate the association between socioeconomic factors and the congruence between the two questionnaires. Congruence was coded as 0 ¼ both TOFHLA and STOFHLA indicated the same categories, 1 ¼ different categories. Data were entered and analyzed using the Statistical Package for Social Sciences (version 16.0; SPSS Inc., Chicago, IL, USA). RESULTS Of the 120 enrolled primary care patients, 105 completed both questionnaires (TOFHLA and STOFHLA), yielding a response rate of 87.5%. Most of the patients were males (56, 53.3%), ranging in age from 21 to 84 years (mean age, years). Most of the patients had completed 8 12 years of school (55, 52.4%) and assessed their material status as average (73, 69.6%).

4 604 A. Jović-Vraneš et al. The total mean score for the TOFHLA was (median, 78; SD ¼ 17.94; range, 0 100) and the total mean score for STOFHLA was (median, 32; SD ¼ 6.16; range, 0 36). Table 1 reports the descriptive statistics (mean, standard deviation, standard error, median and quartiles) for each of the subsets in the TOFHLA and STOFHLA. The mean scores of the patients characteristics, distribution and differences are shown in Table 2. Sex, age, education, self-perceived health and presence of any chronic disease were associated with health literacy in both forms of the questionnaire (long and short versions). Male respondents compared with females had higher health literacy scores. The younger patients (44 years) had a very high mean health literacy score, which decreased with age. Education was also significantly associated with health literacy. Respondents with low education had the lowest health literacy scores ( for the long version and for the short version). Self-perceived health status was associated with health literacy scores, but health literacy scores were not related to selfperceived material status. Visits to health care providers were not associated with health literacy. A comparison of the two questionnaire forms showed statistically significant differences in the mean health literacy scores by the variables of education and economic status, while there was no difference in the other variables. No statistical differences were found between the total mean scores of the two questionnaire forms (Table 2). Adequate health literacy for the TOFHLA and STOFHLA was present among 62 and 87 participants, respectively, and limited in 43 and 18. The kappa values indicated an association between the measurements (k ¼ 0.460; SE ¼ 0.079; p ¼ 0.000). The Cronbach s alpha coefficient of reliability values for the TOFHLA, numeracy and reading comprehension were 0.94, 0.73 and 0.95, respectively, and 0.90 for the STOFHLA. Correlations with TOFHLA and STOFHLA were significant. The STOFHLA had a correlation (r ¼ 0.89) with the long version of the TOFHLA. The area under the curve value of the two tests was (95% CI; range, ). Using a logistic regression, we found that congruence is significantly different in less educated (OR ¼ 0.31, 95% CI ¼ ) and older participant (OR ¼ 8.75, 95% CI ¼ ) (Table 3). DISCUSSION Cross-cultural translation and validation of an analytical tool is a challenging task. The reliability and validity of an instrument is fundamental to research results (Patrick and Beery, 1991; Thomas et al., 2005), but establishing the reliability and validity of an instrument can be a complex process when an existing instrument is used to sample individuals of a different culture and who speak a language other than that of the original intended population (Jones and Kay, 1992; Huey-Shys et al., 2006). Different definitions of health literacy imply that health literacy is a complicated construct that depends on an individual s capacity to communicate and the demands posed by society and the healthcare system. Current widely used measures of health literacy, such as the TOFHLA and REALM, are useful screening tools in health care environments, but they are not comprehensive measures of health literacy, as they measure only selective domains that are thought to be markers of an individual s overall capacity (Nutbeam, 2009). A more comprehensive test is needed to understand the gap between capacities and current demands. Despite this limitation, we decided to translate the TOFHLA, which has Table 1: Descriptive Statistics for the Serbian version (TOFHLA and STOFHLA) (N¼105) Mean Standard deviation Standard error Median Q1 Q3 Maximum TOFHLA a Numeracy Reading comprehension STOFHLA b a TOFHLA range b STOFHLA range 0 36.

5 Health literacy screening tool in primary care patients 605 Table 2: Characteristics of study participants, distribution and differences of health literacy mean scores Variable Number (%) Health literacy (TOFHLA) a Health literacy (STOFHLA) b P Mean SD Mean SD TOFHLA vs. tstofhla c Sex Male 56 (53.3) Female 49 (46.7) p Age, years 44 and under 26 (24.8) (54.3) and over 22 (21.0) p Education Low (8) 13 (12.4) Medium (8 12) 55 (52.4) High (.12) 37 (35.2) p Material status Poor 16 (15.2) Average 73 (69.6) Good 16 (15.2) p Self-perceived health Poor 16 (15.2) Average 55 (52.4) Good 34 (32.4) p Health system utilization No visit to health care 13 (12.4) provider in last year p Health status Any health condition 92 (87.6) p Total a TOFHLA range b STOFHLA range c tstofhla range Table 3: Effect of sociodemographic factors on discordance between the TOFHLA and STOFHLA health literacy categories Variable OR 95% CI p Sex Education Material status Self-perceived health Age been shown to predict knowledge, behaviors and outcomes (Baker, 2006), and studies that use it should not be criticized because it does not offer comprehensive measurements. Nonetheless, additional studies are needed to compare the TOHFLA and REALM with more comprehensive tests to better understand their limitations in research. The TOFHLA and STOFHLA have been translated into Spanish and validated in Spanish-speaking communities. Baron-Epel et al. reported the validation of a Hebrew health literacy test based on the STOFHLA (Baron-Epel et al., 2007). No reports are available on the translations of the English or Spanish versions of the long TOFHLA version and testing its reliability and validity. The Cronbach s alpha reliability coefficient for patients responding to the long English TOHFLA version was 0.96 (0.84 for the numeracy subset and 0.97 for the reading comprehension

6 606 A. Jović-Vraneš et al. subset) and 0.97 for the short TOFHLA version (Nurss et al., 2001). In our research, the Cronbach s alpha coefficients were 0.94, 0.73 and 0.95, respectively, and 0.90 for the short TOFHLA version. In response to the need for a shorter functional health literacy screening tool, the short TOFHLA version was developed. In the original version of the TOFHLA, the short version was correlated (r ¼ 0.91) with the full version (Nurss et al., 2001). According to our results, the Pearson correlation between the TOFHLA and STOFHLA was 0.89, and the association between these two measurements was significant (k ¼ 0.460, p ¼ 0.000; area under the curve ¼ 0.709). In Serbia, there is no gold standard to measure literacy comparable with the tests that are available in English, so our validation cannot make comparisons with other measures in Serbian. Based on data from the literature, the majority of large-scale, cross-sectional health literacy surveys used the short rather than the full TOFHLA version. Our results of health literacy in patients correspond to published results regarding health literacy scores associated with age, sex and education (Gazmamarian et al., 1999; Chew et al., 2004; Wolf et al., 2005; Morris et al., 2006a,b; Wagner et al., 2007; Shieh and Halstead, 2009; Bostock and Steptoe, 2012). Several limitations may restrict the broader application of the present findings. The patient sample used in this study was randomly selected and only reflected one city in Serbia. Therefore, the findings of this study should be limited to primary healthcare centers in Belgrade. Additionally, the primary healthcare patients were outpatients; therefore, we suggest that the TOFHLA be further tested with different populations and settings (i.e. inpatients) and with a larger sample size for future refinement. The assessment of the patients literacy level was not conducted prior to testing health literacy levels. Also, the present cross-sectional study design did not allow us to establish causal relationships among variables. For example, self-perception, compared with actuality, can be very timesensitive and could increase the correlation due to a reporting bias. In conclusion, based on our results, the Serbian translated versions of the TOFHLA may be useful for measuring functional health literacy in primary care patients. There was no difference between the reliability and validity of the short and long TOFHLA versions. The short version is more feasible to administer, whereas the long version can be used for more detailed studies. It is important to have adequate instruments to measure health literacy that provide conceptual and empirical consistency. Research is also needed to identify cultural and age-related issues that intersect with health literacy. Health literacy is a dynamic process that involves both the patient and healthcare provider. Much research has previously focused on patient characteristics, but future research is needed to investigate the relationship between patient/provider characteristics and health literacy. Drawing upon the initial health literacy research and involving the community in programs and projects may be the best formula for success in improving health literacy. FUNDING This work was supported by the Belgrade City Council, Secretary for Health Care and Ministry of Science and Technological Development, Republic of Serbia, Contract No REFERENCES Agency for Healthcare Research and Quality. (2004) Literacy and Health Outcomes. downloads/pub/evidence/pdf/literacy/literacy.pdf (last accessed 1 October 2012). Baker, D. (2006) The meaning and the measure of health literacy. Journal of General Internal Medicine, 21, Baron-Epel, O., Balin, L., Daniely, Z. and Eidelman, S. (2007) Validation of a Hebrew health literacy test. Patient Education and Counseling, 67, Berkman, N. D., Sheridan, S. L., Donahue, K. E., Halpern, D. J. and Crotty, K. (2011) Low health literacy and health outcomes: an updated systematic review. Annals of Internal Medicine, 155, Bland, J. M. and Altman, D. G. (1998) Survival probabilities (the Kaplan-Meier method). British Medical Journal, 317, Bostock, S. and Steptoe, A. (2012) Association between low functional health literacy and mortality in older adults: longitudinal cohort study. British Medical Journal, 344, Chew, D. L., Bradley, A. K. and Boyko, J. E. (2004) Brief questions to identify patients with inadequate health literacy. Family Medicine, 36, Cronbach, L. J. (1951) Coefficient alpha and the internal structure of tests. Psychometrika, 16, Davis, T. C., Long, S. W., Jackson, R. H., Mayeaux, E. J., George, R. B., Marphy, P. W. et al. (1993) Rapid

7 Health literacy screening tool in primary care patients 607 estimate of adult literacy in medicine: a shortened screening instrument. Family Medicine, 25, Gazmararian, J., Baker, W. D., Williams, M. V., Parker, R. M., Scott, T. L., Green, D. C. et al. (1999) Health literacy among Medicare enrollees in a managed care organization. JAMA, 281, Howard, D. H., Gazmararian, J. and Parker, R. M. (2005) The impact of low health literacy on the medical costs of Medicare managed care enrollees. The American Journal of Medicine, 118, Huey-Shys, C., Jiunn-Jye, S. and William, C. (2006) Psychometric testing of the Chinese version of the decisional balance scale (CDBS). Health Education and Behavior, 33, Jones, E. G. and Kay, M. (1992) Instrumentation in crosscultural research. Nursing Research, 41, Kawachi, I. and Berkman, L. F. (1999) Social cohesion, social capital and health. In Berman, L. F. and Kawachi, I. (eds), Social Epidemiology. Oxford University Press, New York. Kickbusch, S. I. (2001) Health literacy: addressing the health and education divide. Health Promotion International, 16, Kickbusch, I., Wait, S. and Maag, D. (2006) Navigating Health. The role of Health Literacy. Alliance for health and the future. International Longeviti Center-UK, London. Morris, S. N., MacLean, D. C., Chew, D. L. and Littenberg, B. (2006a) The single item literacy screener: evaluation of a brief instrument to identify limited reading ability. BMC Family Practice, 7, 21. Morris, S. N., MacLean, D. C. and Littenberg, B. (2006b) Literacy and health outcomes: a cross-sectional study in 1002 adults with diabetes. BMC Family Practice, 7, 49. Nurss, R. J., Parker, M. R. and Baker, W. D. (2001) TOHFLA- Test of Functional Health Literacy in Adults. Peppercorn Books &Press, Snow Camp. Nutbeam, D. (2009) Defining and measuring health literacy: what we can learn from literacy studies? International Journal of Public Health, 54, Parker, R. M., Baker, D. W., Williams, M. V. and Nurss, J. R. (1995) The test of functional health literacy in adults: a new instrument for measuring patients literacy skills. Journal of General Internal Medicine, 10, Patrick, D. L. and Beery, W. L. (1991) Measurment issues: Reliability and validity. American Journal of Health Promotion, 5, Shieh, C. and Halstead, A. J. (2009) Understanding the impact of health literacy on women s health. Journal of Obstetric Gynecologic and Neonatal Nursing, 38, Taylor, W. L. (1953) Cloze procedure: a new tool for measuring readability. Journalism Quarterly, 30, Thomas, B. C., Devi, N., Sarita, G. P., Rita, K., Radmas, K., Hussain, B. M. et al. (2005) Reliability & validity of the Malayalam hospital anxiety & depression scale (HADS) in cancer patients. The Indian Journal of Medical Research, 122, Wagner, C., Knight, K., Steptoe, A and Wardle, J. (2007) Functional health literacy and health-promoting behaviour in a national sample of British adults. Journal of Epidemiology and Community Health, 61, WHO. (2009) Background Note: Regional Preparatory Meeting on Promoting Health Literacy UN ECOSOC. Wide Range Inc. (1993) Wide Range Achievement Test (WRAT 3). Wide Range Inc, Wilmington, DE. Wolf, S. M., Gazmarian, A. J. and Baker, W. D. (2005) Health Literacy and functional status among older adults. Archives of Internal Medicine, 165,