OF HEALTH PRACTITIONERS

Transcription

1 MEASURING CULTURAL AND LINGUISTIC COMPETENCY OF HEALTH PRACTITIONERS by SONJA HARRIS-HAYWOOD, MD, MA Submitted in partial fulfillment of the requirements For the degree of Master of Science Clinical Research Scholars Program School of Medicine CASE WESTERN RESERVE UNIVERSITY May, 2015

2 CASE WESTERN RESERVE UNIVERSITY SCHOOL OF GRADUATE STUDIES We hereby approve the thesis of Sonja Harris-Haywood, MD, MA Candidate for the degree of Master of Science * Committee Chair Susan Flocke, PhD Committee Member Ashwini Sehgal, PhD Committee Member Steve Zyzanski, PhD Date of Defense January 6, 2015 * We also certify that written approval has been obtained for any proprietary material contained therein. 2

3 Acknowledgements Susan Bronheim, Tawara Goode, Wendy Jones, Mary Beth Kavanagh, Laurie Murphy, Shari Snyder Pollack. 3

4 Table of Contents List of Tables... 5 List of Figures... 6 List of Abbreviations... 7 Abstract... 8 Background and Significance... 9 Hypothesis/Specific Aims Methods Study, Sample and Data Collection Procedures Data Analyses Results Factor Correlations Validity and Reliability Discussion Limitations Significance References

5 List of Tables Table 1: CCHPA Domain Index Table 2: Characteristics of Practitioners Table 3: Estimates of Item Difficulty, SE, Fit Statistics Item-Total Correlations Knowledge of Diverse Patient Populations Table 4: Estimates of Item Difficulty, SE, Fit Statistics Item-Total Correlations Adapting Health Care for Diverse Patients Populations Table 5: Estimates of Item Difficulty, SE Fit Statistics Item-Total Correlations Promoting the Health of Diverse Communities Table 6: Associations of Factor Scores and Demographics Table 7: Factor Scores by Training Table 8: Pearson Correlations Between Factors 1, 2 and 3 Scores

6 List of Figures Figure 1: Wright-Item Person Maps for Factors 1, 2 and

7 List of Abbreviations CLC - Cultural and Linguistic Competency, CCHPA - Cultural Competency Health Practitioner Assessment, DIF - Differential Item Function, NCCC - National Center for Cultural Competence. 7

8 Measuring Cultural and Linguistic Competency of Health Practitioners By SONJA HARRIS-HAYWOOD Abstract With a growing body of evidence describing Cultural and Linguistic Competence (CLC) as effective in deceasing disparities in health care and health outcomes it is necessary for health care professionals to demonstrate CLC when caring for patients. To determine their level of CLC it is essential that practitioners have a valid and reliable measure. Responses from two thousand five hundred and four practitioners, including nurses (RN, LPN), clinicians (PA/NP), and physicians (MD/DO), on the 129 item Cultural Competence Health Practitioner Assessment (CCHPA) were analyzed using classical test theory, Rasch modeling and differential item functioning to determine its psychometric properties. Three factors representing domains of Knowledge, Adapting Practice, and Promoting Health for culturally and linguistically diverse populations accounted for 46% of the variance. Among Knowledge factor items, 53% (23/43) fit the Rasch model, item difficulties ranged from logits (least difficult) to logits (most difficult), item separation index (SI) 13.82, and Cronbach's α Forty-seven percent (21/44) Adapting Practice factor items fit the model, item difficulties to logits, item SI 11.59, Cronbach's α 0.88;; and 58% (23/39) Promoting Health factor items fit the model, item difficulties to logits, item SI 22.64, Cronbach's α Scores for practitioners with CLC training statistically differing from practitioners without CLC training established early evidence of validity. The 67-item CCHPA-67 is a 8

9 psychometrically sound self-assessment tool. Immediate next steps are adapting to measure the CLC of other health professionals and linking CLC levels to patient outcomes. Background and Significance Many entities responsible for mandating quality measurements for health care organizations and practitioners are exploring ways to implement standards for cultural and linguistic competence. For instance, the Joint Commission on the Accreditation and Health Care Organizations and the National Quality Forum have recently enacted initiatives to develop standards for incorporating cultural and linguistic competency into the care of diverse patient populations 1,2. Additionally, research funded by The Robert Wood Johnson Foundation revealed that 14 states have either introduced or enacted legislation mandating the integration of cultural and/or linguistic competence into curricula, continuing education, and licensure requirements for health and mental health professionals 3. As more national regulatory and accreditation organizations and state legislatures develop policies that require culturally and linguistically competent practices in health care, there is a need for measures that can quantify practitioner competency in these areas. Currently, organizations and providers are seeking psychometrically sound measures to evaluate the knowledge and skill necessary to provide high quality, culturally and linguistically competent health care to diverse patient populations. A recent review of measures evaluating cultural and linguistic competency curricula for health professionals revealed that only 13% provide acceptable evidence validity or reliability, thereby limiting the effectiveness of the majority of these measures 4. Additionally when 9

10 reviewing cultural competence measures for providers, Kumas-Tan et al. noted that most measures focused primarily on attitudes and knowledge, thus assuming that changes in these domains would result in higher levels of culturally competent care 5. None of the reviewed measures by Kumas-Tan et al. specifically addressed underlying issues, including social inequities, power differentials between provider and patient, and the skills or behaviors needed to practice effectively across culturally and linguistically diverse patient populations. The Cultural Competency Health Practitioner Assessment (CCHPA), an instrument developed by the National Center for Cultural Competence, evaluates practitioners attitudes, knowledge, skills and awareness of social inequities and power differentials required for providing culturally competent care. For example, the CCHPA assesses social inequities by inquiring about practitioners awareness of the influence of income, education, and neighborhood resources on patients patterns of health behavior. The CCHPA assesses power differential by asking questions about participation in training that enables them to about confront bias, discrimination and racism in health and social service systems, and whether information is provided to support patients and their families by providing information that enables advocacy on their own behalf when experiencing discrimination. While the CCHPA development involved focus groups with national content experts in CLC, piloting testing with clinicians caring for racially diverse patient populations and a national sample of over 8000 health professionals, it lacked formal evaluation of validity and reliability. Additionally, the length of the questionnaire is a barrier to completion. Therefore, this study seeks to evaluate the properties of the 10

11 CCHPA by using factor analysis and Rasch modeling to analyze responses of a large sample of nurses, physicians, and physician assistants to produce a shorter reliable and valid version of the CCHPA that can be modified to assess CLC of other health professionals. Hypothesis/Specific Aims The 129-item CCHPA can be shortened but still remain a, valid, and reliable survey to measure the competencies of physicians, nurses, and physical assistants. Current measures lack validity and reliability properties confirmed on a large sample of physician, clinicians and nurses. Primary Aim: To use factor analysis, Rasch modeling and differential item functioning to shorten the CCHPA and then to determine the measure s reliability and validity. Secondary Aim: Use To seek opportunities to further refine the CCHPA to address item gaps that were identify in the Rash modeling and differential item function analyses Tertiary Aim: To seek opportunities to adapt the CCHPA to measure CLC of other health professionals and link the CLC to patient outcomes. Methods Cultural Competency Health Practitioner Assessment (CCHPA) Description The Cultural Competency Health Practitioner Assessment (CCHPA) is a webbased self-assessment instrument created by the National Center of Cultural Competence (NCCC) at the Georgetown University Medical Center. The CCHPA was developed in three stages: stage one instrument development, stage two establishment of 11

12 instrument content validity, stage three instrument focus groups. Each of these stages was conducted by the NCCC and is briefly described here. The work of this thesis begins with the analysis of over 8 thousand of health professionals who completed the survey between 2005 and Stage 1 Literature Review: In 2000, the faculty from the National Center of Cultural Competence (NCCC) and consultants conducted a comprehensive review of both the literature and any existing instruments related to assessing cultural competence of health care practitioners. During this process, instruments and literature from the Office of Minority Health, the NCCC and over 400 Medline citations were examined. Based on this effort, the Mason 1995 Cultural Self-Assessment Questionnaire and the Cultural and Linguistic Competence Policy Assessment (CLCPA), developed by the NCCC were used as the basis for the development of the CCHPA. Stage 2 Item Development and Establishment of Content Validity: In 2001 an expert group consisting of faculty, senior consultants, and staff from the NCCC and Bureau of Primary Health Care (BPHC) developed the initial draft of the CCHPA by modifying CCSAQ items and scales to make them relevant to assessing practitioner cultural competence. This initial draft was critically reviewed by members from the national medical professional organizations (i.e.: National Association of Community Health Centers (NACHC), American Medical Association (AMA), National Hispanic Medical Association (NHMA), university academic programs, and the BPHC) to determine the relevancy of the items and scales to the assessment of health care practitioner cultural competence. A pilot version was developed after several iterations. 12

13 The pilot version of the CCHPA consisted of 138 items, representing 6 subscales: Values & Belief Systems, Cultural Aspects of Epidemiology, Clinical Decision-Making, Life Cycle Events, Cross-Cultural Communication, and Empowerment/Health Management (Table 1). Items in each of the 6 subscales were scored on one of three 4- point Likert scales; scale 1 ranged from 1(not at all) to 4(very well), scale 2 ranged from 1(never) to 4(regularly), and scale 3 ranged from 1 (not at all) to 4(very often). A summary score for each of the 6 subscales, tabulated from the sum of individual items within a subscale, was generated. A lower subscale score indicated an area for improvement, with supplemental readings suggested to respondents to improve their skills in that particular subscale area. Stage 3 Pilot Testing: The 138-item pilot CCHPA was tested using focus groups to acquire feedback on the comprehensibility of the CCHPA from healthcare providers. Eight adult health care practitioners were recruited from 1 of 6 community health centers across the nation serving a diverse patient population. Feedback from the focus groups was utilized to refine the wording and format of the CCHPA items. The 129-item version of the CCHPA was put up on the web in It is this version of the CCHPA that was the focus of the psychometric evaluation for this study. Study, Sample and Data Collection Procedures Between January 2005 and May 2008, 8605 participants voluntarily completed the CCHPA online by accessing the NCCC website. Of those 8605 participants, 2504 identified themselves as physicians (MD/DO), clinicians including physician assistants (PA) and nurse practitioners (NP), or nurses (RN, LPN). Responses were downloaded directly from the NCCC website into a SPSS database. All responses were anonymous 13

14 with no identification of the participant. This thesis focuses on the analyzing the responses of 2504 self-categorized clinicians, physicians, and nurses. Data Analyses A multi-step data analysis process applying an exploratory factor analysis, Rasch modeling and differential item functioning (DIF) analysis methods was used to develop a shorten version of CCHPA. 1) Exploratory Factor Analysis using a principal axes factor analysis with both varimax and oblique rotations on the raw scores of the initial 129 items was conducted to identify the major factors for CCHPA. The number of factors to retain was determined by examining the scree plot, eigenvalues and variance explained by each factor only items with loadings >0.40 were retain for each factor. Three factors were determined suggesting three conceptual domains, 2) Rasch analyses were conducted separately on each factor. For each factor, misfit items were removed and Rasch analysis was rerun. The process was repeated until Rasch analysis results showed all remaining items in a factor exhibited good model fit. 3) DIF analysis was then performed for the items in each subscale after removing misfit items to further identify items that functioned differently in different gender, race/ethnicity, and professional affiliation groups. Bonferroni adjustment was used to control potential inflation of α from multiple comparisons of each item 6. Item selection and elimination was performed for each factor to develop indicative subscales. Items were evaluated by examining the magnitude of their factor loadings, the item-total correlation, Rasch model item fit statistics and DIF. 14

15 Items with low factor loading (<.4) or low item-total correlation (<.3), misfit items or DIF items were deleted from the final version of the CCHPA subscales. The final decision to remove an item, based on DIF was made by the content experts on the study team. Description of Rasch Modeling and Differential Item Functioning (DIF) Rasch modeling is an advanced modern measurement theory model that has been successfully used in healthcare when a measure is thought to assess a single construct (i.e., unidimensionality) 7,8. Rasch modeling models a probabilistic relationship between a person s ability and his/her response to an item in a measure and has several significant advantages over traditional classical test theory methods 9,10. In Rasch modeling, both people and items are placed on the same common metric that is then used to evaluate person s ability and the difficulty levels of the items. The person s ability here is a general term used in Rasch modeling to describe the latent trait to be measured. People have a higher probability of responding favorably to easier items, while they have a lower probability of responding favorably to hard or severe items. Linking item difficulty level to people s ability in Rasch modeling allows the examination of whether people are responding to a measure in a consistent, logical manner. Rasch goodness-of-fit statistics (e.g., Infit and Outfit statistics) are typically used in practice to determine if people respond to a measure as expected by the model (i.e., if the data fit Rasch model well), with Infit and Outfit statistics values close to 1 being considered satisfactory, while values greater than 1.3 or less than 0.7 being considered a misfit 11. Infit and Outfit values greater than 1.3 indicate inconsistent performance while values less than 0.7 show too 15

16 little variation. These statistics allowed for the identification of good and poorly fitting items to the unidimensional model. Differential Item Function (DIF) refers to unequal probabilities of getting an item endorsed between two groups of respondents at the same ability (e.g., cultural competency) level 12. In Rasch modeling, DIF implies that the item difficulty is different for different groups. The presence of DIF indicates an item in a measure may be biased toward a group, which could threaten the validity of the measure and produce misleading results 13. In this study, DIF attributable to race/ethnicity, gender and profession affiliations were assessed. Items presenting with moderate DIF (DIF size >.43 logits) or large DIF (DIF size >.63 logits) were deleted from the final version of CCHPA subscales after careful review by the content experts. Third, when the items for each factor (subscale) were selected, the suitability of the 4-point Likert response format used in the final subscales was assessed through an examination of the distribution of responses for each item and the presence of item thresholds. If the response format is appropriate, all items should have ordered response thresholds, that is consecutive thresholds are expected to demonstrate an increase along the underlying trait being measured. If not, thresholds are said to be disordered and combining response categories is usually recommended 14. Finally to determine the relationship among the factors, a Pearson correlation was performed. Validity and Reliability of the Final CCHPA. Validity of the final CCHPA subscales was examined using multiple sources of information. Practitioner scores for each subscale was determine by adding up the numbers associated with their responses. Lowest score in range for each subscale was determined by adding lowest possible score 16

17 (1) for each item and the highest by adding the highest possible for item (4 or 5). In Rasch modeling, good item fit statistic and DIF information provide evidence of construct validity. An instrument without any DIF items > 0.43 logits further support the construct validity 12,13. In addition, the item-person map, which displays the relative position of respondent ability levels and item difficulties on the same common metric, was examined for item-person overlap, item difficulty spread (range), and item difficulty gapping. Item-person overlap on the common metric is an indication of how well the items discriminate and measure the concept among the respondents. Item location is also assessed for item gapping, which is a large distance between items on the metric. The presence of large gaps along the metric indicates that the measure s ability to discriminate between peoples abilities is low at these points. Rasch Person (and item) separation reliability statistics were used to evaluate the internal consistency of the final CCHPA subscales. Like traditional reliability estimation, Rasch separation statistics are also calculated as the ratio of true measure variance to observed measure variance, with a range of 0 to 1. The separation statistics indicate the range of ability scores being measured or the spread of the item difficulty levels along the common metric. The higher the person separation index, the better range of ability scores; similarly, the higher the item separation, the better the spread of items from least difficult to most difficult. The person reliability statistic is equivalent to the traditional Cronbach s alpha, which is an indicator of test reliability, except Cronbach s alpha is calculated from raw test scores, while Rasch reliability is calculated from the logits estimated from the model. Item separation, which has no traditional equivalent, is usually explained in the same manner as person separation. Item separation depicts the level of confidence that the items would 17

18 have the same respective order among another sample of participants. Reliability statistics close to 1.00 represent excellent internal consistency, indicating that an instrument covers a useful range of item difficulty that can be appropriate for measuring persons with a wide range of the ability being measured (e.g., cultural competency in this study) 8. Rasch and DIF analyses were performed using WINSTEPS 3.69 software. Rasch rating scale model was used (reference). The item difficulty levels were reported for the final CCHPA subscales. To determine if cultural competency scores differ by gender, age, race, profession, ethnicities and years since employer sponsored training, descriptive and inferential statistics including Chi Square and ANOVA were conducted using SPSS 18. The comparisons between groups that should differ on cultural competency (e.g., practitioners with no employee sponsored training on cultural competency versus those who had any employee sponsored training) were conducted to demonstrate discriminate validity of the final CCHPA. Results Sample Characteristics Out of the 2504 respondents (Table 2), the majority were females (87%), and most were less than age 50 (81%). Additionally, 73% self-identified as white with 9% black and 6 % Asian. Approximately seventy-five percent of the respondents were nurses, 14% clinicians, and 12% physicians. Across the sample, over 87% participated in employer-sponsored cultural competency training. 18

19 Item Reduction The factor analysis supported three major factors accounting about 46% of the total variance in the data with factor 1 contributing 33.4% and Factor 2 and Factor 3 contributing less with 7% and 5.2 % respectively. Based on the content of the items, factors were labeled: knowledge of culturally and linguistically diverse populations (Factor 1), adapting practice for culturally and linguistically diverse patient populations (Factor 2), and promoting the health of culturally and linguistically diverse communities (Factor 3). Nine items had Factor loadings less than 0.4 for each of the three factors. Removing these items resulted in 43 items in Factor 1, 44 items in Factor 2 and 39 items in Factor 3. Note that each factor included a small number of items loaded on more than one factor: two items were included in both Factors 1 and 3; four items were included in both Factors 2 and 3. Knowledge of culturally and linguistically diverse populations (Factor 1). Two misfit items were identified in Factor 1 using Rasch item misfit statistics and measurement redundancy. After removing the two misfit items, subsequent DIF analysis was conducted and the results revealed that approximately 18 of the items in Factor 1 exhibited significant DIF across race/ethnicity, gender or profession, with a majority exhibiting moderate DIF (DIF size >.43 logits) and two items exhibited large DIF (DIF size >.63 logits). Specifically, 3 items had DIF for Whites versus Blacks, 8 for Whites versus Asians, 6 for Blacks versus Asians; 4 for gender; 1 for physicians versus clinicians, 5 for physicians versus nurses, and there was no items for clinicians versus nurses. Two items that presented large DIF in the comparison between Blacks versus Asians were I know the impact of poverty on the health and well-being of the 19

20 communities I serve and I know factors affecting the general health status of people in the country of origin of immigrant populations. After discussing the pros and cons of dropping the items with moderate to large DIF the study team agreed to eliminate these items. The final version of Knowledge (Factor 1) included 23 items with good item model and no presence of DIF for race or provider. The estimated item difficulties (in log-odds units, reported as logits ) for the items in the final version of Factor 1 ranged from logits (least difficult) to logits (most difficult) with a mean of 0.00 and standard deviation of 0.54 and the average item-total correlations was 0.7. Table 3 shows the estimated item difficulties with standard errors, Infit and Outfit statistics and item-total correlations from Rasch analysis of the final version of Factor 1. Further evaluation of Factor 1 items with content experts, revealed 7 items measuring knowledge-related cultural competency specific to community service while other items were assessing general knowledge of cultural competency. The scoring range was Delivery of culturally competent care (Factor 2). Fifteen misfit items were identified in Factor 2. After removing the misfit items, the subsequent DIF analysis revealed 19 items in Factor 2 exhibited significant DIF across race/ethnicity, gender, and profession, with 17 presenting moderate DIF and two items exhibited large DIF. Specifically, one item had DIF for Whites versus Blacks, 2 for Whites versus Asians, 2 for Blacks versus Asians; two items presented for gender DIF; no item had DIF for physicians versus clinicians, but six DIF items were identified for physicians versus nurses, and six DIF items for clinicians versus nurses. Two items exhibited large DIF in the comparison between physicians versus nurses and they were I integrate the inclusion 20

21 of extended family members in treatment/discharge planning and I counsel for risk factors based on gender exhibited large DIF in for physicians versus nurses. After careful review by the content experts described above, these DIF items were dropped. Evaluation of Infit and Outfit statistics for the remaining 21 items showed that fit values were less than 1.3 and greater than.7, with item-total correlations greater than.6 and no presence of DIF. The estimated item difficulties for the items in the final version of Factor 2 ranged from -.7 logits (least difficult) to logits (most difficult) with a mean of 0.00 and standard deviation of 0.41 and the average item total correlation was 0.66 (Table 4). The scoring range was Engagement in community based culturally competent services (Factor 3). Nine misfit items were identified in Factor 3 based on Rasch Infit and Outfit statistics. After removing these misfit items, the subsequent DIF analysis indicated there were 10 items that exhibited significant DIF attributed to race/ethnicity, gender and profession, with most items exhibit moderate DIF and only one item with large DIF. Specifically, two items had DIF for Whites versus Blacks, one for Whites versus Asians, one for Blacks versus Asians; no gender DIF item; only one item for physicians versus clinicians, three were identified for physicians versus nurses, and two for clinicians versus nurses. These 10 DIF items were dropped after careful content review. The final version of Factor 3 had 23 good fitting items. The estimated item difficulties for the items in the final version of Factor 3 ranged from logits (least difficult) to logits (most difficult) with a mean of 0.00 and standard deviation of 0.67 and the average items total correlation was 0.66 (Table 5). Scoring range was

22 Factor Correlations The Pearson correlations of Factor 1 with Factors 2 and 3 were and respectively and the highest correlation was between Factor 2 and 3 (r =.665) (Table 8). Validity and Reliability Construct Validity. Fit statistics generated by Rasch analysis are used to determine the quality of items in a measure. Under Rasch model expectation, people with lower ability would obtain lower scores and people with high ability would have high scores on any item. An item will be defined as a misfit item if high ability people tend to score lower on the item, whereas low ability people tend to score higher on the same item. So a misfit item really means that the item does not fit within the single underlying construct that an instrument is intent to measure. Therefore, if all items in a measure are good fit items, it provides evidence of construct validity of the measure 11. In this study, according to fit statistics, all of the items in each of CCHPA final subscales were found to be fit (See Tables 3-5), which suggests that for each subscale, the items were appropriately defined and were measuring a similar construct. Moderate to large DIF items attributed to gender, race/ethnicity, and professional affiliation were removed from the final version of each subscale. The fact that each CCHPA subscale includes no DIF item indicated that each subscale holds measurement invariance across different samples, which further support the validity of the CCHPA. Reliability. The Rasch person separation index was 3.95 for the final version of Factor 1, 2.69 for Factor 2 and 3.48 for Factor 3, respectively. The above person separation indexes translate to person reliability coefficients (equivalent to the traditional Cronbach s alphas) of.94 for Factor 1,.88 for Factor 2 and.92 for Factor 3, indicating 22

23 each of the CCHPA Factors has high internal consistency. The Rasch item separation index was for Subscale 1, for Subscale 2 and for Subscale 3, which represent the item reliabilities of.99 to 1. Although item separation index has no traditional equivalent, it is usually explained in the same manner as person separation index. The item reliability of.99 provides evidence that the items have good variability along the measurement scale, indicating a high degree of confidence that in another sample of participants, the items would be in the same order. Item-person maps. The item-person maps shown in Figure 1 display the location and distribution of both the items in each factor with good fit statistics and the practitioners competency on the same common logit metric. On each map, the persons with higher ability and most difficult items were place on the top of the map, and the numeric scale is shown on the left with persons and items charted on the right side. For Factor 1, the map (on the left panel) shows that most items ranged from to 1.11 logits on the common scale while person ability ranged from to 7.79 logits, suggesting that knowledge of culturally and linguistically diverse populations was assessed well by Factor 1 items when the person ability was in the range of 1.01 to 1.11 logits, but the ability estimates would be less accurate for lower or higher competent people. Lack of items targeting people at both the lower and higher ends on the common scale also suggests there is a need to develop and include easier and more difficult items in the measure. I tem-person map for Factor 2 (on the middle panel) revealed that items on Factor 2 were too easy for most of the respondents and more difficult questions are needed to better target the population. For Factor 3, the map (on the right panel) indicate a good match between the distribution of items (item difficulty) and the sample of the 23

24 participants (person ability), suggesting community engagement-related cultural competency of the majority of the participants was assessed well by items in Factor 3. Response Format Assessment. The appropriateness of the 4-point orderedresponse format (e.g., 1=Not at all/never to 4=Very well/regularly ) used in this study was assessed. If appropriate, each category of such ordered-response format should have well-defined boundaries and values representing each category should reflect the magnitudes of the underlying ability/trait being measured 14. For example, in this study, a selection of category 2 ( Barely/Seldom ) on an item should demonstrate higher practitioner s cultural competency than a selection of category 1 ( Not at all/never ). Rasch analysis of the final version of the CCHPA factors showed both the average measures and Rasch threshold estimates were ordered for each item. Such orders demonstrated an increase of cultural competency among consecutive categories, indicating the response format used in the instrument is appropriate. Discriminant Validity. When analyzing mean factor scores by respondent demographics for Factor 1, there were statistically significant differences in scores by race, ethnicities and profession (p=.001) (Table 6). When compared to Caucasians (63.8), African Americans (69.1) had higher scores followed by Asians (65.0) with effect sizes 0.42 and 0.31 respectively. Additionally, nurses (64.0) scored lower than physician (65.7) and clinicians (66.50) but the effect size was small (0.13 and 0.20 respectively). For Factor 2 the scores differ by gender and ethnicities and differences were significant (p=.001). Females (67.8) scored higher than males (65.1) with an effect size = 0.26) Factor 3 had statistically significant differences in scores for age (p=.001), race and ethnicities (p=.003) and profession (p=.001). Respondents age (58.6) had the 24

25 highest score while those age (51.7) had the lowest with an effect size of 0.43) Asians (56.8) than of African American (55.3) and Caucasian (53.0) but the effects sizes were small (.25 and.10 respectively). Clinicians score higher that Nurses with an effect size (.47) in moderate range. For all three factors, respondents with no training had statistically significant lower scores than respondents with any training, with small effect sizes for Factor 1 and Factor 2 (.19 and 0.34 respectively), and Factor 3 (0.49) with a moderate effect size. These factor correlations are consistent with the sequential nature of the factors. Implementation into practice and engaging the community is predicated on a practitioner s knowledge of cultural and linguistic competency. Additionally, knowledge of how to implement the skills into practice precedes engaging the community in a cultural and linguistically competent manor. Discussion The analysis presented in this paper, confirms the study hypothesis. The CCHPA was modified to a shorten statistically reliable measure of cultural competency with evidence of discriminant and content validity. Reliability was confirmed by each of the 3 factors measuring different aspects of cultural competency, having Cronbach alpha greater than 0.85, item separation scores ranging from and excellent fit of the practitioner-response data to the Rasch model. The Rasch model assumes that the level of cultural competency can accurately be determined by an item s response and difficulty and that competency level is independent of gender, age, race, and profession. The person and item fit statistics provided evidence that practitioners were responding to the items in a consistent manner and that these items had some variability and precision within the observed range of most-to-least-difficult. These fit statistics demonstrate 25

26 moderate differences in levels of cultural competency with minimal redundancy among the items. The CCHPA is based on the Goode adapted Cross model. This model assumes that developing practitioners cultural competence acquires obtaining cultural knowledge and skills, and adapting practice and care to the cultural contexts of the patients, families and communities served 15. The three correlated factors of the CCHPA: knowledge of cultural and linguistic competence (Factor 1), delivery of culturally and linguistic competent care (Factor 2), and engagement in community based cultural and linguistic competent services (Factor 3) are consistent with this model. This consistency strengthens the assessment potential of the CCHPA and has implications for future inventions. It is important to note that this study was not designed to evaluate whether the factors are sequential with regard to skill cultural competency skill attainment. We might spectate that developing interventions to strengthen ones knowledge of cultural and linguistic competency (Factor 1) are likely to benefit implementation in clinical practice (Factor 2) engaging the community (Factor 3) and that innerving at any level (factor) is likely to translate into better patient care at the next stage. However, future studies are needed to confirm. The CCHPA also meets test content and discriminant validity, two of the validity criteria established by the Standards for Educational and Psychological Testing 16. The test content validity is based on the expertise of the instrument developers. A group of national experts in cultural and linguistic competency engaged in an intensive iterative process, including focus groups and pilot testing to develop the instrument items. Discriminant validity was determined by the significant lower mean cultural competency 26

27 scores for all 3 factors between individuals without employer-sponsored cultural competency training. While the effects size was small for Factors 1 and 2 and modest for Factor 3 this association does provide early evidence of the CCHPA s ability to discriminate between individuals with and without training. Difference in Factor scores by race, gender and profession although did not support discriminant validity show sensitivity of CCHPA to group differences. African Americans and Asians had higher mean scores on Factor I and nurses had lower mean scores on Factor 3. It is possible that African-Americans and Asian knowledge of cultural heighten by being a minority in the United States and nurses training in community engagement differ from most clinicians and physician. This study established that the current CCHPA provides a sound basis for further research to establish it as an instrument for measuring cultural competency for research and evaluation. Although, this study provides some evidence that provider race and ethnicity have small associations with or a predictor of higher levels of cultural competency, however the moderate effect size would warrant future studies to substantiate. Therefore, cultural and linguistic competency training and professional development should still focus on all practitioners independent of race to increase the knowledge and skills needed to provide care to patients from racially and ethnically diverse backgrounds. All practitioners need the capacity to address social inequities and power differentials inherent within diverse populations. Additionally, new items in the moderate to difficult range are needed in Factors 2 and 3 to increase the ability to distinguish between practitioners with moderate and high competency levels. Also new items measuring concepts of linguistic competence as defined by Goode and Jones that 27

28 were not evaluated in this version of the CCHPA, will be added in a future version of the instrument. These new items will assess linguistic competencies in communicating with individuals who have cognitive or other disabilities and various levels of health literacy. Limitations Although other measures of cultural and linguistic competency have been developed, few have demonstrated this level of reliability and validity with a large sample. However, the major limitation of this study is the self-selection of the practitioner sample, making it likely that many providers in the sample have a strong interest in cultural and linguistic competency. Therefore, the results could be biased by over-representing individuals having a stronger interest and possibly more knowledge about cultural and linguistic competency. Additionally, an evaluation of this measure with a more representative sample of practitioners is warranted. Future studies should also focus on refining the measure. Specifically, these studies should include analyses to identify the most effective items, to establish test-retest reliability, and to assess sensitivity to change and to test association with patient outcomes. Significance The current version of the CCHPA can be useful to individuals seeking selfassessment tools to measure their level of cultural competence. This self-assessment instrument, with demonstrated psychometric properties, enables health care providers to identify areas for improvement or professional and clinical growth and to develop individualized learning plans to incorporate cultural competence into practice 17. Research has shown that educational interventions for healthcare providers can work to improve knowledge about the impact of culture on the patient- provider encounter and 28

29 cultural specific knowledge 18. In addition, as accreditation bodies for health care settings and education programs require that practitioners develop the knowledge and skills needed for cultural competence. The CCHPA is a psychometrically sound instrument for identifying learning needs, guiding curriculum choices and supervision, and supporting mentoring opportunities in both pre-service and continuing education. As cultural and linguistic competency emerges as a marker for quality and equitable care, the CCHPA has the potential to offer valid and reliable evidence of practitioners knowledge and skill. Our current study evaluating the CLC for a national sample of over four hundred physicians caring for patients with CIGNA health insurance, will provide an opportunity to further refine the CCHPA and address item gaps that were identify in the Rasch modeling and differential item function. (Aim 2) In this study were we are testing new items in the moderate to difficult range for Factors 2 and 3 as well as items evaluating linguistic competency. We are also seeking to use the CIGNA project to link the CLC to patient outcomes. (Aim 3) by establishing associations between practitioners cultural and linguistic competency and health outcomes for racially and ethnically diverse populations. Evidence is emerging linking cultural competency care to improved patient outcomes in diabetes and hypertension CCHPA could help determine if and how the provider s cultural competency is impacting care and affecting patient outcomes. We are also planning to adapt the CCHPA to assess the cultural and linguistic competency of other health providers. A preliminary study adapting the CCHPA to assess the cultural and linguistic competency of nutritionist is in progress. Also a future project developing a patient measure of CLC based on the CCHPA is in process. 29

30 Improving the level of cultural and linguistic competency increases the quality of care and patient outcomes for racial and ethnically diverse patient populations 22. Researchers are exploring how physician CLC affects the relationship with patients and improves clinical outcomes 23. Some studies suggest a positive relationship between patient satisfaction and trust and the level of provider CLC Others believe that CLC is a dimension of patient centered care 27. Many studies are exploring the relationships of patient assessments of physician CLC to patient outcomes 28. As researchers seek to understand the underlining mechanisms of how CLC improves care, having a valid reliable measure, like the CLCHPA will be an asset, allowing us to move forward in the fields of health disparities and measurement science. 30

31 Table 1. CCHPA Domain Index Domains Values and Belief Systems Cultural Aspects of Epidemiology Practitioner Decision-Making Life Cycle Events Cross-Cultural Communication Empowerment/Heal th Management Concepts Being Measured Assesses health practitioners knowledge of values and belief systems of diverse cultural groups and their impact on healthcare access and utilization. Assesses health practitioners knowledge of cultural, environmental, and related etiologic factors that contribute to diseases. Assesses the health practitioners knowledge of culturally defined health beliefs and practices, and the ability to integrate this knowledge into approaches to health care delivery. Assesses the health practitioners knowledge of the cultural implications of various stages of life and life cycle events, and the ability to address them in approaches to health care delivery. Assesses the health practitioners knowledge and skills in communicating with culturally and linguistically diverse groups as it relates to health care access and utilization. Assesses practitioners role in providing information that enables individuals to intervene on their own behalf, advocate and build community capacity for improved health. # of items total domains 129 total Items Note. This table represents all of the 129 items of the CCHPA. Fit statistics determined that 119 of the items were acceptable. All subsequent psychometric analyses were done using the 119 items from the CCHPA that had acceptable fit statistics. 31

32 Table 2. Characteristics of Practitioners Characteristics of Practitioners n* % Gender Female Male Age ** Race < >60 White Black Asian Ethnicity Hispanic, Latino or Spanish origin Non- Hispanic, Latino or Spanish origin Profession Nurse (RN, LPN, BSN) Clinician (P.A./N.P.) Physician (M.D./D.O.) Years since employer sponsored cultural competency training 0 3 years 4 6 years 7 10 years more than 10 years no training Note. *Due to missing data not all responses sum to the total of n=2504 ** Due to the small sample size of n = 40 respondents who identified as Pacific Islander, Native American/Alaskan Native, Latino, and Other were categorized as missing and are not included in the analysis 32

33 Table 3. Estimates of Item Difficulty, Standard Error (SE), Fit Statistics and Item-Total Correlations Knowledge of Diverse Patient Populations Factor 1 Abbreviated item content Item Difficulty 1 SE 2 IN.MSQ 3 OUT.MSQ 4 PTME Q6 Impact of family on health care Q4 Impact of religion on health care Q7 Knowledge of strengths/resiliencies of patients q2a Know diet patterns based on income q1a Know health beliefs of patients q8o Know limited English proficiency impacts health of communities q1b Know illness beliefs of patients q2b Know diet patterns based on education q2d Know diet patterns based on cultural preferences q8j Know unemployment impacts health of communities q1c Know wellness beliefs of patients q3b Know exercise patterns based on education q1d Know help-seeking behaviors of patients q8e Know mental health impacts health of communities q2c Know diet patterns based on neighborhood resources q3a Know exercise patters based on income q8c Know sexual abuse/trauma impact health of communities q3c Know exercise patters based on neighborhood resources q8l Know homelessness impacts health of communities q8f Know oral health impacts health of communities q8p Know environmental factors impact health of communities q1f Know patient preference for traditional healers q1g Know patient preference for traditional healing practices Standard Error (SE), 2 In.MSQ -Infit 3 Out.MSQ Outfit Item, 4 PTME Item Total Correlation 33

34 Table 4. Estimates of Item Difficulty, Standard Error (SE), Fit Statistics and Item-Total Correlations Adapting Health Care for Diverse Patients Populations Factor 2 Abbreviated item content Item Difficulty 1 SE 2 IN.MSQ 3 OUT.MSQ 4 PTME q30c Modify treatment/discharge plans for disabled patients q30a Modify explanation of diagnosis/results for disabled patients q30b Modify explanation of treatment options for disabled patients q30d Modify referrals and resources for disabled patient q31c Ask patients their likelihood of treatment adherence q31b Ask patients what may improve the illness better q15c_1 Integrate information from family in diagnostic protocols q17b Include cultural needs in treatment/discharge plans q32a Written information appropriate for cultural/linguistic needs q26i Modify health information to meet patient s literacy needs q31a Ask patients what they think caused their illness q26h Treatment /discharge plans in patient s preferred form q15b_1 Integrate patient s belief of illness cause in diagnostic protocol q26g Modify health education to meet needs of diverse patients q26e Incorporate culturally-appropriate non-verbal communication q32b Written information appropriate for literacy needs q17e Include cultural preferences for referrals in treatment/discharge plans q17c Include group strengths/resiliencies in treatment/discharge plans q15a_1 Integrate patient s health beliefs & values in diagnostic protocol q17a Include beliefs about healing in treatment/discharge plans q118 Integrate information about cultural strengths and resiliencies Standard Error (SE), 2 In.MSQ -Infit 3 Out.MSQ Outfit Item, 4 PTME Item Total Correlation 34

35 Table 5. Estimates of Item Difficulty, Standard Error (SE), Fit Statistics and Item-Total Correlations Promoting the Health of Diverse Communities Factor 3 Abbreviated item content Item Difficulty 1 SE 2 IN.MSQ 3 OUT.MSQ 4 PTME q21 Stay current on factors contributing to health disparities research q33c Assist patients with obtaining financial resources for services q34 Provide information so patients can advocate if they experience racism q33d Integrate information about cultural strengths, assets, and resiliencies q20c Keep current on race/ethnicity affecting drug metabolism q24d Consider family s cultural beliefs during transition to adulthood q24c Consider family s cultural beliefs during transition to adolescence q35a Advocate through professional/discipline associations q19 Stay current use of medicinal plants and minerals q24b Consider family s cultural beliefs during middle childhood q36 Provide information to communities for health promotion q23c Deliver care to accommodate beliefs about puberty q22a Affiliated with natural networks of support q35e Advocate through social justice community organizations q35h Advocate through faith-based organizations q22g Affiliated with social organizations q35b Advocate through local and state health officials and legislators q33e Offer workshops that help ESL* patients navigate the healthcare system q9f Know general health status of people in the country of origin of refugees q22f Affiliated with cultural advocacy organizations q22e Affiliated with ethnic merchants and businesses q35g Advocate through the media q22c Affiliated with traditional healers Standard Error (SE), 2 In.MSQ -Infit 3 Out.MSQ Outfit Item, 4 PTME Item Total Correlation 35

36 Figure 1: Wright-Item Person Maps for Factors 1, 2, and 3 Factor 1 Factor 2 Factor 3 PERSON - MAP - ITEM <more><rare> 7.# # + # T 4.## +.# ###.# 3.### +.#.#### S.###### 2.## +.############.#####.########## 1.##### +T ITEM36 ITEM6 ITEM7.########### M ITEM26.############ S.##### ITEM12 ITEM14 ITEM23 ITEM32 0.########## +M ITEM10 ITEM13 ITEM25 ITEM4.######## ITEM11 ITEM3 ITEM30 ITEM9.###### S ITEM1 ITEM16 ITEM2 ITEM20 ITEM35 ITEM8.##### -1.##### +T ITEM19.## S.###.##### -2 ## +.#.# -3. T <less><frequ> EACH "#" IS 16. EACH "." IS 1 TO 15 Note. *ESL means English as a second language PERSON - MAP - ITEM <more><rare> 5.########### +.####.# T.### 4. +.#####.###### 3.####### S+.######.###.#.######.########### 2.##### +.######.######.######### M.######.########## ITEM118 1.#### +.############ T.#########.######## ITEM55 ###### S ITEM113 ITEM44 ITEM57 ITEM59.##### ITEM92 ITEM94 0.##### S+M ITEM109 ITEM112 ITEM45 ITEM95 ITEM96.#### ITEM46 ITEM56.### S ITEM110 ITEM111.### ITEM105 ITEM106 ITEM108.## ITEM107.# T -1.## +.#.# T # + <less><frequ> EACH "#" IS 13. EACH "." IS 1 TO 12 PERSON - MAP - ITEM <more><rare> 5.# #. T.# 2. + ###.#.###.# T ITEM71 #### 1.###### S+ ITEM127 ITEM73.###### ITEM74.###### S ITEM42.####### ITEM119.########## ITEM122.######## ITEM125 ITEM128 ITEM75 0.########## +M ITEM69.############ ITEM129 ITEM78 ITEM84.######## M ITEM121 ITEM64 ITEM85.######## ITEM86.############ S ITEM67 ########## ITEM118 ITEM120-1.######### + ITEM117 ITEM68 #######.###### T.###### ####### S.### -2.##### +.##.##.#.#.# -3 T+.#.#.# # + <less><frequ> EACH "#" IS 13. EACH "." IS 1 TO 12 36