SELF-STIGMA IN PEOPLE with mental illness is a

Transcription

1 Psychiatry and Clinical Neurosciences 2017; 71: doi: /pcn Regular Article Development of Japanese version of King s Stigma Scale and its short version: Psychometric properties of a selfstigma measure Masashi Mizuno, PhD, 1 * Sosei Yamaguchi, PhD, 1 Ayano Taneda, PhD, 1 Hiroaki Hori, PhD, MD, 2 Ayako Aikawa, PhD 3 and Chiyo Fujii, PhD, MD 1 Departments of 1 Psychiatric Rehabilitation, 2 Adult Mental Health, National Institute of Mental Health, National Center of Neurology and Psychiatry, Tokyo, 3 Faculty of Human Welfare, Seigakuin University, Saitama, Japan Aim: The study aimed to develop two Japanese versions of King s Stigma Scale, a full version (KSS-J-1) and a short version (KSS-J-2), through psychometric property testing. Methods: The sample included 112 people with mental illness. We tested the constructs of the scales using both confirmatory and exploratory factor analyses. Internal consistency and test retest reliability were tested. We examined convergent validity with self-esteem or perceived stigma, and different group validity, using the Kessler Psychological Distress Scale (K6). Results: Whereas a relatively weak model fit (comparative fit index = 0.66, Tucker Lewis index = 0.63, root mean square error of approximation = 0.097) of KSS-J-1 (full version: 28 items) was found, KSS-J-2 (short version: 17 items), produced by exploratory factor analysis, had a moderate model fit (comparative fit index = 0.90, Tucker Lewis index = 0.89, root mean square error of approximation = 0.063). High internal consistency (KSS-J-1, ω = ; KSS-J-2, ω = ) and moderate test retest reliability (KSS-J-1, interclass correlation = ; KSS-J-2, interclass correlation = ) were reported. Some subscales and the entire scale of KSS-J-1 were significantly correlated with self-esteem and perceived stigma. Conversely, only two subscales in KSS-J-2 were significantly correlated with self-esteem. The scores of each subscale and the entire score for both KSS-J-1 and KSS-J-2 in the high psychological distress group were higher than the low group (KSS-J-1, d = ; KSS-J-2, d = ), except for the Discrimination subscale in KSS-J-2. Conclusion: Both Japanese versions of King s Stigma Scale can be utilized depending on their intended use. Key words: mental illness, perceived stigma, scale development, self-esteem, self-stigma. SELF-STIGMA IN PEOPLE with mental illness is a global concern. Self-stigma occurs when people internalize perceived prejudices and develop negative feelings about themselves, 1 and is a major barrier to personal recovery and social participation. 2 4 *Correspondence: Masashi Mizuno, PhD, Department of Psychiatric Rehabilitation, National Institute of Mental Health, National Center of Neurology and Psychiatry, Ogawahigashimachi, Kodaira, Tokyo , Japan. m-mizuno@ncnp.go.jp Received 8 August 2016; revised 16 September 2016; accepted 18 October Past studies have found several strategies for overcoming mental-health-related stigma in the general public, including social contact with people with mental. 5 However, reducing selfstigma in people with mental illness requires a different approach from that for the general public. 5 In Japan, only a small number of studies have addressed self-stigma, of which several reported that Japanese people are likely to hold more stigmatizing attitudes towards people with mental compared to other countries. 6 This lack of research in self-stigma may be due to a lack of 189

2 190 M. Mizuno et al. Psychiatry and Clinical Neurosciences 2017; 71: reliable measures for self-stigma that can be used in a Japanese setting. 6 In short, there is a need for the development of a comprehensive self-stigma scale in Japan. Previous studies have developed several selfstigma scales and self-stigma-related scales. Link s Stigma Scale was employed as a measure of selfstigma in a number of studies. 7 However, this scale focuses on perceived stigma, rather than self-stigma. Another example is the Internalized Stigma of Mental Illness scale, which can measure internalized stigma, mainly focusing on users beliefs and attitudes towards mental illness. 8 In addition, the Discrimination and Stigma Scale (DISC) mainly measures users experiences of discrimination. 9 To the best of our knowledge, King s Stigma Scale (KSS), originally developed in the UK, is the only scale that can assess both internalized stigma related to a belief of mental illness and experiences of discrimination among people with mental illness. 10 In short, KSS appears to be a more comprehensive scale compared to other scales. Thus, this study aimed to develop a Japanese version of KSS and to test the psychometric properties, including model fit of the scale, internal consistency, test retest reliability, convergent validity, and different group validity. In the developing process, we developed two versions of KSS. One included the same subscales and items as the original scale, whereas the other included only the items with high factor loadings from an exploratory factor analysis to fit in the Japanese context. from schizophrenia (59.8%) and a quarter of them suffered from mood disorders (25.0%). About 80% of the participants had experience with psychiatric admission (Table 1). In addition, the 15 service users in the psychiatric day-care, who were randomly selected, answered the questionnaire twice for test retest reliability. Participants were informed about the study. They were told that their anonymity would be preserved, except for the fact that the research assistant would identify the names of participants to conduct the second survey for the test retest reliability. Participants provided informed consent representing their agreement to participate in the study. This study was approved by the Research Ethics Committee at the National Center of Neurology and Psychiatry (approval number A ). Measurement KSS KSS included 28 items and was composed of the following three subscales: Discrimination, Disclosure, and Positive Aspects. Each item was rated on a 5-point Likert scale. This scale has been confirmed to have: high internal consistency reliability for Discrimination (α = 0.85) and Disclosure (α = 0.87), but not Positive Aspects (α = 0.64); and constructive validity using correlation analysis between each subscale and a self-esteem scale (r = 0.36 to 0.64). METHODS Participants and procedure We conducted a questionnaire survey in a psychiatric day-care, a community support center, and a workshop for peer supporters who have experienced mental illness and work in mental health service agencies as staff members. This workshop was aimed to teach the philosophy, mission, and skills to work as a peer supporter and to exchange information among peer supporters. A total of 144 eligible service users and peer supporters were asked to participate in this study. Finally, 87 service users and 25 peer supporters voluntarily participated in this survey (response rate: 77.8%). A total of 73 participants were male, with a mean age of (SD = 11.09). More than half of the participants suffered Table 1. Characteristics of participants n = 112 n % Sex Male Female Diagnosis Schizophrenia Mood disorders Other axis-i disorders Personality disorders Developmental disorders Others Unknown Experiences of psychiatric admission Yes No Unknown 1 0.9

3 Psychiatry and Clinical Neurosciences 2017; 71: Development of stigma scale 191 The Japanese version of the scale was developed through the following steps. First, our team translated the scale into Japanese and made the first draft. Then, an American medical student and a Japanese clinical psychologist who had lived in the USA performed back-translation of the first draft into Japanese. The author of the original scale checked the back-translation and provided some suggestions. Finally, we slightly modified the wording of some items based on his suggestions. Other scales To examine the convergent and different group validity of KSS, we used three additional scales to assess self-esteem, perceived stigma, and psychological distress. First, we used the Japanese version of the Rosenberg Self-Esteem Scale, which is used internationally. 11,12 This scale is composed of 10 items. The Japanese version of this scale has been confirmed to have high internal reliability and constructive validity. 12 Second, we used the Japanese version of Link s Stigma Scale. 7,13 This scale comprises 12 items and measures the perceived community residents attitudes towards people with mental illness. The Japanese version of this scale has high internal reliability and constructive validity. 13 Third, the Kessler Psychological Distress Scale (K-6) was employed. 14,15 This scale is composed of six items and measures psychological distress. The cut-off point is set at 12/13 points of the total score. 14 Statistical analysis A confirmatory factor analysis was conducted to test the model fit of the Japanese version of KSS (KSS-J- 1: full version) and to examine the constructs of the scale in a Japanese setting. We also conducted an exploratory factor analysis with promax rotation methods to clarify factor structure of KSS fitted in a Japanese context (KSS-J-2: short version), and tested its model fit through confirmatory factor analysis. To test the reliability, we calculated the ω coefficient as internal consistency indicator and the interclass correlation coefficient as test retest reliability indicator. Then, we tested the convergent validity and the different group validity. In terms of convergent validity, Pearson s correlation analyses were conducted on the basis of the hypothesis that KSS had a negative correlation with the Self-Esteem Scale and a positive correlation with Link s Stigma Scale. With regard to the different group validity, an unpaired two-tailed t-test was conducted. The participants were divided into either group by the K-6 cut-off point and into the schizophrenia group or the other diagnosis group. RESULTS Constructs of the scale Confirmatory factor analysis was conducted to evaluate the model fit of KSS (KSS-J-1: full version) in a Japanese setting. The results showed model fit indicators were: comparative fit index (CFI) = 0.66, Tucker Lewis index (TLI) = 0.63, and root mean square error of approximation (RMSEA) = (Table 2). Appendix S1 presents all the items for this scale and shows descriptive statistics for the confirmatory factor analysis. We identified three factors of KSS-J-2 (short version) through using exploratory factor analyses. After the items that showed <0.40 in the one factor and 0.40 in the plural factors were removed, we reconducted the exploratory factor analysis and extracted three factors and 17 items (Table 3, Appendix S2). The first factor in KSS-J-2 consisted of eight items regarding the fear or the worry about disclosure of one s own mental. Therefore, we named the first factor Disclosure. The second factor included six items, which mainly focused on the actual experiences of discrimination. Therefore, the second factor was named Discrimination. Thethird factor consisted of three items, which asked positive views on and experiences of one s own mental illness. We labeled the third factor as Positive Aspects. The results of confirmatory factor analysis for KSS-J-2 demonstrated the following model fit indicators: CFI = 0.90, TLI = 0.89, and RMSEA = Reliability of the scale The ω coefficients of KSS-J-1 were relatively high (ω = ). Similarly, high ω coefficients of KSS- J-2 were obtained in each factor and for the entire scale (ω = ). The test retest reliability ranged from 0.56 to 0.88 for each subscale and the whole scale in KSS-J-1 and ranged from 0.45 to 0.85 for each subscale and the entire scale in KSS-J-2 (Table 4). Validity of the scale In terms of the convergent validity of KSS-J-1, the correlation analyses showed KSS and all three

4 192 M. Mizuno et al. Psychiatry and Clinical Neurosciences 2017; 71: Table 2. Results of confirmatory factor analysis: 28-item full version (KSS-J-1) Items (n = 112) Standardized estimates Mean SD Factor (subscale) 1: Discrimination, 13 items 1 I have been discriminated against in education because of my mental Sometimes I feel that I am being talked down to because of my mental I have been discriminated against by the police because of my mental I have been discriminated against by employers because of my mental Very often I feel alone because of my mental I would have had better choices in life if I had not had a mental illness People s reactions to my mental make me keep to myself I am angry with the way people have reacted to my mental I have not had any trouble from people because of my mental I have been discriminated against by health professionals because of my mental 21 People have avoided me because of my mental People have insulted me because of my mental Having had mental makes me feel life is unfair Factor (subscale) 2: Disclosure, 10 items 4 I do not feel bad about having had mental I worry about telling people I receive psychological treatment I am scared of how other people will react if they find out about my mental 14 I do not mind people in my neighborhood knowing I have had mental I would say I have had mental health problem if I was applying for a job I worry about telling people that I take medicines/tablets for mental health problems 24 I do not feel embarrassed because of my mental I avoid telling people about my mental I feel the need to hide my mental from my friends I find it hard telling people I have mental Factor (subscale) 3: Positive aspects, 5 items 3 Having had mental has made me a more understanding person Some people with mental are dangerous People have been understanding of my mental My mental have made me more accepting of other people Having had mental has made me a stronger person subscales had a significant correlation with selfesteem (Discrimination, r = 0.21, P < 0.05; Disclosure, r = 0.33, P < 0.01; Positive Aspects, r = 0.41, P < 0.01; entire scale, r = 0.39, P < 0.01). We found weak but significant correlations between the entire scale and perceived stigma (r = 0.22, P < 0.05) and between the Positive Aspect subscale and perceived stigma (r = 0.26, P < 0.01) (Table 4). With regard to KSS-J-2, Disclosure and Positive Aspect had significant correlations with selfesteem (r = 0.34, P < 0.01; r = 0.41, P < 0.01). There were no significant correlations between KSS- J-2 and perceived stigma. Table 5 shows the results of the different group validity. The high psychological distress group showed significantly higher scores for the entire

5 Psychiatry and Clinical Neurosciences 2017; 71: Development of stigma scale 193 Table 3. Results of exploratory factor analysis: 17-item short version (KSS-J-2) Items (n = 112) Factor.1 Factor.2 Factor.3 Factor 1: Disclosure, eight items 5 I worry about telling people I receive psychological treatment Very often I feel alone because of my mental I do not mind people in my neighborhood knowing I have had mental I worry about telling people that I take medicines/tablets for mental health problems 17 People s reactions to my mental make me keep to myself I avoid telling people about my mental I feel the need to hide my mental from my friends I find it hard telling people I have mental Factor 2: Discrimination, six items 1 I have been discriminated against in education because of my mental Sometimes I feel that I am being talked down to because of my mental I have been discriminated against by the police because of my mental I have been discriminated against by employers because of my mental I have been discriminated against by health professionals because of my mental 22 People have insulted me because of my mental Factor 3: Positive aspects, three items 3 Having had mental has made me a more understanding person My mental have made me more accepting of other people Having had mental has made me a stronger person scale and each subscale of KSS-J-1 compared to the low group (Discrimination, MD = 5.47, P < 0.01; Disclosure, MD = 5.09, P < 0.01; Positive Aspects, MD = 2.19, P < 0.01; entire scale, MD = 12.71, P <0.01). In addition, the scores of the entire scale and two subscales (Disclosure and Positive Aspects) of KSS-J-2 in the high psychological distress group were significantly higher than those in the low group (Disclosure, MD = 6.14, P < 0.01; Positive Aspects, MD = 1.38, P < 0.01; entire scale, MD = 6.20, P <0.05). Then we compared the KSS scores between people with schizophrenia and people with other mental disorders using the t-test. The results showed there was no significant difference in KSS scores between the groups. DISCUSSION We developed the KSS in a Japanese context. KSS can measure both users attitudes towards their own mental illness together with users experience of discrimination, whereas other measures often focus only on either users attitudes or experience of discrimination. Through the development process, we found moderate model fit values for the scale (KSS- J-1: full version), despite it having the same factors and items as the original scale. Therefore, we examined and developed another version of KSS (KSS-J-2: short version) in a Japanese context. KSS-J-1 (full version) The results of a confirmatory factor analysis for KSS- J-1 showed that the model fit indicators were weak, except for RMSEA (CFI = 0.66, TLI = 0.63, and RMSEA = 0.097). It is noted that RMSEA is superior to other model fit indicators when the confirmatory factor analysis includes many observed variables; a criterion of an acceptable RMSEA value was < ,17 Therefore, the model fit of the Japanese version of KSS (KSS-J-1) may not be extremely poor

6 194 M. Mizuno et al. Psychiatry and Clinical Neurosciences 2017; 71: Table 4. Results of analysis for internal consistency, test retest reliability, and convergent validity n = 112 ω Inter-class correlation 95%CI Self-esteem Perceived stigma KSS-J-1 (full version) All items ** ( ) 0.39** 0.22* Discrimination ** ( ) 0.21* 0.13 Disclosure ** ( ) 0.33** 0.17 Positive Aspects ** ( ) 0.41** 0.26** KSS-J-2 (short version) All items ** ( ) Discrimination ** ( ) Disclosure ** ( ) 0.34** 0.18 Positive Aspects * ( ) 0.41** 0.15 *P < 0.05; **P < when taking into account the number of items in this scale (28 items). In terms of the reliability of KSS-J-1, we confirmed high consistency values (ω = ). The ω values of the scale were high on the basis of the criterion for the alpha value (high > 0.80 moderate = , low < 0.70). 18 In addition, the values of test retest reliability for the entire scale and the Discrimination subscale were moderate to high (ICC = 0.78 and 0.88), but low for the other two subscales according to the criterion for reliability values (high > 0.90 moderate = , low < 0.75). 18 These indicated that KSS-J-1 had sufficient internal reliability but slightly inadequate test retest reliability. We evaluated the validity of the Japanese version of KSS through tests of convergent and different group validity. We found a significant correlation between self-stigma and self-esteem. However, we did not find a significant association between perceived stigma and the Discrimination and Disclosure subscales. A past study reported that people with mental often anticipated a stigmatizing reaction from others even if they had never perceived discriminatory behaviors. 19 In addition, some individuals, particularly those who work, may hesitate to disclose their own mental health problems, irrespective of their personal feelings of others having stigmatizing attitudes towards mental. 20 In short, experience of discrimination and disclosure may not be associated with perceived stigma. Therefore, non-significant correlations between perceived stigma and the Discrimination and Disclosure subscales may not deny the Table 5. Results of different group validity (K6 score) Higher group (n = 26) Lower group (n = 85) n = 112 Mean SD Mean SD P Cohen s d 95%CI KSS-J-1 (full version) All the items P < ( ) Discrimination P < ( ) Disclosure P < ( ) Positive Aspects P < ( ) KSS-J-2 (short version) All the items P < ( ) Discrimination NS 0.28 ( ) Disclosure P < ( ) Positive Aspects P < ( ) Cut-off point of K6 is 12/13. K6, Kessler Psychological Distress Scale; NS, not significant.

7 Psychiatry and Clinical Neurosciences 2017; 71: Development of stigma scale 195 validity of the scale. With regard to the different group validity, the results showed that the scores for each subscale and the total scores in the high psychological distress group were higher than those of the low groups. People with strong depressive symptoms are likely to be disinterested in others and daily activities. The significantly lower scores of the low psychological distress group contributed to further validation of the scale (KSS-J-1). On the other hand, the diagnoses were not associated with the degree of self-stigma. KSS-J-2 (short version) Due to the low or moderate model fit values for KSS-J-1, we examined KSS-J-2. The exploratory factor analysis for KSS-J-2 removed 11 items and identified the following three factors: Disclosure, Discrimination, and Positive Aspects. There was a difference in items included in each factor between the KSS-J-1 and KSS-J-2. Specifically, the items regarding fears and the worry of disclosure were included in the Discrimination subscale of the original version. However, such items were classified into another factor group, namely Disclosure, in KSS-J-2. As fear and worry of disclosure seem to be conceptually different from discrimination, including these items in the Disclosure subscale in KSS-J-2 appeared to better reflect the constructs of the subscale. A recent study that developed a Japanese version of another selfstigma scale from the USA, namely the Internalized Stigma of Mental Illness scale, also removed some items and changed the factors of the scale to fit a Japanese context. 21 Given the cultural influence on self-stigma, the changes in the items and the factors of the self-stigma scale are not inappropriate means in its development process. 22 Indeed, the model fit values of KSS-J-2 produced by the confirmatory factor analysis were superior to those in KSS-J-1 and appear to be acceptable (CFI = 0.90, TLI = 0.89, and RMSEA = 0.063). These values suggest that KSS-J-2 is a more appropriate scale in a Japanese setting in terms of construct validity. However, the results of the convergent validity and different group validity of KSS-J-2 were inconsistent with the results of KSS-J-1. These results may be attributed to the differences of scale constructs between Stigma Scale KSS-J-1 and KSS-J-2. The subscales in Stigma Scale KSS-J-2 appear to reflect the constructive concepts more appropriately. The discrimination factor in Stigma Scale KSS-J-2 consisted of only items concerning the experiences of discrimination, although the Discrimination subscale in KSS-J-1 included not only items regarding past experience of discrimination but also items concerning fear and worry about disclosure of one s own mental illness. In this study, the discrimination factor in KSS-J-2 was not associated with the level of selfesteem and psychological distress. It is considered that experience of discrimination itself is a different concept from self-stigma. 23 In the cases where individuals internalize shame and fear from others repetitive discrimination and rejection, such internalized feelings are likely to cause low self-esteem, which may lead to individuals low psychological state. 24 Indeed, other factors in KSS-J-2 had the item asking whether fear was correlated with the level of self-esteem. In addition, as stated above, the absence of an association between experience of discrimination and perceived stigma does not appear to nullify the validity of the scale, as people with mental illness often hold perceived stigma, irrespective of whether they experienced actual discrimination or not. 19 The diagnosis was not related to self-stigma as in KSS-J-1. In terms of reliability, high consistency values of KSS-J-2 were obtained (ω = ). We also found moderate and high test retest values of the entire scale and the two subscales of KSS-J-2 (ICC = ), except for the Positive Aspects subscale (ICC = 0.45), for which the ICC was also low in KSS-J-1. These may indicate that KSS-J-2 had overall sufficient internal reliability and moderate test retest reliability. Limitations This study has some limitations. First, the participants in this study included the service users in a psychiatric day-care or a community support center. However, users of other types of services may have different levels of stigma (e.g. users of employment service agencies and inpatients). In addition, we recruited service-users and peer supporters at a workshop. Particularly peer-supporters may not be representative of other peer-supporters across Japan. Second, around half of the participants in this study suffered from schizophrenia, whereas in the development process of the original version of KSS, the participants had multiple diagnoses. In short, the generalizability of this study is limited, and this poses a challenge for the future. Third, we did not

8 196 M. Mizuno et al. Psychiatry and Clinical Neurosciences 2017; 71: assess symptom severity for participants, which may be associated with level of self-stigma. Implication for future research and practices We developed the Japanese versions of KSS, including the full version, which has the same items and the factors with the original scale, and a short version. A major strength of these comprehensive scales is that the scales can assess not only internalized feelings and beliefs of own illness but also experience of discrimination. Although the short version seems to be more fitted in a Japanese context, we may be able to accordingly use both scales for an international comparison and evaluation of selfstigma in a clinical setting. ACKNOWLEDGMENTS We thank all the participants and staff who collaborated in the study. We are profoundly grateful to Professor Michael King, author of the original Stigma Scale, for permitting us to develop a Japanese version of the scale. This study was supported by grants from the JSPS/MEXT (KAKENHI no. 15H03432). DISCLOSURE STATEMENT All authors declare no conflicts of interest. AUTHOR CONTRIBUTIONS M.M. and S.Y. contributed to writing the draft of the manuscript and the statistical analysis. M.M., S.Y., T.A, A.A., and C.F. contributed to the design and management of the study. H.H. contributed to the translation process for the scale development. All authors have approved the final version of the manuscript. REFERENCES 1. Latalova K, Kamaradova D, Prasko J. Perspectives on perceived stigma and self-stigma in adult male patients with depression. Neuropsychiatr. Dis. Treat. 2014; 10: Leamy M, Bird V, Boutillier CL, Williams J, Slade M. Conceptual framework for personal recovery in mental health: Systematic review and narrative synthesis. Br. J. Psychiatry 2011; 199: Perlick DA, Rosenheck RA, Clarkin JF et al. Stigma as a barrier to recovery: Adverse effects of perceived stigma on social adaptation of persons diagnosed with bipolar affective disorder. Psychiatr. Serv. 2001; 52: Sirey JA, Bruce ML, Alexopoulos GS, Perlick DA, Friedman SJ, Meyers BS. Stigma as a barrier to recovery: Perceived stigma and patient-related severity of illness as predictors of antidepressant drug adherence. Psychiatr. Serv. 2001; 52: Griffiths KM, Carron-Arthur B, Parsons A, Reid R. Effectiveness of programs for reducing the stigma associated with mental disorders. A meta-analysis of randomized controlled trials. World Psychiatry 2014; 13: Ando S, Yamaguchi S, Aoki Y, Thornicroft G. Review of mental health-related stigma in Japan. Psychiatry Clin. Neurosci. 2013; 67: Link BG. Understanding labeling effects in the area of mental disorders: An assessment of the effects of expectations of rejection. Am. Sociol. Rev. 1987; 52: Ritsher JB, Otilingam PG, Grajales M. Internalized stigma of mental illness: Psychometric properties of a new measure. Psychiatry Res. 2003; 121: Brohan E, Clement S, Rose D, Sartorius N, Slade M, Thornicroft G. Development and psychometric evaluation of the discrimination and stigma scale. Psychiatry Res. 2013; 208: King M, Dions S, Shaw J et al. The stigma scale: Development of a standardized measure of the stigma of mental health. Br. J. Psychiatry 2007; 190: Rosenberg M. Society and the Adolescent Self-Image. Princeton University Press, Princeton, Yamamoto M, Matsui Y, Yamanari Y. The structure of perceived aspects of self. Jpn. J. Educ. Psychol. 1982; 30: (in Japanese). 13. Shimotsu S, Sakamoto S, Horikawa N, Sakano Y. Reliability and validity of the Link s devaluation-discrimination scale. Jpn. J. Psychiatr. Treat. 2006; 21: (in Japanese). 14. Furukawa TA, Kawakami N, Saitoh M et al. The performance of the Japanese version of the K6 and K10 in the World Mental Health Survey Japan. Int. J. Methods Psychiatr. Res. 2008; 17: Kessler RC, Andrews G, Colpe LJ et al. Short screening scale to monitor population prevalences and trends in non-specific psychological distress. Psychol. Med. 2002; 32: Hair J, Black B, Babin B, Anderson R, Tatham R. Multivariate Data Analysis, 6th edn. Pearson Prentice Hall, Upper Saddle River, NJ, Toyoda H. Notes for discussion of structural equation modeling. Jpn. J. Behaviormetrics 2002; 29: (in Japanese). 18. Lundberg M, Grimby-Ekman A, Verbut J, Simmonds M. Pain-related fear: A critical review of the related measures. Pain Res. Treat doi: /2011/ Üçok A, Brohan E, Rose D et al. Anticipated discrimination among people with schizophrenia. Acta Psychiatr. Scand. 2012; 125:

9 Psychiatry and Clinical Neurosciences 2017; 71: Development of stigma scale Brohan E, Henderson C, Wheat K et al. Systematic review of beliefs, behaviours and influencing factors associated with disclosure of a mental health problem in the workplace. BMC Psychiatry 2012; 12: Tanabe Y, Hayashi K, Ideno Y. The Internalized Stigma of Mental Illness (ISMI) scale: Validation of the Japanese version. BMC Psychiatry 2016; 16: Boyd JE, Adler EP, Otilingam PG, Peters T. Internalized Stigma of Mental Illness (ISMI) scale: A multinational review. Compr. Psychiatry 2014; 55: Brohan E, Slade M, Clement S, Thornicroft G. Experiences of mental illness stigma, prejudice and discrimination: A review of measures. BMC Health Serv. Res. 2010; 10: Corrigan PW. The impact of stigma on severe mental illness. Cogn. Behav. Pract. 1998; 5: SUPPORTING INFORMATION Additional Supporting Information may be found in the online version of this article at the publisher s web-site: Appendix S1. Subscales and items of KSS-J-1 in Japanese. Appendix S2. Subscales and items of KSS-J-2 in Japanese.