Robert Karasek, BongKyoo Choi, Per-Olof Ostergren, Marco Ferrario, and Patrick De Smet

Transcription

1 International Journal of Behavioral Medicine Copyright C 2007 by 2007, Vol. 14, No. 4, Lawrence Erlbaum Associates, Inc. Testing Two Methods to Create Comparable Scale Scores between the Job Content Questionnaire () and -Like Questionnaires in the European JACE Study Robert Karasek, BongKyoo Choi, Per-Olof Ostergren, Marco Ferrario, and Patrick De Smet Background: Scale comparative properties of -like questionnaires with respect to the have been little known. Purpose: Assessing validity and reliability of two methods for generating comparable scale scores between the Job Content Questionnaire () and -like questionnaires in sub-populations of the large Job Stress, Absenteeism and Coronary Heart Disease European Cooperative (JACE) study: the Swedish version of Demand-Control Questionnaire (DCQ) and a transformed Multinational Monitoring of Trends and Determinants in Cardiovascular Disease Project (MONICA) questionnaire. Method: A random population sample of all Malmo males and females aged (n = 682) years was given a new test questionnaire with both instruments (the and the DCQ). Comparability-facilitating algorithms were created (Method I). For the transformed Milan MONICA questionnaire, a simple weighting system was used (Method II). Results: The converted scale scores from the -like questionnaires were found to be reliable and highly correlated to those of the original. However, agreements for the high job strain group between the and the DCQ, and between the and the DCQ (Method I applied) were only moderate (Kappa). Use of a multiple level job strain scale generated higher levels of job strain agreement, as did a new job strain definition that excludes the intermediate levels of the job strain distribution. Conclusion: The two methods were valid and generally reliable. Key words: instruments, scale comparability, job strain, validity, reliability, kappa Introduction The Job Content Questionnaire (, Karasek, 1985) has very possibly become the most commonly Robert Karasek, Department of Work Environment, University of Massachusetts Lowell, U.S.A. and Institute for Psychology, Copenhagen University, Denmark; BongKyoo Choi, Department of Work Environment, University of Massachusetts Lowell, U.S.A.; Per-Olof Ostergren, Department of Health Sciences, Lund University, Malmo University Hospital, Sweden; Marco Ferrario, Medicina del Lavoro e Preventiva, Dipartimento di Scienze Cliniche e Biologiche, Università degli studi dell Insubria, Varese, Italy; Patrick De Smet, Laboratoire d Epidemiolgie et de Medecine Sociale, Universite Libre de Bruxelles, Belgique. The authors thanks the JACE study group (M. Kornitzer, Chairman [Brussels]; G. de Backer [Ghent]; M. Romon-Rousseau/C. Boulenguez [Lille]; I. Houtman [Leiden]; L. Wilhelmsen [Gothenburg]; P.-O. Ostergren [Malmo]; M. Ferrario [Milan]; and S. Sans [Barcelona]) for support on this project. We also appreciate Tania Araújo for her review and valuable comments on the manuscript. Correspondence concerning this article should be addressed to Robert Karasek, Department of Work Environment, University of Massachusetts Lowell, Lowell, MA 01854, USA. Robert Karasek@uml.edu used psychosocial job hazard assessment instrument (Hurrell, Nelson, & Simmons, 1998; Landsbergis & Theorell, 2000) to test the Demand-Control model (Karasek, 1979; Karasek & Theorell, 1990). However, other questionnaires exist with theoretically consistent scales, even if question wordings and response formats differ, for example, the Theorell Questionnaire used in Sweden (DCQ: Theorell et al., 1988), sections of the Whitehall Questionnaire (Bosma, Marmot, Hemingway, Nicholson, Brunner, & Stansfeld, 1997; Marmot, Smith, & Stansfeld, 1991), the Danish MON- ICA (WHO, Multinational Monitoring of Trends and Determinants in Cardiovascular Disease Project) Questionnaire (Netterstrøm, Kristensen, Damsgaard, Olsen, & Sjøl, 1991), and the Copenhagen Psychosocial Questionnaire (Kristensen, Borg, & Hannerz, 2002) hereafter referred to as -like questionnaires. In many cases, these alternative instruments arose early in the history of testing the demand-control model and evolved into separate significant national research traditions. In this paper, we address the compatibility challenges posed by two of these: the DCQ and the MONICA questionnaire. 189

2 KARASEK ET AL. Internationally, progress has been made in several areas of scale validation and comparative scale validation. Studies have tested the reliability and validity of the scales (mainly, skill discretion, decision authority, psychological demands, supervisor support, and coworker support scales) in both mature market economies and newly industrialized countries (Araújo & Karasek, 2005; Brisson, Blanchette, Guimont, Dion, Moisan, & Vezina, 1998; Cedillo-Becerril, 1999; Cheng, Luh, & Guo, 2003; Kawakami & Fujigaki, 1996; Kawakami, Kobayashi, Araki, Haratani, & Furui, 1995; Pelfrene, Vlerick, Mak, De Smet, Kornitzer, & De Barker, 2001). Cross-national validity and reliability of the scales have been examined in broadly representative industrial society populations, including eight populations in the European Union (Karasek, Brisson, Kawakami, Houtman, Bongers, & Amick, 1998; Karasek, Choi, Ostergren, Ferrario, & De Smet, 2003). All of the assessments have generally found acceptable validity and substantial comparability of the scales across populations, albeit with some significant exceptions, as in the case of the psychological demand scale. However, little is known of the comparative properties of the -like questionnaires with respect to the. There is little analytical information about the differential, statistical properties across those alternative instruments. Direct numerical comparisons of scale scores from those instruments with the are, of course, not possible. Also, there is little information about the characteristics of the job strain groups defined by these instruments tested, for example, on the same population. The definition of high strain group is usually formulated in terms of cut-points on two scales (of decision latitude and psychological demands in case of the ), and thus comparable exposure groups cannot necessarily be defined with the scale scores from different instruments compromising meta-analytic possibilities. This information becomes increasingly important as public health discussions continue to search for analytical summaries of epidemiological information (De Lange, Taris, Kompier, Houtman, & Bongers, 2003; Spector & Jex, 1998) for assessment of the psychosocial job hazards in a global economy. Direct numerical comparisons rely on the internal property of the scales (an assumption routinely invoked in regression analysis in much research in the field, if rarely rigorously tested), and then on the equivalence of scale statistics, such as mean and standard deviation. This requires both testing and correction routines to transform scales of one instrument to be equivalent to those of another the standard scale. A natural opportunity arose in the recent examination of the cross-national validity and reliability of the JACE- database from five European countries (Belgium, France, Italy, The Netherlands, and Sweden) to test the validity for the -like questionnaires and 190 to develop correction routines to allow joint analyses of these data under the auspices of the JACE scientific committee (Karasek, Choi, Tak, Cifuentes, Hemmingsson, & Villadsen, 2002; Karasek et al., 2003). The JACE prospective epidemiology study to compare the distribution of the Karasek job stress scales for the same categories of occupations in different European countries (Houtman et al., 1999, p. 54) primarily used the to measure the perceived job stressors of workers in five European countries but two -like instruments were also used in the two centers (Swedish Malmo sample and Milan sample of two sub-sites) of the JACE study. While both questionnaires are conceptually very similar to the, they are different from the in terms of the numbers of items, detailed item wording or response categories (Table 1). The goal of the present study was to test the relevance of two methods (Method I and Method II, see Method section below) for generating the comparable scale scores of decision latitude and psychological demands between the and the two -like questionnaires. For Method I, a separate test study was undertaken at the Malmo JACE site. Additionally, new formulations of the job strain term were formulated to facilitate compatibility across instruments. To our knowledge, no other published studies have compared the scale scores properties and the agreement for the high strain group between the and the DCQ within the same population. Materials and Methods The JACE- Database The eight populations studied came from five European countries in eight research centers from 1991 to 1998: Belgium (Ghent and Brussels), France (Lille), Italy (Milan), The Netherlands (Leiden), and Sweden (Malmo, Gothenburg 1993, and Gothenburg 1994). The Spanish Barcelona JACE sample was not available for this study. Apart from three Swedish centers which used random population samples, all centers recruited a more or less diverse private and public employee population from a broad range of organizations that agreed to participate (also, the Milan sample includes only public employees from six occupationally diverse departments of the city administration). The JACE prospective epidemiology study primarily used the Karasek s to measure the perceived job stressors of workers in five European countries but two -like questionnaires were also included: for the Swedish Malmo sample and the Milan sample. The number of participants in each study ranged from 798 to 11,405 for a total of 37,161 participants (35% women and 65% men), yielding the largest complete data across multiple occupations to date. The sample sizes of Milan and Malmo were 4,850 and

3 TESTING TWO METHODS TO CREATE SCALE SCORES 6,072, respectively. They consisted of 29.4% of the total JACE- population. The age boundaries span mainly from 35 to 59, as is typical for the primary study focus on cardiovascular epidemiology. The Malmo sample s age ranged from 46 to 59, and its age means were the highest in both women and men. Generally, eight samples included all full major occupation groups (Group 1 to Group 9, except Group 6: skilled agricultural and fishery workers) identified according to the International Standard Classification of Occupations (ISCO) major group (one-digit) codes of International Labor Office (ILO, 1990; see Appendix I). But the occupation compositions of each sample were different by sample and gender so that a weighting method was developed based on the frequency of the ISCO major groups to generate standardized scale scores, comparable across the JACE- samples (Choi, Karasek, Ostergren, Ferrario, & De Smet, 2004; Karasek et al., 2003). For most of the samples, broad distributions of detailed occupations (according to four-digit ISCO codes) were found within each of ISCO major groups of occupations. However, in the Milan men and women samples, one or two occupations represented the one-digit ISCO major groups (e.g., police officers, 93.6% of Group 5 in Milan men). Means of years of education of eight samples varied from 10 years to 14 years by sample site with some of these relatively, occupationally, homogeneous subsamples having the lowest years of education. The education variable, years of education, had in several JACE sites, missing data that was replaced by an estimation of years based on educational level attained (Milan, Leiden, Malmo, and Gothenburg 1993 samples). Instruments and the -Like Questionnaires For most sites in the JACE study, the 36 questions of the recommended 49 questions of the were used (the most commonly used full format): six skill discretion items, three decision authority items, five psychological demand items (four additional psychological demand items added for making a revised psychological demand scale were excluded both because of non-availability across all sites and serious predicting validity difficulties), five physical demand items, four job insecurity items, four supervisor support items, four coworker support items, and one skill utilization item. The two heterodox -like questionnaires were used in the older studies initiated before the JACE protocol was approved in mid-1993: the Malmo sample and the Milan sample for two of six sub-sites. For the Malmo sample (n = 6,072), eleven questions from the DCQ were used instead of the fourteen questions. This questionnaire had only six items assessing decision latitude (two on decision authority and four on intellectual discretion) in contrast to the s nine items of decision latitude (three on decision authority and six on skill discretion). Also, it employed a different response format based on frequency, i.e., often, sometimes, seldom, and never, instead of Table 1. The Comparison of Items and Response Categories between the, the Swedish version of the DCQ, and a Transformed MONICA Questionnaire Number/Wording of Items DCQ a MONICA Questionnaire) b Scales Skill discretion 6 learn new things 4 learn new things? 4 learn new things repetitive work same things over and over again? repetitive work require creative require creativity? require creative or high skill level high skill level? to use my imagination variety high skill level develop own abilities Decision authority 3 allow own decisions 2 decide on how you do your work? 2 decide how I have to do my work little decision freedom decide on what you do in your work? decide how much work I have (decide how I do my work) to do -X lot of say Psychological demand 5 work very fast 5 work very fast? 5 fast-paced job work very hard work very hard? work very hard no excessive work too much demand? not asked to work too much enough time enough time? enough time conflicting demands conflicting demands? conflicting orders Response category Strongly agree, Agree, Disagree, Often, Sometimes, Seldom, Strongly agree, Agree, Disagree, and Strongly disagree and Never and Completely disagree a Used in the Malmo sample. b Used in the two sub-sites of Milan sample. X: Question dropped at the analysis step. 191

4 KARASEK ET AL. agreement/disagreement response categories, i.e., strongly agree, agree, disagree, and strongly disagree (see Table 1). The psychological demand items were very similar to those of the. For two questions of decision authority of the DCQ, their meanings were similar to those of the but their wordings differed. Second, two sub-sites of the Milan sample (n = 970, roughly 20% of the Milan sample) used a questionnaire similar to the MONICA questionnaire (Netterstrøm et al., 1991) before the JACE protocol was approved in mid The questionnaire was much closer to the than the DCQ. Its skill discretion scale had four of six questions very close to the wording (closer than the DCQ used in Malmo). The decision authority scale had two questions similar to the questions, but their wordings differed. Later, only one question ( decide how to I have to work ) was selected for the decision authority scale score. Its response format was the same as that of the, except that it used completely agree instead of strongly agree. The psychological demand items were very similar to those of the. Method I in Malmo Sample Converting the scale scores. Center s experience with international scale comparisons has shown the importance of identifying the specific source of scale score discrepancies that arise in the context of use of -like instruments (including translation imperfections). Even if there is theoretical congruence between the scales of the and a -like questionnaire, the -like scales can still yield different scores because: (a) remaining wording differences affect response level, (b) response sets differ, (c) different numbers of questions are used in the scales, and (d) diverse other causes all of which must still be adjusted for. We introduce below a multi-step-method that relies first on a separate survey that includes both the and the DCQ in the same random population sample. Analytically, the method involves use of multiple simple adjustment steps that allow the scale components of the and the DCQ to be made more congruent in a stepwise manner. This allows tracing the location of the likely larger differences in easily explainable terms, such as frequency distribution differences and different numbers of questions. The final step of the method resolves all remaining mathematical discrepancies adjustment in a final z-scoring step. An additional advantage of the multi-step method is that each of the separate questions is brought into closer alignment that facilitates use and interpretation of separate question contributions in further analysis. An alternative method is also presented, which uses a simple regression equation to develop correction score coefficients. However, this essentially mixes all 192 sources of discrepancy together and does not allow identification of any intermediate sources. Step I: Administration of the and the DCQ in a Random Population Survey. In order to create the comparable scale scores of decision latitude and psychological demand between the and the DCQ, a separate study was undertaken by the Malmo JACE center. A random population sample of all Malmo males and females aged (n = 682) years was given a new test questionnaire, which included both the DCQ questions and the questions (response rate = 85%) in Step II: Frequency Distribution Adjustment by Question. The preliminary analytic task was to match the response distributions of the eleven Malmo DCQ questions with the equivalent questions. A scale (level) correction (+0.5, 0, or 0.5) was made for each of the five questions (jobrepet: repetitive work; jobhiski: high skill level; jobfreed: little decision freedom; jobinflu: lot of say; jobextm: enough time) of the DCQ whose response distributions were quite different from those of the (see the example of jobinflu question, Figure 1). Increments no finer than one-half step on the four level scales were made in order to maintain verbal interpretability of the adjusted responses. Step III: Correction for Number of Questions in Scale. As the second analytic step, the scales scores with the scale-corrected versions of the Malmo DCQ questions were created using a weighting system to achieve the same weight as the scales by compensating for missing questions. For example, the decision authority scale was multiplied by a factor of 1.5 to compensate for a missing question. Step IV: Final Z-score Mathematic Adjustment. Finally, z-scoring transformation equations were applied to the Malmo DCQ scales (of Step II), eliminating any remaining discrepancies, and creating their equivalents in terms of the scales of the. For example, before this final step in adjustment, the mean and standard deviation of the Malmo (DCQ) skill discretion scale scores (of Step III) were and 6.239, respectively. Those of the skill discretion scale scores from the same population were and 6.028, respectively. Thus, at first, the Malmo DCQ scale scores were converted to their respective z-scores (i.e., (skilldi-37.04)/6.239) in the Step IV of Table 2), which functioned as the new z-scores with mean and standard deviation of the (i.e.,

5 TESTING TWO METHODS TO CREATE SCALE SCORES 50 Percent Jobinflu() Jobinflu(DCQ) 10 0 Missing Response Note. Job influence question: lot of say question in the and decide on what you do in your work? question in the DCQ. Response categories: 1 = strongly agree to 4 = strongly disagreefor the. 1 = Often to 4 = Never for the DCQ. Figure 1. The response frequency comparison of job influence question from the and DCQ on the Malmo test sample (n = 682). Note. Job influence question: lot of say question in the and decide on what you do in your work? question in the DCQ. Response categories: 1 = strongly agree to 4 = strongly disagree for the. 1 = Often to 4 = Never for the DCQ. (((skilldi 37.04)/6.239) ) in the Step III of Table 2). This final correction step thus results in only minor additional changes in the scores. The final equivalent Malmo scale scores from the original Malmo DCQ sample (n = 6,072) were calculated through the process of Steps II, III, and IV. Regression method alternative to Method I. A methodologically simple alternative method was also employed. The scales scores from the random population survey (n = 682) were regressed on the DCQ scales scores yielding a regression coefficient and a constant term that could then be used to make DCQ scale scores equivalent to scale scores. The regression equations were reported in Appendix II. Table 2. The Process of Correcting Malmo DCQ Job Scales to the Equivalents of Job Scales Step I: Administration of both and -like questionnaire in a random population survey (n = 682) Step II: Adjust the frequency levels of the Malmo DCQ questions to approximate levels. After this step the Malmo DCQ questions are as close as possible to their equivalents Malmo DCQ Jobrepet = jobrepet 0.5 Jobhiski = jobhiski 0.5 Jobfreed = jobfreed +0.5 Jobinflu = jobinflu 0.5 Jobextm = jobextm 0.5 Step III: Create the scales with level-corrected versions of the Maimo-DCQ questions, via simple multiplication Skilldi = (joblearn + (5-jobrepet) + jobcreat + jobhiski) Deciaut = (jobinflu + (5-jobfreed)) Psy dem = ((jobfast + jobhard) 3) + ((15-(jobconfl + jobexces + jobextm)) 2) Decilat = (skilldi + deciaut) Step IV: Apply the final z-score correction to the level-corrected question-based scales (only minor changes occur here) Skill Discretion = ((((skilldi 37.04)/6.239) 6.028) ). Decision Authority = ((((deciaut 33.13)/9.053) 7.941) ). Psych. Demands = ((((psy dem 33.94)/6.769) 6.432) ). Decision Latitude = Skill Discretion + Decision Authority Note. jobrepet: repetitive work item, jobhiski: high skill level item, jobfreed: little decision freedom item, jobinflu: lot of say item, and jobextm: enough time item. 193

6 KARASEK ET AL. Comparability tests for finally converted scale scores. The means and standard deviations of Malmo converted scale scores were compared with those of the remainder of the JACE- data (ISCO major-group composition adjusted, Karasek et al., 2003) and the original U.S. Quality of Employment Surveys (QES) database (Karasek & Theorell, 1990) which was used to study the original. The Pearson correlations of skill discretion, decision authority, and decision latitude (combined skill discretion with decision authority), and psychological demands scales and correlations of the scales with age and education were examined and compared with those of other samples of the JACE (Karasek et al., 2003). Cronbach s alphas of skill discretion, decision authority, and psychological demand scales from both the DCQ and the on the test sample were reported. Pearson correlations of the scales between the and DCQ were also reported with the maximum possible correlations between the imperfect measures (formula , 1 Cohen & Cohen, 1983, p. 69). Cronbach s alphas of the respective DCQ and scales were used as the reliability values in the formula. The agreements of high-risk populations (high job strain populations) from different instruments (the DCQ, DCQ [Method I applied], and on the test sample [n = 682]) were assessed with Kappa coefficients. For practical reasons, sensitivity and specificity statistics of the identification of job strain subjects with the DCQ and the DCQ (Method I applied), arbitrarily using the as the standard were also reported. The job strain term has been most commonly utilized as a simple dichotomous term based on mean cut point of the two demand/control model scales (below the population mean on decision latitude and above the mean on psychological demands, see Figure 2a) although other forms have been recommended. To reduce the problem of including the misclassification-prone center of the population distribution, different formulations emphasizing more extreme exposures have been advanced (see also Karasek, Theorell, Schwartz, Schnall, Piper, & Michela, 1988; Landsbergis, Schnall, Warren, Pickering, & Schwarz, 1994). They exclude the segment of the strain population that is the closest to the population mean (Figure 2e). We have developed such a new job strain term methodology. It involves dividing both the psychological demands and the decision latitude scales into mean-based tertiles (using means ±0.43 [standard deviation] for cut-points) and mean-based quartiles (using means and means ± Formula (Cohen & Cohen, 1983, p. 69). r x y = r xy rxx r yy (r x y : attenuation (unreliability) corrected correlation between the observed scores (X and Y), r xy : correlation between the observed scores (X and Y), r xx : reliability of X, andr yy : reliability of Y). [standard deviation] for cut-points), which are then used to form bivariate distributions of 9 and 16 cells, respectively (Figure 2b and Figure 2c). For dichotomization to identify a high job strain population in the context of the mean-based tertiles and mean-based quartiles, the three corner cells were defined as high job strain and the remainder as low job strain (Figure 2b and Figure 2c). Additionally, it was tested whether median-based cut-points; a simple median-based strain term, a tertilebased strain term (using 33 percentile and 66 percentiles of scales), and a quartile-based strain term (using 25 percentile, 50 percentile, and 75 percentile of scales) yield different agreement results than the above mean and standard deviation-based formulations in the test sample. Furthermore, the mean-based tertile and quartile distributions were used to generate continuous fiveand seven-level job strain scales by stepping incrementally along the diagonal from low to high job strain, which were combined with the diagonally adjacent pairs of cells (for the seven-level scale, see Figure 2d). The agreements of DCQ and Method I with the for the five- and seven-level job strain scales were reported with both the correlation coefficients and their maximum possible correlations (formula , Cohen & Cohen, 1983, p. 69). In this case, the reliability values of the job strain levels for each instrument in the formula were indirectly calculated by using the formula (Cohen & Cohen, 1983, p. 69). That is, inputting the correlations between decision latitude and psychological demands and Cronbach s alphas of those scales into the formula Method II in Two Sub-Sites of Milan Converting the scale scores. Method II was one component of a multi-site database checking procedure employed by the Center (for details, see Karasek et al., 2002). This procedure consists of four steps: a) translation validation test, b) basic variable property reviews, c) scale and variable content-based validity review, and d) detailed investigations of unusual associations. These steps are not mutually exclusive, but intertwined so that this checking cycle continues until there are no more suspected errors in the database when compared to reference databases such as the combined three U.S. QES surveys (Karasek & Theorell, 1990, Appendix I). 2 Formula (Cohen & Cohen, 1983, p. 69). r (A B)(A B) = [(r AA + r BB )/2] r AB 1 r AB (r (A B)(A B) : reliability of the difference between the observed scores (A and B), r AA : reliability of A, r BB : reliability of B, and r AB : correlation coefficient between A and B). 194

7 TESTING TWO METHODS TO CREATE SCALE SCORES Figure 2. Dichotomous and continuous definitions for job strain groups. The examination of patterns of missing values of variables (questions) by sub-site of Milan sample reconfirmed the fact that the MONICA questionnaire was administrated in two sub-sites of Milan sample (n = 970, roughly 20% of the Milan sample). The wordings of the eleven questions in the Milan MON- ICA questionnaire were compared to the equivalents of. The one question ( I decide how much work I have to do ) of the decision authority scale in the Milan MONICA questionnaire was dropped because the wording of that question was not close enough to the wording to be acceptable. A weighting system was used to achieve the same weight as the scales (Method II which is equivalent to Method I, Step II). The skill discretion and decision authority scales were multiplied by factor of 1.5 or 3 to compensate for missing questions in the process of calculating the final scale scores. This simple and obvious procedure can create potentially useful estimated means, however, the standard deviation will be inflated. Comparability tests for finally converted scale scores. As in Method I, the means and standard deviations of Milan converted scale scores were compared with other standards (ISCO major-group composition adjusted; Karasek et al., 2003; Karasek & Theorell, 1990). The correlations of skill discretion and decision authority scales and items, age, and education were examined and compared with those of the remaining foursubunits (used the ) of the Milan sample and those of other samples of the JACE (Karasek et al., 2003). All analyses were conducted using a standard statistical analysis program (SPSS version 10.0). Results Interchangeability of Scale Scores from the Malmo DCQ Questions (Method I) Table 3 shows that the means and specially standard deviations of each scale between the new converted Malmo data, other JACE samples, and QES U.S. data were similar. The average correlation coefficients between decision latitude and psychological demands scales in the JACE- database were 0.59 for men and 0.52 for women (Karasek et al., 2003), which were very consistent with those of the U.S. QES data (0.56 for men, 0.52 for women, cited from the 195

8 KARASEK ET AL. Table 3. Comparison of Means and Standard Deviations of Converted Malmo and Milan Scale Scores with Those of Other Standard References Men (Means) Women (Means) Malmo DCQ Malmo Test Other US Malmo DCQ Malmo Test Other US (Converted) () JACE QES Milan (converted) () JACE QES Milan Skill disc (SD) Decis aut (SD) Decis lat (SD) Psyc dem (SD) Note. Skill disc. = skill discretion, Decis aut. = decision authority, Decis lat. = decision latitude, Psyc dem. = psychological demand. SD = standard deviation. Malmo DCQ (converted): finally converted Malmo DCQ data (n = 6,072) through Steps I, II, and III. Malmo test () = questionnaire on subjects (n = 682). Other JACE = ISCO major group adjusted JACE- data without Malmo and Milan sample. US QES: U.S. quality of employment surveys data (Karasek & Theorell, 1990). Milan = ISCO major group adjusted Milan data with all the six subsites samples. article of Karasek et al., 1998). Relatively low correlation coefficients (0.38 for men and women) between skill discretion with decision authority were observed in Malmo sample. When the same formulas used in Malmo sample (Step II, Table 2) were applied to other samples in place of the official formulas for the skill discretion and decision authority scales, the averages of correlations between the two scales in the total JACE decreased: from 0.59 to 0.43 for men, and from 0.52 to 0.36 for women. Thus the missing questions appear to account for the correlation difference. Age in Malmo sample had no strong correlations with any scales, as in the total JACE (Karasek et al., 2003). The years of education in Malmo sample had fairly strong and positive correlations with skill discretion, decision authority, and decision latitude, again as in the total JACE. There were slight, but consistent, negative correlations between age and educations in Malmo sample for men and women, which was the same case in the total JACE. The Cronbach alphas in the and the DCQ were as follows, respectively: 0.79 and 0.68 for skill discretion, 0.75 and 0.74 for decision authority, and 0.68 and 0.75 for psychological demands. The correlations of psychological demands scales with decision latitude in the DCQ and were 0.03 and 0.08, respectively. The internal correlations between the skill discretion and decision authority were markedly lower for the DCQ (0.38) than for the (0.70). The correlations between the and DCQ scale scores were 0.76 for skill discretion, 0.60 for decision authority (maximum possible correlation =.81), and 0.71 for psychological demand, respectively. The kappa coefficients for high job strain between the DCQ and, using the as the standard, were using means for cut-point, using meanbased tertiles, and (95% CI: to 0.604) using mean-based quartiles. After application of Method 1, kappa coefficients between the DCQ (Method I applied) and were (means for cut-point), for mean-based tertiles, and (95% CI: to 0.593) using mean-based quartiles for cut-points (Table 4). The specificities of the DCQ and DCQ (Method I applied) for low job strain were uniformly high (range 0.84 to 0.93), particularly for the high job strain definition using mean-based quartiles for cut-points (0.93). The sensitivities of the DCQ and DCQ (Method I applied) for high job strain were uniformly and moderately high (range: 0.51 to 0.70). The lowest was observed in the high job strain definition for the DCQ using means for cut-points and the highest in the high job strain definition for the DCQ and DCQ (Method I applied) using mean-based tertiles for cut-points (0.70). Table 4 also shows that the kappa statistics for high job strain between the DCQ and and between DCQ (Method I applied) and, using median-based cut-points, differed only slightly. The means-based formulation kappa was slightly higher than the medianbased in the case of tertiles, while slightly lower in the case of quartiles case (0.529 vs ; Table 4). The kappa statistics for high job strain group between the observed scales and the predicted scales from the DCQ scales using the simple regression equation adjustment were slightly lower than those between the DCQ (Method I applied) and (e.g., vs for the high job strain group using means for cut-points). The correlation coefficients between the DCQ and the with the continuous five- and seven-level job strain scales (diagonals) were 0.64 (maximum possible correlation =.88) and 0.70 (maximum possible correlation =.96), respectively. The correlation coefficients of the DCQ (Method I applied) with the between the five- and seven-level job strain diagonals were

9 TESTING TWO METHODS TO CREATE SCALE SCORES Table 4. The Kappa Statistics for High Job Strain from the DCQ, DCQ (Method I Applied), and (Predicted from the Regression Equations) Using the as the Standard in the Malmo Test Population (n = 682) Cut-Points for High Job Strain Mean - Mean - Median - Median - Scale sources for comparison Mean Tertiles a Quartiles b Median Tertiles c Quartiles d DCQ DCQ (Method 1 applied) (predicted from the regression equations e ) Sensitivity/Specificity DCQ-.51/.91.70/.84.57/.93.65/.87.66/.84.68/.90 DCQ (Method 1 applied)-.59/.88.70/.85.65/.90.61/.88.72/.81.66/.90 (predicted from the regression equations)-.51/.91.70/.83.57/.93.65/.86.65/.84.68/.90 Mean Str = 0 Str = 1 Kappa = DCQ Str = 0 DCQ Str = Specificity = Sensitivity 24.6% = % Total n = 651 Mean Str = 0 Str = 1 Kappa = M/dcq Str = 0 M/dcq Str = Specificity = Sensitivity 24.6% = Total n = 23.6% 651 M/dcq Str = 0 M/dcq Str = 1 Meantertiles Meanquartiles M/dcq Str=0 M/dcq Str=1 Str = 0 Str = 1 Kappa = Specificity = Sensitivity 26.0% = Total 29.5% n=651 Str = 0 Str = 1 Kappa = Specificity = Sensitivity 15.4% = Total n = 18.6% 651 a Mean-tertiles: mean ± 0.43 (standard deviation). b Mean-quartiles: mean and mean ± 0.67 (standard deviation). c Median-tertiles: 33 percentile and 66 percentile. d Median-quartiles: 25, 50, and 75 percentiles. e See the Appendix II. Str = 0: low job strain group, Str = 1: high job strain group. M/dcq: DCQ (Method I applied, see Table 2). (maximum possible correlation =.88) and 0.69 (maximum possible correlation =.94), respectively. Interchangeability of Scale Scores from the Milan MONICA Questions (Method II) The final modified scale scores of Milan sample including two sub-sites using the MONICA questionnaire were comparable with those of other standard data (see Table 3). The scale scores for skill discretion, decision authority, and decision latitude were relatively low, partly because of the socio-demographic characteristics of Milan sample (lowest years of education and narrow distribution of occupations within each of ISCO major group). In the Milan MONICA sample, most correlation coefficients between skill discretion items, the one item of decision authority, age, and education, were in expected directions, although weaker than normal in the six Milan sub-sites overall. However, for males (n = 88) in the Milan MONICA sample, three problematic correlations were observed: a) correlation of repetitive work question was not in the hypothesized direction (= positive) with other skill discretion questions, b) education was negatively associated with decision authority scale (r =.14), and c) the one question of decision authority and skill discretion had no positive correlation. Interestingly, in a sub-site women sample of the Milan sample using the, a very weak and negative correlation of education with decision latitude was observed. The population of that sub-site consisted of those who were transferred to this working group due to reduced work capacities, as a consequence of health problems or of burnout processes. Milan women s sample had a strong and negative correlation (r =.44) between age and education, which might reflect a major intergeneration education differential. Discussion This study was the first to assess the validity of converting scale scores from -like questionnaires to the equivalents of the scales. Despite different response categories and different items of psychological demand and decision latitude scales between the, DCQ, and a transformed MONICA questionnaire, the converted scale scores from two different questionnaires in Malmo (n = 6,072) and two subsites 197

10 KARASEK ET AL. (n = 970) of Milan samples through the use of special scoring algorithms generally appeared to be usable approximations of the. This comparability was further illuminated in another article showing scale intercorrelations, means, and variance consistency across populations (Karasek et al., 2003). Application of these two scale correction methods made it easier to compare the psychosocial job characteristics of the Malmo and the Milan two sub-sites samples with those of other JACE samples. Application of Method I allows localization of the sources of discrepancy: the primarily source of the discrepancy in scores between the and the DCQ was differential distribution of responses in several questions. Adjustment was also necessary for the different number of questions in the scales. Thereafter, few major scale differences remained to be finally adjusted in the z-scoring correction. At one level, there is significant similarity across these instruments after adjustment. The skill discretion and psychological demands scale scores from the DCQ were very highly correlated (greater than 0.70) with those of the as expected, particularly given the maximum possible correlation (=.81). However, the correlations between the skill discretion and decision authority, sub-components of decision latitude, were substantially lower for the DCQ than for the, showing the decision latitude construct to be a more homogenous construct. In Malmo sample, where a scale correction method derived from an independent sample containing both questionnaires was applied, no substantial problems related to the converted scales scores were found in examining correlations of the scales and the converted -like (DCQ) scales by age and education, as well as in comparisons of them with other standard references values. The reduced correlations between skill discretion and decision authority in the DCQ appears to arise from questions missing with respect to the. In Milan, the MONICA instrument was closer to the original scales: two of three scales had either exactly the same or very similar question texts. In this case, a less quantitatively rigorous, procedure-based scale correction process was applied. While scale levels and most of the scale inter-correlations appeared appropriate, several correlations involving education and decision latitude were not as predicted from reference populations. The four-step procedure used in the Milan case (Method II) reviews each component question, then each scale, then correlations, and finally occupational group mean scores. The steps are not mutually exclusive, but intertwined in a checking cycle that continues until all major suspected uncertainties or non-comparabilities in the database are accounted for. This second method also appeared to be useful for researchers and has sometimes been used by the Center as the only available approach, but its accuracy 198 can only be inferred from fairly labor-intensive reviews of comparative scale properties, and it has been difficult to report results concisely. At a more detailed level of comparison between and -like scales, the goal of insuring that the job strain populations assess risk in a comparable manner across instruments is more difficult. To begin with, most of the questionnaires involved have only moderately high statistical reliability. Indeed, the has a design goal of including a range of related sub-constructs in the same scale, which automatically limits internal scale coherence. The result is error variance in the scales that will diminish the possibility of high correlations between job strain groups based on slightly differing instruments in the same population. To partially assess the effects of this, we have reported analyses reflecting maximum possible correlations corrected for the error variance, as a comparison base. At the level of the job strain construct, the agreement between instruments was only moderate even after the Method I was applied. The kappa statistics for the agreement between the -based and DCQ-based definitions of the high job strain group fall in the moderate range (Landis & Koch, 1977). As a reference, it could be noted that these kappas represent higher levels of agreement than is found between alternative assessment strategies for many physical measures of workplace ergonomic risk, which might be presumed to be less prone to interpretative assessment error (Buchholz, Punnett, & Park, 2003; Burt & Punnett, 1999). Nevertheless, the job strain assessment from the DCQ and DCQ (Method I applied) included considerable numbers of misclassifications by comparison to the original. Arbitrarily using the as the gold standard in this inter-site study, the DCQ and DCQ (Method I applied) presented high consistency with the for identifying low job strain cases (only 7% to 19% as false positives), but had 28% to 49% high job strain cases misclassified as false negatives. Furthermore, the kappa statistics for the high job strain group (using mean-based cut-points) between the DCQ and improved only marginally when the sophisticated three-step scale correction (Method I) was employed. It is difficult to claim more than approximate comparability of job strain classification even after the correction procedure, probably because of the underlying moderate reliability of these types of instruments. The regression-based adjustment method, a simple alternative to Method I suggested by an anonymous reviewer, yields scores that are similar (slightly less successful) in terms of kappa statistics, and thus, we cannot argue strongly against it. However, it provides the researcher with no information about the source of the discrepancies, the advantage of Method I. Our results also show that mean-based and medianbased job strain definitions yield very similar findings,

11 TESTING TWO METHODS TO CREATE SCALE SCORES giving us no empirical basis for recommending one approach over the other. However, theoretical argument for use of the mean-standard deviation based scores is worth stating explicitly (these have been recommended in other Demand/Control model contexts; Karasek et al., 1988; Karasek & Theorell, 1990). The basis of the argument is sociological. It claims that each wellbounded social collectivity (such as a nation) can be considered a unity with respect to distribution of many important social parameters. The primary dynamic of differentiation of these parameters is then the social division of labor that distributes these parameters around a mean value, with compensating advantages and disadvantages/characteristics measurable by the standard deviations. For example, the society will create both high and low decision latitude jobs, and jobs with both primarily mental loads and jobs with primarily physical loads (Durkheim, 1893/1964). A well-known implication of this theory is that the use of national means to define job strain may raise the problem that there are few true job strain subjects in some study samples. A less-discussed implication is that the sociological argument above also implies that the standard deviation units of population parameters (of decision latitude, for example) will have similar meaning across different societies, even if the absolute scale units do not, and thus can be used as a measurement standard across many populations. Thus, social policy analysis for example, costs and benefits of workplace changes in different countries is more feasible when means and standard deviation parameters are employed. Thus, we continue to recommend the mean-based definitions. Improving Job Strain Classification Agreement The better empirical news is that use of special tertile- and quartile-based job strain definitions which remove the potentially ambiguously classified mid-populations improved kappa agreement statistics and sensitivities over the dichotomous job stain terms, although specificities remained relatively unchanged. Such a definition of job strain is also consistent with an analytic redefinition of the four-quadrant full Demand/Control model (active, passive, high strain, low strain) adding a mid-population fifth group (Figure 2e) (the tertile-based job strain definition has the disadvantage that overlaps the boundary of the active and passive quadrants as defined in the often-used mean-based dichotomous job strain term (Figure 2a)). If confirmed by further research testing predictive validity, such formulations could be useful for epidemiologists wishing to improve consistency of results testing job strain/demand-control model. A second promising variation in job strain term formulation is to use multiple level job strain scales. Clearly there was stronger agreement between the and the DCQ with the five- and seven-level forms than with the dichotomous form that is commonly used. The observed correlations between the different forms of job strain scale can be considered quite high (greater than 0.7), given the computed maximum possible correlations ( ). However, a flag might be raised here because the extent of multiplicative interaction between demand and control would remain unexplored using such a formulation (many studies have of course lacked statistical power to test for differences of a reasonable size between linear and multiplicative forms). Here, the recommended solution for such multi-level scales would be to allow modeling of weaker associations between job stain and outcome at lower strain levels, and allow for an increasing association at higher strain levels (truncating to avoid very high levels), indirectly assessing the interaction. The literature that has confirmed the associations between job strain and cardiovascular diseases (CVD) are based on a diversity of measures of decision latitude and psychological demands. Many of them are putatively close to the two -like questionnaires reviewed here and other -like questionnaires (sections of Whitehall study and the Copenhagen instrument). However, we found that the use of theoretically similar instruments could generate fairly different job strain populations. This differential misclassification (actually disagreement in classification) of exposure could bias the effects of job strain on health outcomes toward the null in multi-study comparisons (for other sources of bias toward the null in the cardiovascular disease studies, see Belkić et al., 2000; Belkić, Landsbergis, Schnall, & Baker, 2004). However, we also identified two methods that appeared to diminish this problem, and it would be reasonable to expect that other methods could be developed in the future. We would certainly recommend that this approach of assessing multiple instruments in the same sample should be applied in test samples for several other instruments, notably the Whitehall and Copenhagen questionnaires, since multiple forms will likely continue to be used and need integrative interpretation. The final verdict on the interchangeability of formulations will come as their predictive validity is assessed in the future. References Araújo, T. M., & Karasek, R. A. (2005, March). Reliability and validity of the Job Content Questionnaire (): An empirical test in a developing country comparing formal and informal jobs. Paper presented at the 4th International Conference on Work Environment and Cardiovascular Diseases, Newport Beach, CA. Belkić, K., Landsbergis, P., Schnall, P., & Baker, D. (2004). Is job strain a major risk factor for cardiovascular disease? Scandinavian Journal of Work, Environment and Health, 30,