APPENDIX 11A An informal analysis of multilevel Imagine we are studying the blood pressure of a number of individuals (level 1) from different neighbourhoods (level 2) in the same city. We start by doing a simple ecological analysis with blood pressure aggregated at the neighbourhood level. Figure 11A.1 shows the mean blood pressure of the city (M c ) and the specific means of the neighbourhoods. This gives the neighbourhood residual (R n ) (see Appendix 11D), corresponding to the difference between the city mean and the neighbourhood mean. The of the neighbourhoods (V n ) is a summary measure of the overall differences around the overall average blood pressure (M c ). To calculate it, we sum the residuals and divide by the number of neighbourhoods (N n ). However, the sum of the residuals is 0 (the positive and negative residuals cancel each other out), so before the summation (Σ) we take the square of every residual. In this way, the V n is the average measure of the neighbourhood differences and always has a positive value: V n = Σ (Rn2 ) N n Formula 11A1 We now perform a simple individual-level analysis. We can see in Figure 11A.2 that the individual residual (R i ) is the difference between the individual blood pressure (BP i ) and the overall M c (the same M c as in the ecological design). The R i is indicated as a green shadow. The individual (VV i ) is the sum of the squared individual residuals (Ri 2 ) divided by the number of individuals (N i ): V i = Σ (Ri2 ) N i Formula 11A2 Finally, in Figure 11A.3 we see a multilevel design in which the individual residual (green shadow) has been decomposed into within-neighbourhood (yellow shadow) and between-neighbourhood (blue shadow) components. Therefore the BP i is the sum of M c plus R n plus R i. In this case, the total individual-level (V Ti ) can be decomposed into a between-neighbourhood component (V n ) and a within-neighbourhood component (V ni ). Once we understand the multilevel components of individual, the concept of a partition coefficient (VPC) is evident. The VPC is just the share of the total individual that is at the higher level (i.e. the neighbourhood). In the case of multilevel structures with a simple hierarchical construction (as in the case of individuals within neighbourhoods), the VPC corresponds to the intraclass correlation coefficient (ICC): ICC = V n V n + V ni Formula 11A3 The term correlation is used because the ICC corresponds to the correlation in the outcome (e.g. blood Figure 11A.1 Mean blood pressure of a city (M c ) and specific means of the neighbourhoods within it. 1
2 Appendix 11A: An informal analysis of multilevel City mean blood pressure (M c ) (BP i ) (R i ) level residual Figure 11A.2 residual (R i ) : the difference between individual blood pressure (BP i ) and overall mean blood pressure (M c ). Neigbourhood level residual level residual City mean V n Neigbourhood Figure 11A.3 Multilevel design deconstructing the individual residual (green shadow) into withinneighbourhood (yellow shadow) and between-neighbourhood (blue shadow) components. pressure) between two individuals randomly chosen from the same neighbourhood. The ICC can be defined as the extent to which members of a group (cluster, class, area) resemble one another more than they resemble members of other groups. The ICC provides highly relevant information for understanding multilevel effects [2]. If we assume that the second-level units (e.g. the neighbourhoods) are relevant for understanding individual differences in blood pressure, a share of the total individual differences will be found at the neighbourhood level. The larger this share, the more relevant the neighbourhood influence; if the ICC is close to 0, the neighbourhoods are not relevant to the outcome and should not be given too much consideration. VPC can be calculated for both continuous (e.g. blood pressure) and discrete (e.g. use or no use of medication) variables [4,53]. For discrete variables, alternative measures to the VPC, such as the pairwise odds ratio (PWOR; a measure of clustering) [35,54] and the median odds ratio (MOR; a measure of heterogeneity) [50], also provide adequate interpretations of the general contextual effects (see Appendix 11C). In addition, novel measures based on the area under the ROC curve (AU-ROC) are under development [36,42,43]. Example one: the contextual effect of a random sample Continuing with the analysis of blood pressure, imagine that we draw 100 random samples from 100 individuals among the population of a given city. We then perform a
Appendix 11A: An informal analysis of multilevel 3 multilevel analysis with the random samples at the second level (i.e. instead of neighbourhood, we have random samples at the second level) and calculate the ICC. What do we expect the value of the ICC to be? The correct answer is: The ICC will always be close to 0, because the fact of belonging to one or another random sample does not have any influence on individual blood pressure. Example two: the astute physician and the sample size Imagine that a colleague has done a study on three patients, measuring their blood pressure at a single point in time. He consults a statistician, who tells him that the study has no statistical power he needs at least 300 patients. The colleague thinks for a couple of days and decides to replicate each patient s information 100 times, so that instead of three measurements, he has 300. The colleague goes back to the statistician and gives her the new data. After 15 minutes, she tells him that the effective sample size [23] of the study is 3 because the ICC (intrapatient correlation of blood pressure measurements) is 100%. Further, she says that what he has done is a case of scientific misconduct and she will report the incident to the ethical committee. What the colleague did was to create three artificial clusters of 100 patients. The analysis needs to consider that the patients outcome is correlated in each cluster. In this case the intracluster correlation coefficient is 1 (100%). The statistician corrected the (faked) sample size of 300 by taking into account the intracluster (i.e. intrapatient) correlation using a formula denominated design effect (DE) [55]: DE = 1 + (n c 1) ICC Formula 11A4 where n c is the number of individuals in the cluster (in c our case, measurements in a single patient), so: DE = 1 + (100 1) 1 DE = 100 Thereafter, the statistician obtained the effective number of measurements in the sample as: c n EN = N c DE Formula 11A5 where N c is the (faked) number of individuals (in our c case, measurements). The effective number of individuals becomes: = 100 EN 300 100 EN = 3 When we investigate clusters (e.g. wards, hospitals, healthcare centres, neighbourhoods, municipalities, counties, countries, etc.), we need to consider that individuals may be correlated within them. We must correct for this intracluster correlation in order to obtain the real statistical sample and so provide correct estimations of uncertainty (SE). The higher the ICC, the smaller the effective sample and the lower the statistical power, and thus the wider the confidence interval. Statistically, two different individuals can be considered (almost) one, as in the case of homozygous twins.
APPENDIX 11B Are differences between averages relevant to understanding total individual variation? When evaluating medical practice variation, the main issue is not to evaluate differences between institutional averages but, rather, to quantify to what degree those differences are relevant to understanding the total individual variation. Figure 11B.1 presents two (fictive) ecological analyses from two different cities, showing the mean systolic blood pressure (SBP) of a number of neighbourhoods according to average level of socioeconomic deprivation (expressed as standardized neighbourhood income, where higher values denote more deprivation). We can observe clear differences between neighbourhoods both in their mean SBP and in their degree of deprivation, with an evident association between the two variables, such that the greater the deprivation, the higher the average blood pressure. In fact, the regression coefficient is similar (β 1.2) in Figure 11B.1a and in Figure 11B.1b. A simple visual observation suggests that the results of the analyses are highly comparable in both cities. Based on this analysis, we would launch similar public health interventions in both cities (e.g. promote blood pressure control in the more deprived neighbourhoods). Figure 11B.2 is based on the same studies as Figure 11B.1, but here a multilevel analysis shows both the average values of the neighbourhoods and the individual-level values. Using this information, we can evaluate the intraclass correlation coefficient (ICC), showing the share of total individual differences in SBP that are at the neigbourhood level. We can observe two very different situations. In Figure 11B.2a, the individual within the neighbourhoods is very high and, therefore, the ICC is very low. In fact, most of the variation is between individuals. However, in Figure 11B.2b, the intraneighbourhood individual is small and most of the individual is at the neighbourhood level, so the ICC is very high. Observe that the association between both average blood pressure and neighbourhood deprivation is similar in both cities A and B (β 1.2) and is about the same as was found in the ecological analyses of Figure 11.8. That is, we can find similar differences between averages with very different scenarios of around the means. If the within-neighbourhood is large relative to the between-neighbourhood, there is a considerable overlap of the individual distributions (Figure 11B.2). ECOLOGICAL ANALYSIS Mean systolic blood pressure (mm Hg) 143 142 141 140 139 138 137 136 2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0 (a) Standardised neighbourhood income Mean systolic blood pressure (mm Hg) β = 1.2 143 β = 1.2 142 141 140 139 138 137 136 2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0 (b) Standardised neighbourhood income Figure 11B.1 Fictive ecological analyses comparing systolic blood pressure (SBP) with socioeconomic deprivation. 4
Appendix 11B: Are differences between averages relevant to understanding total individual variation? 5 Figure 11B.2 Variation in systolic blood pressure (SBP) between and within neighbourhoods and association between an index of socioeconomic deprivation and SBP. The β-coefficient is of the same magnitude in both (a) and (b), but the intraclass correlation coefficient (ICC) is much higher in (b) than in (a). Systolic blood pressure (mm Hg) (a) 250 143 240 ICC = very low 230 β 1.2 142 220 210 200 141 190 180 140 170 160 150 139 140 130 138 120 110 Systolic blood pressure (mm Hg) 100 90 80 2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0 136 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0 Standardised neighbourhood income (b) Standardised neighbourhood income 137 ICC = very high β 1.2 For instance, say we aim to launch an intervention to reduce blood pressure in this fictive city. In situation (b), we should focus on specific neighbourhoods. However, in situation (a) it would not be appropriate to do so as many individuals in the deprived neighbourhoods would receive unnecessary treatment, while many individuals in need of treatment in the nondeprived neighbourhoods would not be treated. An extended explanation of these ideas can be found elsewhere [2,41]. Observe, however, that a concrete context such as the neighbourhood might be of relevance in conditioning an individual s outcome (e.g. SBP) but still provide a very low ICC. To understand this idea, imagine we aim to launch an intervention to reduce blood pressure in a city and plan to focus on the neighbourhood level. Figure 11B.3 shows the differences in mean SBP between neighbourhoods before (PRE) and after (POST) an intervention. The thick continuous line represents the overall SBP, while the dotted thin lines represent the SBP means of the neighbourhoods. Figure 11B.3a shows that before the intervention, there is a high neighbourhood variation in SBP, which corresponds with a high partition coefficient (VPC; e.g. 25%). We can also observe a high overall SBP mean. Following the reasoning just discussed, we conclude that the neighbourhoods are of relevance for understanding individual differences in blood pressure, and suggest that an intervention should be directed at the bad neighbourhoods (i.e. those with the higher SBP mean). Therefore, we start an intervention, and after some time we observe that the bad neighbourhoods have considerably improved the SBP of their patients and have achieved similar SBP means to those of the good neighbourhoods before the intervention. Thus, after the intervention, the ICC is only 4%. This low VPC suggest that the neighbourhood level no longer explains the overall individual differences in SBP, since all the neighbourhoods have a similar SBP mean and the overall mean SBP has improved. In Figure 11B.3b, all the neighbourhoods show a high SBP, so the neighbourhood level does not explain the Figure 11B.3 Differences in mean systolic blood pressure (SBP) between neighbourhoods before (PRE) and after (POST) an intervention. Systolic blood pressure VPC = 25% VPC = 4% (a) PRE POST Systolic blood pressure VPC = 4% VPC = 4% (b) PRE POST
6 Appendix 11B: Are differences between averages relevant to understanding total individual variation? overall individual difference in SBP (i.e. all the neighbourhoods have the same inappropriately high SBP mean). In this second scenario, the ICC is also low and the overall mean SBP is high (i.e. bad). Therefore, in this case, we launch an intervention directed at all neighbourhoods. After some time, we observe that all neighbourhoods have considerably improved the SBP of their patients. We find that the VPC remains low but the overall mean SBP is better (i.e. lower). Crucially, although the neighbourhood level conditions individual SBP in both scenarios, the ICC provides information of relevance in planning prevention strategies. Thus, in Figure 11B.3a (preintervention), the intervention should be directed at specific neighbourhoods with a high SBP, while in Figure 11B.3b the intervention should be directed at the whole population, not to a subset of specific neighbourhoods. The multilevel regression analysis (MLRA) approach combines an analysis of averages and an analysis of components of. In doing so, it differs from the classical studies, which focused only on differences between averages. These classical studies often stated that there were large differences between (say) neighbourhoods. However, this prompts the question of what is a large [30]. MLRA solves this question by considering that large geographical differences exist when the differences between averages explain a large share of the total individual differences [10].
APPENDIX 11C The median odds ratio In simple terms, the median odds ratio (MOR) can be interpreted as the increased (median) odds of having a given outcome if an individual moves to (say) another neighbourhood with higher risk. The MOR is defined as the median value of the distribution of odds ratios (ORs) when randomly picking pairs of individuals with the same covariates but from different higher-level units (e.g. healthcare units, neighbourhoods) and comparing the individual from the highest-risk unit with that from the lowest-risk unit. We compute the MOR as: 2 MOR exp ( 0.95 σu ) Formula 11C1 In the absence of variation, MOR = 1. The higher the MOR, the more relevant the higher-level unit to understanding the individual outcome. The MOR translates the neigborhood to the widely used OR scale, which makes the MOR comparable with the OR of individual or neigbourhood variables. Further information can be found elsewhere [4,50,56]. Even if both the intraclass correlation coefficient (ICC) and the MOR are based on the same neigbourhood ( u 2 σ ), the MOR is conceptually a measure of heterogeneity, rather than of clustering (like the ICC). The MOR is an alternative way of expressing neigbourhood from a probabilistic perspective. 7
APPENDIX 11D Shrunken residuals or information-weighted residuals In Appendix 11A, we discussed the concept of a residual. In multilevel regression analysis (MLRA), the residuals of the higher-level units (e.g. healthcare unit, HCUs) are termed shrunken residuals. Shrunken indicates that the residual is weighted by a shrinkage factor (SF) (Formula 11D1), which reflects the information available: Therefore, when weighting by SF, if the number of patients in a HCU is small and the individual is large, the HCU is shrunken towards the overall mean systolic blood pressure (SBP). The SF provides an improved estimation of the rank that avoids fallacious results produced by statistical noise. SF = Variance between HCUs Variance between HCUs Variance within HCUs + Number of patients Formula 11D1 8
APPENDIX 11E Variance as a function of individual variables: the random intercept and random slope multilevel analysis In Appendix 11A, we discussed the concept of and how the total individual can be decomposed into within- and between-contextual unit (e.g. neighbourhoods) components. In multilevel regression analysis (MLRA), the simplest model has only a random term for the intercept. This model assumes that all individuals in the same neighbourhood share the same contextual unit residual value. In more elaborate MLRA models, we introduce individual-level variables and estimate the association between the individual variable and the outcome. In such models, we assume that the individual-level association is the same in all contextual units. Figure 11E.1a shows the association between body mass index (BMI) centred around the mean and systolic blood pressure (SBP) in different populations of the World Health Organization (WHO) Monica Project [37]. Since the slope is the same in all populations, the population (i.e. the population differences) is the same for all individual BMI values, corresponding with the population intercept. However, we can discard this assumption and consider that the population context influences people differently, so that the strength of association between BMI and SBP is different in different populations. In statistical terms, we say that we allow the slope of the regression coefficient to be random at the population level. Figure 11E.1b shows this phenomenon. The population thus becomes a function of the individual-level variable. In Figure 11E.1b, we can see that the differences between neighbourhoods are larger for individuals with higher BMI, while in Figure 11E.1a the differences are of the same magnitude for all individuals, no matters their BMI value. Therefore, MLRA allows us to model the at the different levels as a function of specific explanatory variables [3,39]. 194 Neighbourhood differences 220 Neighbourhood differences 171 190 SBP 148 SBP 160 125 130 102 100 11.3 0.0 11.3 22.5 33.8 45.0 11.3 0.0 11.3 22.5 33.8 45.0 (a) BMI (CENTRED) (b) BMI (CENTRED) Figure 11E.1 Association between body mass index (BMI) and systolic blood pressure (SBP) in different populations of the World Health Organization (WHO) Monica Project [37]. Each line represents the population-specific association (slope of the regression line). In (a), the slope is the same for all neighbourhoods; in (b), it is different in different neighbourhoods. In (a), the population differences in mean SBP are the same for all individuals; in (b), they are larger for individuals with higher BMI. 9