The Sedentary Time and Activity Reporting Questionnaire (STAR-Q): Reliability and Validity Against Doubly Labeled Water and 7-Day Activity Diaries

Transcription

1 American Journal of Epidemiology The Author Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please Vol. 180, No. 4 DOI: /aje/kwu150 Advance Access publication: July 19, 2014 Practice of Epidemiology The Sedentary Time and Activity Reporting Questionnaire (STAR-Q): Reliability and Validity Against Doubly Labeled Water and 7-Day Activity Diaries Ilona Csizmadi*, Heather K. Neilson, Karen A. Kopciuk, Farah Khandwala, Andrew Liu, Christine M. Friedenreich, Yutaka Yasui, Rémi Rabasa-Lhoret, Heather E. Bryant, David C. W. Lau, and Paula J. Robson * Correspondence to Dr. Ilona Csizmadi, Department of Cancer Epidemiology and Prevention Research, CancerControl Alberta, Alberta Health Services, Richmond Road Diagnostic and Treatment Centre, 1820 Richmond Road SW, Calgary, Alberta T2T 5C7, Canada ( ilona.csizmadi@albertahealthservices.ca). Initially submitted November 8, 2013; accepted for publication May 19, We determined measurement properties of the Sedentary Time and Activity Reporting Questionnaire (STAR-Q), which was designed to estimate past-month activity energy expenditure (AEE). STAR-Q validity and reliability were assessed in 102 adults in Alberta, Canada ( ), who completed 14-day doubly labeled water (DLW) protocols, 7-day activity diaries on day 15, and the STAR-Q on day 14 and again at 3 and 6 months. Three-month reliability was substantial for total energy expenditure (TEE) and AEE (intraclass correlation coefficients of 0.84 and 0.73, respectively), while 6-month reliability was moderate. STAR-Q-derived TEE and AEE were moderately correlated with DLW estimates (Spearman s ρ s of 0.53 and 0.40, respectively; P < 0.001), and on average, the STAR-Q overestimated TEE and AEE (median differences were 367 kcal/day and 293 kcal/day, respectively). Body mass index-, age-, sex-, and season-adjusted concordance correlation coefficients (CCCs) were 0.24 (95% confidence interval (CI): 0.07, 0.36) and 0.21 (95% CI: 0.11, 0.32) for STAR-Q-derived versus DLW-derived TEE and AEE, respectively. Agreement between the diaries and STAR-Q (metabolic equivalent-hours/day) was strongest for occupational sedentary time (adjusted CCC = 0.76, 95% CI: 0.64, 0.85) and overall strenuous activity (adjusted CCC = 0.64, 95% CI: 0.49, 0.76). The STAR-Q demonstrated substantial validity for estimating occupational sedentary time and strenuous activity and fair validity for ranking individuals by AEE. motor activity; physical activity; reproducibility of results; sedentary lifestyle; questionnaires; validation studies Abbreviations: AEE, activity energy expenditure; CCC, concordance correlation coefficient; DLW, doubly labeled water; ICC, intraclass correlation coefficient; MET, metabolic equivalent of task; STAR-Q, Sedentary Time and Activity Reporting Questionnaire; TEE, total energy expenditure. As the need for comprehensive assessments of daily physical activity has grown, the limitations of existing methods have become apparent (1, 2). Many physical activity questionnaires have been developed and validated, but most are not well suited for the study of new behavioral parameters of interest, such as activity energy expenditure (AEE). In a review of physical activity questionnaires that were validated against doubly labeled water (DLW) the gold standard for estimating total energy expenditure (TEE) in free-living individuals we reported that overall, questionnaire performance in estimating AEE was poor when compared with DLW (3). However, none of the questionnaires were originally designed to estimate AEE. While objective measures continue to evolve in precision and accuracy, they are limited as stand-alone measures when contextual information pertaining to habitual behavior is desired (4). It is generally agreed that, for at least the foreseeable future, there will continue to be a need for logistically feasible, cost-effective tools with known validity that can assess a broad range of habitual behaviors in large populations (4, 5). We designed the Sedentary Time and Activity Reporting Questionnaire (STAR-Q), a past-month activity questionnaire, 424

2 STAR-Q Reliability and Validity Studies 425 to address the need for a self-administered questionnaire that could assess all types and intensities of physical activity, including sedentary behavior, across all activity domains for the purpose of ascertaining AEE among adults in large-scale epidemiologic studies. Following several phases of development and cognitive testing (6), the next step was to determine the STAR-Q s validity and reliability. We assessed validity by comparing STAR-Q-derived AEE and TEE estimates with those derived using DLW. We also compared activities reported in the STAR-Q with intensity- and domain-specific activities reported prospectively in a 7-day activity diary. In addition, we assessed the reliability of the STAR-Q for estimating AEE by administering it on 3 occasions. METHODS The STAR-Q The STAR-Q (see Web Appendix 1, available at oxfordjournals.org/) is a quantitative, self-administered pastmonth questionnaire designed to capture information on all types of activities and sedentary behaviors across all domains: eating, personal/medical care, occupation, transportation, household, yard work, caregiving, exercise, light leisure (e.g., television-watching, personal computer time, reading, hobbies, etc.), stair-climbing, and other activities. Development and cognitive testing of the STAR-Q have been previously described (6). Briefly, detailed physical activity data from 18,838 participants enrolled in the Alberta Tomorrow Project, a Canadian cohort study (7), were used to identify and group common activities with similar energy expenditures to create activity-related questionnaire items. Openended reporting of some activities was also allowed on the STAR-Q. 1 STAR-Q Validation Study (TEE, AEE) STAR-Q1 Past-Month (Day 14) 0 DLW 14 Days 1 Study participants Study participants comprised 1) urban residents of the Calgary, Alberta, Canada, area who were recruited through posters and worksite distribution lists and 2) participants in the Alberta Tomorrow Project. The Alberta Tomorrow Project is a study of a large, geographically dispersed cohort established in 2001 to examine the relationship between lifestyle and chronic diseases in Alberta (7). Invitations to participate in the current study were mailed to Project participants who met eligibility criteria for the current study, based on characteristics reported upon enrollment in the Project. Eligibility criteria were as follows: body mass index (weight (kg)/height (m) 2 ) 35, weight stability (±2.5 kg for at least 3 months) with no intention to gain/lose weight in the near future, not being pregnant or breastfeeding, absence of metabolic disorders (e.g., diabetes, thyroid), and no use of medications that modified water balance. In addition, we set the upper age limit for eligibility at 60 years to avoid incomplete collections of urine due to bladder urine retention. The minimum age of eligibility for participants recruited from the community (i.e., the urban Calgary sample) was set at 30 years, which is comparable to the Alberta Tomorrow Project (age 35 years). All participants provided written informed consent, and ethical approval was obtained from the Alberta Cancer Research Ethics Committee of Alberta Health Services and the Conjoint Health Research Ethics Board of the University of Calgary. Participants received Can$100 upon study completion. Reliability study design To assess the consistency of the STAR-Q for estimating energy expenditure over a longer duration, participants completed the STAR-Q 3 times over a 6-month period (Figure 1). 3 STAR-Q2 Past-Month 6 STAR-Q3 Past-Month 7 Past-Year Diet History Questionnaire (Day 0) 7-Day Activity Diary (Started on Day 15) Figure 1. Overview of the study design for a study conducted to validate the Sedentary Time and Activity Reporting Questionnaire (STAR-Q), Alberta, Canada, Numbers along the horizontal axis indicate the number of months relative to study baseline (indicated as 0 ). The reliability study occurred across months 0, 3, and 6, comparing responses between the first, second, and third administrations of the STAR-Q (STAR-Q1, STAR-Q2, and STAR-Q3). AEE, activity energy expenditure; DLW, doubly labeled water; TEE, total energy expenditure.

3 426 Csizmadi et al. The first STAR-Q (STAR-Q1) was completed 14 days after DLW dosing (day 0). The second and third STAR-Qs (STAR-Q2 and STAR-Q3) were completed approximately 3 and 6 months after day 14, respectively. STAR-Q1 was self-administered at the test facility, and following completion, responses were reviewed for completeness and clarity of the text. STAR-Q2 and STAR-Q3 were also self-administered but were sent to participants and returned by mail. AEE estimation from the STAR-Q. Responses on the STAR-Q were used to estimate AEE STAR-Q,TEE STAR-Q,energy expenditure in specific domains, and daily numbers of hours spent in intensity-specific activities and in sedentary behavior. All self-reported activities were assigned activity codes and metabolic equivalent of task (MET) values from the Compendium of Physical Activities (8, 9). Where total reported time spent in physical activity, sedentary behavior, and sleep was less than 24 hours/day, a MET value of 1.2 was assigned to the unaccounted-for time. Reported frequency, duration, and intensity of activities were combined to obtain a single estimate of energy expenditure in MET-hours/day for each activity. Activity-specific estimates of MET-hours/day were then summed across all activities for each subject to obtain the total average energy cost in MET-hours/day. Those estimates were subsequently applied to our estimations of average past-month TEE STAR-Q and AEE STAR-Q as follows: TEE STAR-Q ¼½ð0:9METs hours of sleep) þðmet-hours/day weight ðkgþþš 1:1 and AEE STAR-Q ¼½MET-hours/day weight ðkgþš ½1:0METs weight ðkgþ total hours of activity/dayš; where TEE STAR-Q was estimated from reported nightly hours of sleep from the STAR-Q multiplied by 0.9 METs (average energy expenditure during sleep (8, 9)) and added to total daily MET-hours during time spent awake multiplied by weight (in kilograms) measured at baseline. An additional 10% was added for energy expenditure attributed to the thermic effect of food. AEE STAR-Q was estimated from total daily MET-hours accumulated during waking time multiplied by weight (in kilograms) measured at baseline, from which energy expenditure attributed to resting metabolism (1.0 METs weight activity hours/day) was subtracted. Finally, estimates of amounts of time spent in each activity domain and at various intensities sedentary ( 1.5 METs), light (> METs), moderate (> METs), and vigorous (>6.0 METs) were derived. We also determined AEE per kilogram of body weight (kcal/kgday) for DLWandSTAR-Q-derivedestimatesinanefforttoaccountfor the known tendency of body mass or lean mass to inflate the association between questionnaire-derived AEE and DLW-derived AEE (10, 11). Validation study design An overview of the 21-day validation study protocol is provided in Figure 1. The DLW protocol was completed over a period of 14 days, with the STAR-Q1 being completed on day 14. Participants were instructed to begin prospectively recording daily activities in the 7-day diary on day 15. The relative timing of the STAR-Q1 and diary administrations was intended to minimize learning effects and altered awareness of behavior that might have occurred had the diaries been completed first. 7-Day activity diary. A 7-day activity diary was employed to compare domain- and intensity-specific AEE and duration estimates, respectively, with those of STAR-Q1. The 7-day diary was adapted from physical activity records described by Conway et al. (12) and was designed to ascertain sleep time as well as all activities engaged in (of 10 minutes duration and those requiring a change in physical effort) and posture while awake. Respondents recorded the time at which each activity began, described the activity, and selected one of 10 listed domains and subdomains (corresponding closely to the STAR-Q), posture (reclining, sitting, standing, walking, or in motion (e.g., swimming, cycling, dancing)), and the perceived intensity of the activity (effort level 1, 2, 3, or 4; definitions were provided). Participants also reported stair-climbing (number of flights up and down). Completed diaries were returned to the test center by mail. Summary estimates of energy expenditure were derived for the 7-day diaries using reduction procedures as described for the STAR-Q. Anthropometric measures and resting metabolic rate. Height and weight were measured at the test facility to the nearest 0.1 cm and 0.1 kg, respectively, on days 0 and 14 of the DLW protocol by a certified exercise physiologist using standard procedures (13). The TBF-310 Tanita Body Composition Analyzer and Scale (Tanita Corporation of America, Inc., Preston, Washington (TheCompetitiveEdge. com)) was used to measure body weight and to estimate percentage of body fat. Body mass index was determined as body weight (kg) divided by height (m) squared. Resting metabolic rate was estimated by means of the Schofield equation, which takes into account age, height, and weight (14). DLW protocol. A 14-day DLW study was used as the primary criterion method against which AEE STAR-Q and TEE STAR-Q estimates were validated (Figure 1). Following an overnight fast, participants provided baseline urine and saliva samples for determination of background isotope levels (day 0). DLW doses were administered orally to provide each subject with 2.5 g 10 atom % of oxygen-18 ( 18 O) per kg of total body water and 0.18 g 99 atom % of deuterium ( 2 H) per kg of estimated total body water (Rotem Inc., Topsfield, Massachusetts). Each participant was asked to abstain from food and fluid intake for 4 hours after dosing. Postdose saliva samples were collected at 3 and 4 hours for the measurement of total body water from deuterium-isotope dilution. The Canadian Diet History Questionnaire (15), a food frequency questionnaire that ascertains past-year dietary intake, was completed on day 0 for food quotient estimation (proxy for respiratory quotient). Second-void urine samples were collected on days 1 and 8, with sampling time recorded on a time sheet and stored in a refrigerator in a 125-mL container until pick-up by study staff within the next day. On day 14, following an overnight fast, participants collected the final urine sample and returned to the test center for dosing with 0.18 g 99 atom % deuterium per kg of total body water, followed by postdose saliva sample collections as described for

4 STAR-Q Reliability and Validity Studies 427 day 0. To measure the decline in isotope enrichment, samples were batch-analyzed in duplicate using the Isoprime Stable Isotope Ratio Mass Spectrometer (Isoprime Ltd., Cheadle Hulme, United Kingdom). A MultiFlow-Bio module for Isoprime (Isoprime Ltd.) equipped with a Gilson 222XL Autosampler (GV Instruments Ltd., Manchester, United Kingdom) was used for daily energy expenditure measurements. Data processing was performed using IonVantage software (2012) for Isoprime (Isoprime Ltd.). Stability tests were performed each day before testing, yielding standard deviations of 0.026% for deuterium and 0.004% for oxygen-18. TEE was calculated according to the method of Racette et al. (16), using a modified Weir equation and an assumed respiratory quotient of In addition, a food quotient (proxy for respiratory quotient) was estimated from the Diet History Questionnaire (15). AEE was subsequently derived from TEE as follows: AEE DLW ¼½TEE DLW 0:90Š f½ðrmr/24þ ð24 hours of sleepþš þ ½0:9ðRMR/24Þ hours of sleepšg; where RMR represents resting metabolic rate. As with the self-reported data, resting metabolic rate was estimated using the Schofield equation (14). The thermic effect of food (10%) was subtracted from TEE, and resting metabolic rate was subtracted after accounting for a 10% reduction in energy expenditure during the self-reported sleep hours from STAR-Q1. Statistical analysis Baseline characteristics were examined for participants who did not return a STAR-Q or had implausibly high STAR- Q-derived AEE levels. Their characteristics were compared with those of the rest of the group. Reliability study. Descriptive statistics were calculated for all derived variables. Agreement across STAR-Q administrations was assessed using intraclass correlation coefficients (ICCs) based on 2-way mixed models without interaction (17), treating subjects as random effects and the activity assessment methods as fixed effects. Concordance correlation coefficients (CCCs) were estimated using variance components of a 2-way mixed-effects model (18). CCCs are practically and asymptotically equivalent to ICCs but do not require analysis-of-variance model assumptions, can be adjusted for potential covariates, and can measure 3-way concordance (e.g., STAR-Q1 vs. STAR-Q2 vs. STAR-Q3). Adjustment was made for baseline age, measured body mass index, and sex, since measurement error can vary with these characteristics (19 21), and also for season (dichotomized: May October, November April), which can influence physical activity behavior (22, 23). Logarithmic transformations were carried out for all variables demonstrating marked departures from normality (indicated in the Results section), and all zero values were converted to 0.1. For assessment of the internal validity of the data, 1,000 bootstrap samples were drawn, providing robust 95% confidence intervals around estimated CCCs. Validity study. STAR-Q-derived TEE STAR-Q and AEE STAR-Q estimates were compared with those from DLW and 7-day diaries using post hoc Wilcoxon signed-rank tests. Relative agreement was assessed using ICCs based on 2-way mixed models without interaction (17) and CCCs using variance components, reflecting accuracy and precision, in a 2-way mixedeffects model with adjustment for age, sex, and season (18). Bland-Altman plots (24) were used to appraise group-level agreement and to identify trends in under- and overestimation of AEE and TEE by the STAR-Qrelative to DLW. The 95% limits of agreement (25) around the mean difference showed with 95% certainty the range in which individual differences would occur for a similar population. Sample size was estimated for 2 mixed-effects analysis-of-variance models, assuming an ICC of at least 0.65 with a 95% confidence interval of ±0.10 (26). We did not experience the possible 20% loss to follow-up or unusable data that we anticipated, so our sample size exceeded our goal of 83 participants. All analyses were conducted with SAS/STAT (version 9.2; SAS Institute, Inc., Cary, North Carolina), PC-SAS (27), and R (version 2.12; R Foundation for Statistical Computing, Vienna, Austria ( org)) (28). Statistical significance tests were 2-sided. RESULTS Participation Between July 2009 and July 2010, 106 participants were recruited; data collection was completed by January One participant was excluded based on illegible STAR-Q responses; 5 did not return STAR-Q2s, and 5 did not return STAR-Q3s. Others were excluded based on a priori criteria for implausible values (AEE >3,600 kcal/day). Final sample sizes were 102 (STAR-Q1), 96 (STAR-Q2), and 97 (STAR- Q3). Estimated CCCs comparing all 3 STAR-Qs for reliability included only those participants with data for all 3 STAR- Qs (n = 91). Nine participants who did not return at least 1 STAR-Q were similar to the remaining participants in terms of age, body mass index, sex, and percentage of body fat. Persons excluded because of implausibly high AEEs (n = 7)were mostly men (86% men, as opposed to 40% in the analyzed group). In the validation study, of 102 participants with valid STAR- Q1 data, 1 did not return a diary, 1 submitted a diary with insufficient detail for activity estimation, and 1 was excluded based on a diary-estimated AEE greater than 3,600 kcal/day. Two DLW measures were excluded because of incomplete equilibration and high baseline urine isotope levels. Thus, 99 and 100 subjects were included in the diary and DLW comparisons, respectively. On average, study participants were middle-aged, Caucasian, employed, and well-educated (Table 1). Although participants represented a range of physical activity levels using Institute of Medicine (29) criteria and DLW estimates, TEE DLW showed that the majority were active (physical activity level 1.60 <1.90) or very active ( physical activity level 1.90). The median self-reported time required for STAR-Q1 completion was 34 minutes. The proportions of STAR-Qs completed during the warmer months (May October) were 45.7% (STAR-Q1), 53.0% (STAR-Q2), and 43.0% (STAR-Q3). Estimated energy expenditure Numbers of hours per day of self-reported activity (Table 2 and Web Table 1) were reasonable, on average. However,

5 428 Csizmadi et al. Table 1. Characteristics of Participants in the STAR-Q Validation Study Who Completed STAR-Q1, a Alberta, Canada, median physical activity levels derived from the STAR-Qs and diary were relatively high (approximately 2.0, or very active based on Institute of Medicine criteria, in comparison with the DLW-derived average physical activity level of 1.8 ( active using Institute of Medicine criteria)). TEE DLW estimation using a respiratory quotient of 0.85 or a Diet History Questionnaire-estimated food quotient were similar, with median TEEs differing by less than 20 kcal/day (data not shown); hence, we assumed a respiratory quotient of 0.85 in all subsequent TEE DLW analyses. Reliability Males (n = 41) b Females (n = 61) b Mean (SD) Range No. % Mean (SD) Range No. % Age, years 50.6 (6.9) (8.6) Body mass index c 25.9 (3.1) (2.9) % body fat (BIA) 21.8 (6.2) (6.5) Predicted RMR, kcal/day d 1,787 (132) 1,533 2,035 1,380 (84) 1,184 1,705 Physical activity level e 1.8 (0.4) (0.4) Education Less than bachelor s degree Bachelor s degree or higher Unknown/missing data Ethnicity Caucasian Asian Other Unknown/missing data Paid or volunteer work STAR-Q1 (n = 102) STAR-Q2 (n = 96) STAR-Q3 (n = 97) Physical activity level (n = 101) e < < < Abbreviations: BIA, bioelectrical impedance analysis; DLW, doubly labeled water; RMR, resting metabolic rate; STAR-Q, Sedentary Time and Activity Reporting Questionnaire. a The first STAR-Q (STAR-Q1) was completed 14 days after DLW dosing (day 0). b Sample size at STAR-Q1. c Weight (kg)/height (m) 2. d Estimated using the Schofield equation (14). e Physical activity level = total energy expenditure derived from DLW divided by predicted RMR (n = 101 based on available DLW results). apart). When comparing STAR-Q1 with STAR-Q2, goodto-excellent ICCs (>0.70) were observed for TEE, AEE, sleep, overall occupational activity, occupational sedentary behavior, and television-watching. Remaining activities were associated with fair-to-moderate ICCs. Generally, adjusted CCCs (CCC adj ) were lower than ICCs, but associations remained moderate to good (approximately ) for TEE, AEE, sleep, overall occupational activity, occupational sedentary behavior, and television-watching. Notable differences between unadjusted and adjusted CCCs were found for TEE and AEE; covariate adjustment otherwise had little impact. ICCs and CCCs for kcal/day and MET-hours/day are presented in Table 3. Generally the ICCs for STAR-Q1 versus STAR-Q2 (completed 3 months apart) were higher than ICCs for STAR-Q1 versus STAR-Q3 (completed 6 months Validity STAR-Q1 versus 7-day activity diary. On average, the diary numbers of hours per day closely approximated a

6 STAR-Q Reliability and Validity Studies 429 Table 2. Hours of Activity and Energy Expenditure (Median and Interquartile Range) Derived From STAR-Q Responses, a 7-Day Diary, and Doubly Labeled Water in the STAR-Q Validation Study, Alberta, Canada, hour period, while the STAR-Q underestimated hours per day by just over 2 hours (Table 2).Reportingonsedentary time probably contributed to this difference (Figure 2). Additionally, total occupational activity, occupational light activity, total light leisure activity, and sedentary behavior during light leisure activity accounted for less time using the diary (Figure 3). Results of the STAR-Q1-versus-diary comparison are presented in Table 4. For TEE and AEE, negligible median differences were observed at the group level; however, larger interquartile ranges indicated high between-subject variability. Agreement and consistency for TEE, AEE, and AEE/kg body weight were moderate to good depending on the test statistic; Spearman correlations were relatively high (ρ s = 0.74, ρ s = 0.61, and ρ s = 0.45, respectively), whereas CCC adj values were lower (0.51, 0.45, and 0.41). Results for overall occupational activity, occupational sitting, and occupational sedentary time were consistently high, with CCC adj values of approximately 0.70, and CCC adj was 0.64 for overall strenuous activity (hours/day). In contrast, consistency between STAR-Q1- and diary-derived estimates for light activities were poor (CCC adj < 0.30) overall and within occupation and exercise domains. Adjusting CCCs for covariates had the strongest effect for TEE and AEE but an otherwise negligible impact. STAR-Q1 versus DLW. On average, the STAR-Q1 significantly overestimated energy expenditure relative to DLW (P < 0.001; Wilcoxon signed-rank test), with a median difference (STAR-Q1 minus DLW) of 367 kcal/day (interquartile range, 800 kcal/day) for TEE and a median difference of 293 kcal/day (interquartile range, 769 kcal/day) for AEE. Bland-Altman plots (Figure 4) revealed significant inverse trends in which overestimation of energy expenditure occurred for persons with the lowest TEE DLW (Figure 4A) or AEE DLW (Figure 4B), while for persons with the highest TEE DLW or AEE DLW, energy expenditure was underestimated by the STAR-Q. This negative trend was most pronounced for AEE/kg body weight (ρ s = 0.61, P < 0.001; Figure 4C). Assessment Method a STAR-Q1 (n = 102) STAR-Q2 (n = 96) STAR-Q3 (n = 97) 7-Day Diary (n = 103) DLW (n = 103) Total hours/day, including sleep 21.2 (4.5) 21.9 (4.7) 21.5 (5.4) 24.0 (0.2) b Total activity, MET-hours/day c 30.0 (12.7) 31.5 (15.0) 31.7 (14.2) 33.6 (7.3) b AEE, kcal/day c 1,202 (628) 1,242 (697) 1,281 (643) 1,236 (575) 1,016 (661) b AEE, kcal/kgday c 16.9 (7.2) 18.0 (10.5) 18.6 (10.1) 18.0 (7.4) 13.7 (7.1) b TEE, kcal/day 3,238 (1,273) 3,235 (1,078) 3,261 (1,105) 3,231 (1,104) 2,808 (1,038) b Physical activity level d 2.0 (0.5) 2.0 (0.5) 2.1 (0.6) 2.1 (0.4) 1.8 (0.3) b Abbreviations: AEE, activity energy expenditure; DLW, doubly labeled water; MET, metabolic equivalent of task; RMR, resting metabolic rate; STAR-Q, Sedentary Time and Activity Reporting Questionnaire; TEE, total energy expenditure. a STAR-Q1, STAR-Q2, and STAR-Q3 represent the first, second, and third administrations of the STAR-Q. b The median difference between STAR-Q1 and the reference method (i.e., diary or DLW) was significantly different from zero (P < 0.001, 2-sided test) in a Wilcoxon signed-rank test. c Excludes sleep. d TEE/predicted RMR. Compared with DLW estimates, TEE STAR-Q1 was moderately correlated (ρ s = 0.53, P < 0.001), as was AEE STAR-Q1 (ρ s = 0.40, P < 0.001), while the correlation for body-weightadjusted AEE STAR-Q1 was slightly lower (ρ s = 0.34, P = 0.001). ICCs using log-transformed variables were also fair to moderate, with the ICC for TEE being 0.41 (95% CI: 0.23, 0.56), that for AEE being 0.20 (95% CI: 0.01, 0.38), and that for AEE/kg body weight being 0.13 (95% CI: 0.07, 0.32). CCCs with log-transformed TEE, AEE, and AEE/kg body weight and adjusted for covariates were 0.24 (95% CI: 0.07, 0.36), 0.21 (95% CI: 0.11, 0.32), and 0.19 (95% CI: 0.08, 0.27), respectively. DISCUSSION The STAR-Q has reasonable reliability for estimating pastmonth AEE and, on average, fair validity for ranking individuals by AEE based on Spearman correlation comparisons with DLW. However, DLW comparisons also revealed proportional measurement error, with trends of AEE overreporting by the least active persons and AEE underreporting by the most active persons; hence, validity estimates based on adjusted CCCs were lower. Compared with prospectively completed 7-day activity diaries, the STAR-Q performed particularly well in estimating energy expenditure related to overall occupational activity, occupational sedentary and sitting time, and strenuous activity, but agreement was weaker for light activity. To our knowledge, only 1 other questionnaire was designed with an objective similar to our own. Besson et al. (30) reported results for validation of the Recent Physical Activity Questionnaire against accelerometry and DLW. Similar to the STAR-Q, this questionnaire ascertains past-month activity and demonstrates substantial validity for ranking individuals on vigorous-intensity activity. Relative to the STAR- Q, Besson et al. described somewhat better validity for ranking individuals on TEE (ρ s =0.67 vs. ρ s = 0.53) and AEE/kg body weight (ρ s =0.39vs.ρ s = 0.34) but lower validity for ranking them on sedentary time (ρ s =0.27 vs. ρ s = 0.40)

7 430 Csizmadi et al. Table 3. Intraclass and Concordance Correlation Coefficients (MET-hours/day) for Reliability Assessments in the STAR-Q Validation Study, Alberta, Canada, a STAR-Q1 vs. STAR-Q2 (n = 95) STAR-Q1 vs. STAR-Q3 (n = 96) STAR-Q1 vs. STAR-Q2 vs. STAR-Q3 (n = 91) ICC 95% CI ICC 95% CI Unadjusted CCC b 95% CI c Adjusted CCC d 95% CI c TEE, kcal/day e , , , , 0.73 AEE, kcal/day e , , , , 0.69 Sleeping f , , , , 0.80 Stair-climbing, flights/day g , , , , 0.58 Active sitting h , , , , 0.58 Overall activity i Sedentary , , , , 0.59 Light e , , , , 0.64 Moderate e , , , , 0.55 Strenuous e , , , , 0.69 Exercise e,i , , , , 0.70 Light e , , , , 0.59 Moderate e , , , , 0.43 Strenuous e , , , , 0.66 Occupational activity e,i , , , , 0.81 Sitting j , , , , 0.76 Sedentary , , , , 0.77 Light e , , , , 0.67 Moderate e , , , , 0.59 Light leisure activity e , , , , 0.64 Television-viewing , , , , 0.77 Computer use e , , , , 0.71 Reading , , , , 0.52 Abbreviations: AEE, activity energy expenditure; CCC, concordance correlation coefficient; CI, confidence interval; DLW, doubly labeled water; ICC, intraclass correlation coefficient; MET, metabolic equivalent of task; STAR-Q, Sedentary Time and Activity Reporting Questionnaire; TEE, total energy expenditure. a STAR-Q1, STAR-Q2, and STAR-Q3 represent the first, second, and third administrations of the STAR-Q. b Not adjusted for any variable. c 95% CIs were estimated by means of bootstrapping for both adjusted and unadjusted CCCs. d Adjusted for sex, age, and body mass index at baseline, as well as season of STAR-Q completion. e Log-transformed data. f Excludes napping. g Results are shown for climbing up stairs (self-reported number of flights per day); results for going down stairs were very similar. h Represents the sum of all activities performed while sitting and expending energy at > METs. i Intensity categories were defined as follows: sedentary (excluding sleep), 1.5 METs; light, >1.5 3 METs; moderate, >3 6 METs; strenuous, >6 METs. Strenuous occupational activity was not analyzable because of insufficient data. j MET level 2.5. (30). In contrast to the STAR-Q, the Recent Physical Activity Questionnaire did not ascertain the number of hours slept per night, the duration of the workweek, or details on household activities. When comparing reported behaviors with 7-day diaries, the STAR-Q performed well, on average, despite nonoverlapping reference periods. True differences in energy expenditure might have occurred between the two assessment periods, perhaps accounting for some discrepancy between the STAR-Q and the diary. However, areas of substantial agreement between the STAR-Q and diary are informative for guiding the optimal use of the STAR-Q in future studies (e.g., occupational sedentary behavior, strenuous activity). These stronger results were not unexpected, since occupational and strenuous activities have been reliably reported in other studies (30, 31). On the other hand, activities with poor agreement highlight problematic domains and intensities for example, lightintensity activity that may require alternate assessment methods combining objective and self-reported data. The strengths of our study relative to past DLW studies (3) include a large sample size, appropriate use of DLW as a criterion method, and the use of multiple test statistics, including

8 STAR-Q Reliability and Validity Studies Active and Inactive Time, hours/day the novel use of the CCC. The CCC is a rigorous estimate of agreement that allows for the adjustment of covariates that might contribute to between-subject variability. Adjusted coefficients may be lower than unadjusted estimates; however, the remaining explained variance more clearly reflects the Intensity of Activity STAR-Q1 STAR-Q2 STAR-Q3 7-Day Diary Sleep Sedentary Light Moderate Vigorous Figure 2. Self-reported mean duration (hours/day) of active and inactive behavior by intensity in the Sedentary Time and Activity Reporting Questionnaire (STAR-Q) validation study, Alberta, Canada, STAR-Q1, STAR-Q2, and STAR-Q3 represent the first, second, and third administrations of the STAR-Q. Active and Inactive Time, hours/day measure of interest (18). For example, a CCC adj of 0.21 for AEE indicates that STAR-Q-derived AEE explained 21% of DLW-derived AEE. To our knowledge, only 2 other DLW validation studies have conducted analyses that might be considered analogous to this. Masse et al. (32) Vigorous Moderate Light Sedentary STAR-Q1 STAR-Q2 STAR-Q3 7-Day Diary STAR-Q1 STAR-Q2 STAR-Q3 7-Day Diary STAR-Q1 STAR-Q2 STAR-Q3 7-Day Diary Occupation Light Leisure Exercise Activity Domain Figure 3. Self-reported mean duration (hours/day) of active and inactive behavior by domain in the Sedentary Time and Activity Reporting Questionnaire (STAR-Q) validation study, Alberta, Canada, STAR-Q1, STAR-Q2, and STAR-Q3 represent the first, second, and third administrations of the STAR-Q.

9 432 Csizmadi et al. Table 4. Agreement and Correlations (MET-hours/day) Between STAR-Q1 a and 7-Day Diary Activity, by Intensity b and Domain, in a Validity Assessment of the STAR-Q (n = 99), Alberta, Canada, Median Difference, STARQ1 Diary (IQR) Spearman Correlation ICC 95% CI Coefficient (r s ) c Unadjusted Adjusted d 95% CIe CCC CCC f 95% CIe TEE, kcal/day g 98.2 (615.2) , , , 0.63 AEE, kcal/day g,h 81.7 (540.5) , , , 0.63 AEE, kcal/kgday g,h 1.3 (7.8) , , , 0.59 Sleeping i 0.8 (0.8) , , , 0.48 Stair-climbing, flights/day j 0.1 (8.5) , , , 0.52 Active sitting g,k 0.0 (0.4) 0.30 l , , , 0.49 Overall activity b Sedentary 3.9 (4.9) , , , 0.39 Light g 0.6 (7.3) 0.26 m , , , 0.41 Moderate g 0.2 (5.0) , , , 0.61 Strenuous g 0.0 (3.3) , , , 0.76 Exercise, sports, and 0.2 (4.2) , , , 0.58 leisure activity b,g Light g 0.0 (0.3) 0.21 n , , , 0.49 Moderate g 0.0 (0.8) , , , 0.54 Strenuous g 0.0 (3.4) , , , 0.63 Occupational activity b,g 1.4 (3.6) , , , 0.84 Sitting o 0.3 (2.2) , , , 0.81 Sedentary 0.1 (1.8) , , , 0.85 Light g 0.6 (3.0) 0.24 p , , , 0.31 Moderate g 0.0 (0.3) 0.30 l , , , 0.64 Abbreviations: AEE, activity energy expenditure; CCC, concordance correlation coefficient; CI, confidence interval; ICC, intraclass correlation coefficient; IQR, interquartile range; MET, metabolic equivalents of task; STAR-Q, Sedentary Time and Activity Reporting Questionnaire; TEE, total energy expenditure. a The first STAR-Q (STAR-Q1) was completed 14 days after doubly labeled water dosing (day 0). b Intensity categories were defined as follows: sedentary (excluding sleep), 1.5 METs; light, >1.5 3 METs; moderate, >3 6 METs; strenuous, >6 METs. Strenuous occupational activity was not analyzable because of insufficient data. c P < (2-sided t test) for Spearman correlations unless otherwise indicated. d Not adjusted for any variable. e 95% CIs were estimated by means of bootstrapping for both adjusted and unadjusted CCCs. f Adjusted for sex, age, and body mass index at baseline, as well as season of STAR-Q completion. g Data were log-transformed for ICC and CCC analyses. h Excludes sleep. i Excludes napping. j Results are shown for climbing up stairs (self-reported number of flights per day); results for going down stairs were very similar. k Represents the sum of all activities performed while sitting and expending energy at > METs. l P = m P = n P = o MET level 2.5. p P = used regression analyses to show that 2 different types of questionnaires (checklist and global), when administered to 260 women, explained 6.4% and 5.0% of DLW-derived AEE, respectively. Neuhouser et al. (20) reported that the Women s Health Initiative Personal Habits Questionnaire, the Arizona Activity Frequency Questionnaire, and the 7-Day Physical Activity Recall, administered to 450 women, explained 7.6%, 4.8%, and 3.4% of DLW-derived AEE, respectively. While our body mass index-adjusted analysis controlled for variance due to misreporting that may be associated with body mass index, it may not have sufficiently accounted for the known tendency for body mass or lean body mass to inflate the association between questionnaire- and DLWderived AEE (33). We therefore also divided AEE by kilograms of body weight, but we observed only a negligible decrease in the CCC adj (from 0.21 to 0.19), whereas larger decreases occurred with Spearman correlations (from

10 STAR-Q Reliability and Validity Studies 433 A) B) C) STAR-Q1 TEE DLW TEE, kcal/day STAR-Q1 AEE DLW AEE, kcal/day STAR-Q1 AEE DLW AEE, kcal/kg day 3,000 2,000 1, ,000 3,000 2,000 1, , ,000 3,000 4,000 5,000 DLW TEE, kcal/day ,000 1,500 2,000 2,500 3,000 DLW AEE, kcal/day DLW AEE, kcal/kg day Figure 4. Bland-Altman plots illustrating agreement with doubly labeled water (DLW) in the Sedentary Time and Activity Reporting Questionnaire (STAR-Q) validation study, Alberta, Canada, Part A (validity comparison for total energy expenditure (TEE) (kcal/day); n = 100) depicts the relationship between the difference between measures and the DLW measure (Spearman correlation coefficient (ρ s )= 0.25, P = 0.012; 2-sided t test); the mean difference of kcal/day and the 95% limits of agreement (LOA) ( 1,116.20, 2,045.16) are depicted by 3 horizontal lines. Part B (validity comparison for activity energy expenditure (AEE) (kcal/day); n = 100) depicts the relationship between the difference between measures and the DLW measure (ρ s = 0.50, P < 0.001; 2-sided t test); the mean difference of kcal/day and the 95% LOA ( , 1,488.60) are depicted by 3 horizontal lines. Part C (validity comparison for AEE (kcal/ kgday); n = 100) depicts the relationship between the difference between measures and the DLW measure (ρ s = 0.61, P < 0.001; 2-sided t test); the mean difference of 4.15 kcal/kgday and the 95% LOA ( 12.86, 21.17) are depicted by 3 horizontal lines to 0.34) and ICCs (from 0.20 to 0.13). The majority of DLW validation studies published to date have not accounted for these effects of body mass (3). Our results from the reliability study indicate that at the group level, past-month assessments of AEE can provide reasonable estimates of usual AEE. Although we adjusted for the effects of season in our CCC analyses, its impact was found to be negligible; yet we do not know whether true AEE was stable across assessment periods. Fluctuations in physical activity behavior have been associated with variations in temperature and weather conditions (22, 23, 34). It is possible that persons who are active in one season may remain active year-round by participating in season-appropriate activities. Limitations of our study need to be acknowledged. First, we were unable to measure resting metabolic rate directly and instead used a standard equation to estimate it. While we accounted for body weight, age, and sex, other factors that predict resting metabolic rate (e.g., body composition) could have affected the validity of DLW-derived AEE (since AEE DLW =[TEE DLW 0.90] {[(RMR/24) (24 hours of sleep)] + [0.9(RMR/24) hours of sleep]}). Second, we used 7-day diaries as a criterion measure for validating domain- and intensity-specific estimates of AEE and activity, but we acknowledge that the diary method is an imperfect gold standard. Third, these results can only be generalized to middle-aged Canadian adults who are generally educated, employed, and physically active. Fourth, since ICCs are lower when between-subject variability is low relative to measurement error (whether random or systematic) (35), it is possible that the validity and reliability of the STAR-Q were underestimated for some domains and activities with restricted variability (e.g., sleeping, sedentary behavior) (Table 4). Fifth, our analyses did not quantify subject-related sources of measurement error, as has been reported by others (19, 20), since we felt it was beyond the scope of the present study. In summary, using rigorous methods, we showed that the STAR-Q on average had fair validity for ranking individuals by overall AEE and was reliable over a period of several months. Our study also highlighted systematic measurement error that was proportional to AEE, which should not be ignored in the design and analysis of future etiological studies that use the STAR-Q. The comprehensive design of the STAR-Q allows for the assessment of all domains of activity at various intensities, including sedentary behavior, making it a potentially useful tool for investigating emerging hypotheses related to AEE and health. It is widely acknowledged that traditional self-reported measures will continue to be relied upon in epidemiologic studies, ideally combined with objective measures of activity and energy expenditure. The STAR-Q is designed to meet the needs of these evolving, hybrid research methods. ACKNOWLEDGMENTS Author affiliations: Department of Cancer Epidemiology and Prevention Research, CancerControl Alberta, Alberta Health Services, Calgary, Alberta, Canada (Ilona Csizmadi,

11 434 Csizmadi et al. Heather K. Neilson, Karen A. Kopciuk, Farah Khandwala, Andrew Liu, Christine M. Friedenreich); Department of Oncology, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada (Ilona Csizmadi, Karen A. Kopciuk, Christine M. Friedenreich); Department of Community Health Sciences, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada (Ilona Csizmadi, Christine M. Friedenreich, Heather E. Bryant); Department of Mathematics and Statistics, Faculty of Science, University of Calgary, Calgary, Alberta, Canada (Karen A. Kopciuk); School of Public Health, University of Alberta, Edmonton, Alberta, Canada (Yutaka Yasui); Institut de recherches cliniques de Montréal and Département de nutrition, Faculté de médecine, Université de Montréal, Montréal, Québec, Canada (Rémi Rabasa-Lhoret); Division of Cancer Control, Canadian Partnership Against Cancer, Toronto, Ontario, Canada (Heather E. Bryant); Department of Medicine, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada (David C. W. Lau); Department of Biochemistry and Molecular Biology, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada (David C. W. Lau); Department of Cancer Measurement, Outcomes, Research and Evaluation, CancerControl Alberta, Alberta Health Services, Edmonton, Alberta, Canada (Paula J. Robson); and Department of Agricultural, Food and Nutritional Sciences, Faculty of Agricultural, Life and Environmental Sciences, University of Alberta, Edmonton, Alberta, Canada (Paula J. Robson). This work was supported by the Canadian Institutes of Health Research (grant MOP-86632) and the Alberta Cancer Research Institute (grant 24265). We acknowledge Ruth Gbewonyo (study coordinator) and Jason Ng (research assistant) of CancerControl Alberta (Alberta Health Services) for their contributions in study coordination, participant recruitment, and data collection; Diane Mignault (Institut de recherches cliniques de Montréal) for performing the stable-isotope analysis; Farit Vakhetov (CancerControl Alberta) for designing and managing the interview, questionnaire, and biosample databases; and the staff of the Alberta Tomorrow Project (CancerControl Alberta) for their assistance with participant recruitment. Preliminary reliability study results were presented by Dr. Ilona Csizmadi at the 8th International Conference on Diet and Activity Methods, Rome, Italy, May 14 17, Conflict of interest: none declared. REFERENCES 1. Warren JM, Ekelund U, Besson H, et al. Assessment of physical activity a review of methodologies with reference to epidemiological research: a report of the exercise physiology section of the European Association of Cardiovascular Prevention and Rehabilitation. Eur J Cardiovasc Prev Rehabil. 2010;17(2): Atkin AJ, Gorely T, Clemes SA, et al. Methods of measurement in epidemiology: sedentary behaviour. Int J Epidemiol. 2012; 41(5): Neilson HK, Robson PJ, Friedenreich CM, et al. Estimating activity energy expenditure: how valid are physical activity questionnaires? Am J Clin Nutr. 2008;87(2): Troiano RP, Pettee Gabriel KK, Welk GJ, et al. Reported physical activity and sedentary behavior: why do you ask? J Phys Act Health. 2012;9(suppl 1):S68 S Sesso HD. Invited commentary: a challenge for physical activity epidemiology. Am J Epidemiol. 2007;165(12): Neilson HK, Ullman R, Robson PJ, et al. Cognitive testing of the STAR-Q: insights in activity and sedentary time reporting. J Phys Act Health. 2013;10(3): Bryant H, Robson PJ, Ullman R, et al. Population-based cohort development in Alberta, Canada: a feasibility study. Chronic Dis Can. 2006;27(2): Ainsworth BE, Haskell WL, Leon AS, et al. Compendium of Physical Activities: classification of energy costs of human physical activities. Med Sci Sports Exerc. 1993;25(1): Ainsworth BE, Haskell WL, Whitt MC, et al. Compendium of Physical Activities: an update of activity codes and MET intensities. Med Sci Sports Exerc. 2000;32(9 suppl): S498 S Westerterp KR. Physical activity as determinant of daily energy expenditure. Physiol Behav. 2008;93(4-5): Ainsworth BE, Haskell WL, Herrmann SD, et al Compendium of Physical Activities: a second update of codes and MET values. Med Sci Sports Exerc. 2011;43(8): Conway JM, Seale JL, Jacobs DR Jr, et al. Comparison of energy expenditure estimates from doubly labeled water, a physical activity questionnaire, and physical activity records. Am J Clin Nutr. 2002;75(3): Douketis JD. Body weight classification. CMAJ. 2005;172(10): Schofield WN. Predicting basal metabolic rate, new standards and review of previous work. Hum Nutr Clin Nutr. 1985; 39(suppl 1): Csizmadi I, Kahle L, Ullman R, et al. Adaptation and evaluation of the National Cancer Institute s Diet History Questionnaire and nutrient database for Canadian populations. Public Health Nutr. 2007;10(1): Racette SB, Schoeller DA, Luke AH, et al. Relative dilution spaces of 2 H- and 18 O-labeled water in humans. Am J Physiol. 1994;267(4):E585 E McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996;1(1): Carrasco JL, Jover L. Estimating the generalized concordance correlation coefficient through variance components. Biometrics. 2003;59(4): Ferrari P, Friedenreich C, Matthews CE. The role of measurement error in estimating levels of physical activity. Am J Epidemiol. 2007;166(7): Neuhouser ML, Di C, Tinker LF, et al. Physical activity assessment: biomarkers and self-report of activity-related energy expenditure in the WHI. Am J Epidemiol. 2013;177(6): Tooze JA, Troiano RP, Carroll RJ, et al. A measurement error model for physical activity level as measured by a questionnaire with application to the NHANES questionnaire. Am J Epidemiol. 2013;177(11): Newman MA, Pettee KK, Storti KL, et al. Monthly variation in physical activity levels in postmenopausal women. Med Sci Sports Exerc. 2009;41(2): Matthews CE, Freedson PS, Hebert JR, et al. Seasonal variation in household, occupational, and leisure time physical activity: longitudinal analyses from the Seasonal Variation of Blood Cholesterol Study. Am J Epidemiol. 2001;153(2):