Reliability and Validity of a Brief Tool to Measure Children s Physical Activity

Size: px

Start display at page:

Download "Reliability and Validity of a Brief Tool to Measure Children s Physical Activity"

Barrie Cole
6 years ago
Views:

1 Journal of Physical Activity and Health, 2006, 3, Human Kinetics, Inc. Reliability and Validity of a Brief Tool to Measure Children s Physical Activity Shujun Gao, Lisa Harnack, Kathryn Schmitz, Janet Fulton, Leslie Lytle, Pamela Van Coevering, and David R. Jacobs, Jr. Background: We assessed the validity and reliability of a modified Godin-Leisure-Time Exercise Questionnaire in youth in grades 6 through 8. Methods: The questionnaire was completed by 250 children twice at a 1 wk interval to assess reliability. After the second questionnaire administration the children wore an accelerometer for 7 d (criterion measure). Results: Pearson correlations between the first and second reports of frequency of participation in strenuous and moderate physical activity were 0.68 and 0.51, respectively. Self-reported participation in strenuous activity was weakly correlated with strenuous activity as measured by accelerometer (r = 0.23, P = 0.01). A weak non-significant correlation was found between reported versus measured engagement in moderate activity (r = 0.13, P = 0.07). Conclusion: Findings suggest the questionnaire evaluated in this study may be of very limited use for assessing childrenʼs physical activity. Key Words: physical activities, consistency, accuracy, adolescents, Godin questionnaire Although surveillance of physical activity level is essential in monitoring progress toward national objectives for this health behavior, brief and valid assessment tools as required for use in surveillance systems for youth are lacking. Among a number of physical activity questionnaires developed for use in children, just two, the Godin Leisure-Time Exercise Questionnaire and the 1999 Youth Risk Behavior Survey (YRBS) question are both brief and designed for self-administration. One study 1 evaluated the Godin questionnaire in youth and found modest correlations between the questionnaireʼs activity score and two other self-reported instruments in 5th, 8th, and 11th grade students (r = 0.32 to 0.39). The YRBS question has not been evaluated. 2 To build on this previous work, we modified the Godin questionnaire and evaluated its reliability and validity in children in grades 6 through 8. In addition, we evaluated the validity of the 1999 YRBS question. Gao, Harnack, Lytle, Van Coevering, and Jacobs are with the Division of Epidemiology and Community Health, School of Public Health, University of Minnesota, Minneapolis, MN Schmitz is with the Division of Epidemiology, University of Pennsylvania School of Medicine, Philadelphia, PA Fulton is with the Centers for Disease Control and Prevention, Division of Nutrition and Physical Activity, National Center for Chronic Disease Prevention and Health Promotion, Atlanta, GA

2 416 Gao et al. Methods Modified Godin Questionnaire and the YRBS Physical Activity Question The Godin questionnaire, which was originally developed to assess adultsʼ participation in leisure time exercise, is composed of three questions. The first question asks how many times per week on average the participant engages in strenuous exercise for more than 15 min during free time. The second and third questions ask about engagement in moderate and mild exercises. For each activity level, activities and sports are provided as examples. To improve the relevance of the questionnaire for use with middle school-age children, we modified the instrument so that the sample exercises and sports provided for each category better matched the common activities of youth this age. 3 Graphic depictions of some activities were inserted. Because physical activity behaviors in youth likely differ by time of year, an additional change included segregating the report of activities by summer versus school year (i.e., the summer vacation period vs. the rest of the year). The 1999 YRBS included a single question to assess participation in strenuous physical activity in a past week: On how many of the past 7 days did you exercise or participate in physical activity for at least 20 minutes that made you sweat and breathe hard, such as basketball, running, swimming laps, fast bicycling, fast dancing, or similar aerobic activities? Reliability and Validity Study Children in grades 6 through 8 were recruited to participate. To assess test-retest reliability, the modified Godin questionnaire was administered twice, with the second administration occurring 1 wk following the initial administration. Concurrent validity of the modified Godin questionnaire and YRBS question were assessed by asking participants to wear an accelerometer (Manufacturing Technologies, Inc., Fort Walton Beach, FL) for seven consecutive days following the second administration of the modified Godin questionnaire. The accelerometer measurement was collected after administration of the questionnaire due to concerns that the act of wearing the accelerometer could make children more aware of their physical activity patterns, thereby improve self-reporting of physical activity. A total of 597 students in an ethnically diverse urban middle school in Minneapolis, MN, were invited to participate. Students were instructed to read through the materials and return signed parental consent and child assent forms. Of the 597 students invited to participate, 316 consented and 286 completed the first questionnaire administration (48% participation rate). Study procedures were approved by the University of Minnesota, the Minneapolis Public Schools, and the Centers for Disease Control and Prevention institutional review boards. Study measurements were conducted in the school during homeroom period (the first period of the day) between January and May Accelerometers were placed on the participants immediately after the second questionnaire administration and were retrieved 1 wk later. They were set to record counts in 1-min epochs. Participants were instructed to wear the accelerometer on a belt around their

3 Measuring Children s PA: Reliability and Validity of a Brief Tool 417 midline above the right hip. They were asked to wear the device during waking hours, except when they participated in activities in which they got completely wet (i.e., showering or swimming). Efforts to encourage compliance included verbal instructions, practical demonstration, and reminder visits to the homeroom (two per homeroom). The accelerometer was selected as the criterion measure because it has been validated in laboratory and field settings with children and has been found to be both reliable (intraclass correlation coefficients = 0.81 to 0.84) 4 when worn for 6 d and valid (R 2 = 0.84 for mixed linear regression of accelerometer vs. metabolic equivalents [MET] based on oxygen consumption measured using a Cosmed K4b2 portable metabolic unit). 5 The counts per minute from the device have also been validated against heart rate monitoring in children. 4 When using the accelerometers with children, 7 d of wear are recommended to obtain a valid measure of usual physical activity level. 6 Analyses were conducted using SAS version 8.2 (SAS Institute, Inc., Cary, NC). Reliability analyses were restricted to participants who provided complete responses to all of the physical activity questions for both administrations and had complete information for the following measurements: age, grade level, sex, race, height, and weight. Of the 286 participants who completed one or more study measures, 250 remained after the application of these criteria. To ensure our criterion measure of physical activity was optimal, we further restricted validity analyses to those with 7 d of reliable accelerometer data. 6 Hence, the sample for these analyses was smaller (n = 114). To identify unreliable days, we applied a three-hour rule, whereby a day of data was deemed unreliable if it included cumulative continuous zero count readings for three or more hours during waking hours (conservatively defined as 8:00 AM to 9:00 PM on weekdays and noon to 9:00 PM on weekends). 7 Wilcoxon signed rank tests, Pearson Product-Moment Correlation (PPM), percent agreements, and kappa coefficients were calculated to examine reliability of the modified Godin questionnaire. For validity analyses, PPMs were calculated between the questionnaire and accelerometer measures of frequency of participation in strenuous and moderate physical activities. To facilitate these comparisons, estimates of the number of days per week of participation in strenuous and moderate physical activities were calculated from the accelerometer data using two approaches. The first approach involved analyzing accelerometer data to identify the number of days in which a student participated in strenuous and moderate activity for more than 15 min and comparing this with the questionnaire response on the number of days that they took part in strenuous and moderate exercise and sports activities for more than 15 min in an average week. To estimate the days per week of strenuous activity, we tabulated the number of days with more than 15 min of vector magnitude readings of more than 5200 counts/min. To estimate the days per week of moderate activity, we calculated the number of days with more than 15 min of vector magnitude readings from 3000 to 5200 counts/min. The second approach focused on the 1999 YRBS question that asks the participants to report the number of days during the past 7 d that they participated in a strenuous physical activity for 20 min or longer. The number of days with 20 or more minutes of vector magnitude readings of more than 5200 per week was tabulated.

4 418 Gao et al. Table 1 Demographic Characteristics of Participants Included in Reliability (n = 250) and Validity (n = 114) Analyses Reliability sample Validity sample % (n) % (n) P-value b Age (93) 38.6 (44) (88) 38.6 (44) (69) 22.8 (26) 0.28 Sex Female 58.8 (147) 59.6 (68) Male 41.2 (103) 40.4 (46) 0.80 Race White American 59.6 (149) 53.5 (61) African American 16.8 (42) 19.3 (22) Other 23.6 (59) 27.2 (31) 0.20 Grade 6th 26.0 (65) 32.5 (37) 7th 36.0 (90) 32.5 (37) 8th 38.0 (95) 35.0 (40) 0.10 BMI a Normal weight 74.4 (186) 67.5 (77) At risk for overweight/ overweight 25.6 (64) 32.5 (37) 0.02 a BMI (body-mass index, kg/m 2 ) categories were defined as follows based on growth chart 2000: normal weight: < 85th percentile, at risk for overweight: 85th ~ 94th percentile, or overweight: 95th percentile according to age and sex. b Chi-square test of the difference in the two samples Results The demographic characteristics of participants are shown in Table 1. Mean estimates of participation in strenuous, moderate, and mild exercise activities from the first and second administration of the modified Godin questionnaire were compared and found to be similar in most cases (Table 2). Kappa coefficients for the exact agreement between responses to the activity questions in the first and second administration of the questionnaire ranged from 0.36 to The PPMs between physical activity estimates from the first and second administration of the questionnaire ranged from 0.48 to The PPMs in all subgroups by sex, race, grade level, and body mass were consistent in the direction and magnitude of association with the whole group in most cases (data not shown). The average number of days per week of participation in strenuous and moderate physical activities as reported during the first administration of the modified Godin questionnaire and as determined from 7 d of accelerometer data are shown in Table 3. For the strenuous activity category, the average number of days per week reported by the participants was 3.61, whereas the average number based on

5 Measuring Children s PA: Reliability and Validity of a Brief Tool 419 Table 2 Reliability of the Modified Godin Questionnaire to Assess Physical Activity (N = 250) S1 a S2 b P- value d Agree e PPM c Mean (SD) Mean (SD) % Kappa r P-value School year Strenuous 3.7 (2.0) 3.2 (1.9) < < Moderate 3.6 (2.0) 3.6 (1.8) < Mild 4.2 (2.4) 4.4 (2.3) < Summer Strenuous 4.2 (2.0) 4.2 (1.9) < Moderate 4.1 (2.1) 4.2 (2.1) < Mild 4.4 (2.4) 4.5 (2.4) < a 1st survey administration b 2nd survey administration c Pearson Product-Moment Correlation between the 2 survey administrations d Wilcoxon signed rank test for mean difference between the two survey administrations e % from the 2 survey administrations with exact agreement SD = standard deviation Table 3 Mean (Days/Week) and Correlation of Physical Activities Derived from the Modified GLTEQ, YRBS, and Actigraph (N = 114) Questionnaire Mean d/wk P- PPMc Actigraph value b (threshold) a r P -value Modified GLTEQ Strenuous (> 5200)0 < ( 3000) Moderate ( ) YRBS (> 5200)0 < ( 3000) a Thresholds (counts/min) of Actigraph for physical activities were changed for sensitivity analysis purpose b Wilcoxon signed rank test for mean difference between questionnaire and Actigraph c Pearson Product-Moment Correlation between questionnaire and Actigraph GLTEQ = Godin Leisure-Time Exercise Questionnaire; YRBS = Youth Risk Behavior Survey.

6 420 Gao et al. accelerometer readings was A similar magnitude of overestimation was seen for the YRBS question. The average number of days per week of moderate activity reported by the participants was similar to the average number of days based on the accelerometer readings. The PPMs between the number of days per week of participation in strenuous and moderate physical activities as reported during the first administration of the modified Godin questionnaire and as determined from the accelerometer data were 0.23 (P = 0.01) and 0.13 (P = 0.07), respectively (Table 3). The PPM between the number of days per week of participation in strenuous physical activities as derived from 1999 YRBS question and as determined from 7 d of accelerometer data was 0.10 (P = 0.15). Sensitivity analyses conducted by including participants with smaller numbers of days of reliable accelerometer data (e.g., 3 or more days) and by reducing the cut points for strenuous and/or moderate activities (e.g., to 3000 from 5200 vector magnitude counts/min for strenuous activity) yielded similar correlations. Discussion The validity of the question for assessing strenuous activity included in the modified Godin questionnaire was found to be weak, with a correlation of 0.23 between self-reported and accelerometer-measured participation in strenuous activity. The correlation appeared to be lower than that of the Godin questionnaire among adults (r = 0.38), which may be due to the better memory of the adults or their better understanding of the definition of activity. 8 Apart from the low correlation between the reported and the measured participation in strenuous activity, there was considerable overestimation of self-reported compared with accelerometer-measured strenuous physical activity (3.61 vs d/wk). There are several possible explanations for this. First, social desirability bias may have led the participants to report a spuriously high frequency of strenuous activity Another possibility is that participants were accurate in reporting the frequency with which they participate in activities considered to be strenuous, with these activities not captured by the accelerometer due to an inherent limitation of this device in recording certain movements 8,11 (e.g., bicycling, throwing, catching, or lifting) or the nature of the activities that require the participants to take off the accelerometer (e.g., swimming and, possibly, organized sports). It is also possible that some of the participants were unsure of how frequently they participate in strenuous activities, and consequently selected one of the middle response options (3 or 4 d/wk). Indeed, similar reporting of participation in moderate and strenuous activities (average, 3.21 and 3.61 d/wk for moderate and strenuous activities, respectively) provide support for this supposition, and may explain why the average frequency of reported and measured participation in moderate activity were comparable, whereas there was no correlation between these two measures. Another possibility is that the children reported frequency of engagement in any bout of strenuous physical activity regardless of length (e.g., counted bouts of strenuous physical activity less than 15 min in duration). In consideration that children in this age group do not engage in physical activity in a continuous pattern, such as 15-min bouts, but in shorter bursts throughout the day, 11 reported frequency of engagement in strenuous activity would likely be over-reported if this type of reporting error occurs.

7 Measuring Children s PA: Reliability and Validity of a Brief Tool 421 The validity of the YRBS question for assessing strenuous physical activity appeared to be comparable to the strenuous activity question in the modified Godin questionnaire. Also, there was overestimation. The poor correspondence between reported and measured strenuous activity could be attributable, in part, to the different time frames assessed by the YRBS question (past 7 d) and the accelerometer (2 wk post administration of the YRBS question). With respect to the assessment of moderate activity, there was a non-significant weak correlation between the accelerometer and the modified Godin questionnaire (r = 0.13, P = 0.07). The correlation was comparable to the findings of one study that evaluated the Godin questionnaire in adults (r = 0.03, P > 0.05). 12 The low validity observed in our study may reflect poor ability of abstract thinking and memory of children in recalling participation in moderate physical activities. 8 Several limitations must be considered in interpreting findings. First, because we restricted the validity analyses to the participants with 7 d of accelerometer data deemed to be reliable, the generalization of the findings may be limited. However, the sensitivity analyses indicated that the restriction of the study sample appeared not to contribute to the poor validity of the instrument. Another limitation is that our criterion measure (accelerometer) measures both leisure and non-leisure-time activity, whereas the modified Godin questionnaire and the YRBS physical activity question both inquire about leisure time activity only. Hence, the accelerometer data may provide an overestimate of moderate and strenuous leisure time physical activity, although the extent to which children regularly participate in moderate or strenuous non-leisure-time activities is likely minimal. Finally, we established cut points for count readings deemed to correspond with moderate and strenuous physical activity based on data from one study conducted among girls in the 8th grade from urban areas of Baltimore, MD, Minneapolis/St. Paul, MN, and Columbia, SC. The study had been conducted to calibrate accelerometer count cut points for moderate and strenuous physical activity by simultaneously measuring oxygen consumption (VO 2 ) and accelerometer counts across a broad spectrum of common activities. 5 Welk et al., based on a study involving 3648 students from grades 1 to 12, proposed a regression equation for the age-specific count ranges corresponding to moderate (3 to 5.9 METs) and strenuous ( 6 METs) activity. 13 Based on the equation, the intensity cut points should be 1135 to 1706 counts/min for moderate, and 3908 to 4932 counts/min for strenuous activity for our study population, which indicated the cut points (3000 vs. 5200) we selected might be irrelevant. However, the sensitivity analyses conducted indicated that the cut point selection did not obviously influence the validity of the instrument. In conclusion, study findings suggest the Godin questionnaire and 1999 YRBS physical activity question may be of very limited use for assessing physical activity in children. Acknowledgments This work was supported under a cooperative agreement by grant U36/CCU from the Centers for Disease Control and Prevention through the Association of Schools of Public Health.

8 422 Gao et al. References 1. Sallis JF, Buono MJ, Roby JJ, Micale FG, Nelson JA. Seven-day recall and other physical activity self-reports in children and adolescents. Med Sci Sports Exerc. 1993;25: Brener ND, Collins JL, Kann L, Warren CW, Williams BI. Reliability of the Youth Risk Behavior Survey Questionnaire. Am J Epidemiol. 1995;141: Kann L, Warren CW, Harris WA, Collins JL, Douglas KA, Collins ME et al. Youth Risk Behavior Surveillance United States, Journal of School Health. 1995;65: Janz KF, Witt J, Mahoney LT. The stability of childrenʼs physical activity as measured by accelerometry and self-report. Med Sci Sports Exerc. 1995;27: Treuth MS, Schmitz K, Catellier DJ, McMurray RG, Murray DM, Almeida MJ et al. Defining accelerometer thresholds for activity intensities in adolescent girls. Med Sci Sports Exerc. 2004;36: Trost SG, Pate RR, Freedson PS, Sallis JF, Taylor WC. Using objective physical activity measures with youth: how many days of monitoring are needed? Med Sci Sports Exerc. 2000;32: Van Coevering P, Harnack L, Schmitz K, Fulton JE, Galuska DA, Gao S. Feasibility of using accelerometers to measure physical activity in young adolescents. Med Sci Sports Exerc. 2005;37: Matthews CE. Use of self-report instruments to assess physical activity. In: Welk GJ, editor. Physical activity assessments for health-related research. Champaign, IL: Human Kinetics, 2002: Marlowe D, Crowne DP. Social desirability and response to perceived situational demands. J Consult Psychol. 1961;25: Warnecke RB, Johnson TP, Chavez N, Sudman S, OʼRourke DP, Lacey L et al. Improving question wording in surveys of culturally diverse populations. Ann Epidemiol. 1997;7: Welk GJ, Corbin CB, Dale D. Measurement issues in the assessment of physical activity in children. Res Q Exerc Sport. 2000;71(2 Suppl):S59-S Godin G, Shephard RJ. A simple method to assess exercise behavior in the community. Can J Appl Sport Sci. 1985;10: Trost SG, Pate RR, Sallis JF, Freedson PS, Taylor WC, Dowda M et al. Age and gender differences in objectively measured physical activity in youth. Med Sci Sports Exerc. 2002;34:

Best Practices in the Data Reduction and Analysis of Accelerometers and GPS Data

Best Practices in the Data Reduction and Analysis of Accelerometers and GPS Data Stewart G. Trost, PhD Department of Nutrition and Exercise Sciences Oregon State University Key Planning Issues Number of