The Context and Purpose of Executive Function Assessments: Beyond the Tripartite Model Janina Eberhart & Sara Baker Neuroscience & Education Conference 4 th June 2018
Executive Function (EF) EF is important for planning, problem-solving, goal-directed behaviour and organizing of information (Welsh, Pennington, & Groisser, 1991) EF includes working memory, inhibitory control and shifting (Miyake et al., 2000) -> tripartite model Associated with school readiness, academic achievement and social-emotional behaviour (Blair, 2002, Blair & Razza, 2007, Riggs et al., 2006)
Approaches to measure EF Lab-based assessments administered in highly standardized conditions lab-based tasks far removed from real world (Diamond, 2013) Parent or teacher reports more global measure based on a variety of experiences with child subjective Observations objective and ecologically valid time consuming
Lab-based assessments <.30 (Toplak et al., 2013)? Parent or teacher reports? Observations
Why is this relevant? EF is en vogue -> comprehensive curricula developed to enhance EF Evidence of studies rely mostly on lab-based EF assessments How are lab-based EF related to applied EF? Applied EF particularly important for practitioners in a school context
Research Questions? Lab-based assessments? Teacher reports? Observations
Sample: Methods 39 Reception children (53.8% girls) from three different schools in Cambridgeshire and Greater London 4-5 years old (M = 5.13 years) Schools primarily low SES (pupil premium: 8.9%, 38.3%, 49.76%*) Procedure: Children s EF skills measured with lab-based measures Teachers completed an online questionnaire on children s school readiness Children videoed in their naturalistic classroom environment * national average is 14%
Lab-based EF measures: Spin the Pots (Diamond et al., 1997) WM Day and Night (Gerstadt et al., 1994) inhibitory control Dimensional Change Card Sort (DCCS) (Zelazo, 2006) shifting Design Fluency (Korkman et al., 2007) self-directed EF Covariates: Measures Ravens (Raven et al., 1998) general ability Wechsler Preschool and Primary Scale of Intelligence (WPPSI) (Wechsler, 2012) - receptive vocabulary
Measures Observations: Regulation-Related Skills Measure (RRSM) (McCoy et al., 2017) 4 segments: (1) teacher-led activity (2) transition after teacher-led activity (3) child-led activity (4) transition after child-led activity Video coding on 16 items e.g. children s ability to control physical movements, to pay attention, to wait, to ignore distraction, and to inhibit impulsive behavior Last 5 min of a teacher-led and child-led activity Inter-rater agreement (Cohen s Kappa: teacher-led: k =.73; child-led: k =.81)
Measures Teacher report: Brief Early Skills and Support Index (BESSI) (Hughes et al., 2015): 30 items on factors that support school readiness 4 subscales: Behavioural Adjustment Language & Cognition Daily Living Skills Family Support
Behavioural Adjustment
Results Table 1 Means, Range and Standard Deviation of the EF tasks, the covariates, the RRSM, and the BESSI Behavioural Adjustment scale. Variables N Min Max Mean SD Spin the Pots (WM) 39 0.44 1.00 0.71 0.18 DCCS (shifting) 39 6.00 24.00 15.33 5.70 Day and Night (inhibitory control) 37 1.00 16.00 12.00 3.58 Design Fluency (self-directed EF) 38 2.00 19.00 11.32 5.67 Observed EF - Teacher-led 39 2.22 4.00 3.54 0.45 Observed EF - Child-led 31 2.38 4.00 3.70 0.35 BESSI Behavioural Adjustment 39 0.00 12.00 9.64 2.95 Ravens (general ability) 39 10.00 27.00 17.49 4.48 WPPSI - Receptive vocabulary 39 11.00 28.00 20.05 5.31
Results Table 2 Partial correlations between the EF tasks, the RRSM, and the BESSI Behavioural Adjustment scale, while controlling for Ravens, WPPSI receptive vocabulary and age in months. Variables 1 2 3 4 5 6 1. Spin the Pots (WM) 2. DCCS (shifting) -0.20 3. Day and Night (inhibitory control) 0.40* 0.11 4. Design Fluency (self-directed EF) 0.44* -0.01 0.28 5. Observed EF Teacher-led -0.06 0.03-0.33-0.26 6. Observed EF Child-led -0.22 0.06-0.11 0.21 0.08 7. Behavioural Adjustment (BESSI) -0.01 0.01-0.23 0.00 0.49* 0.24 Note: *p <.05, **p <.001
Conclusion Lab-based assessments Teacher reports Observations
Conclusion Jones et al., 2017
Conclusion Different measurement approaches might capture different aspects of EF Measurement approach should be carefully chosen -> go beyond the tripartite model Evidence for programs or interventions should be critically considered We need : more research on the relationship between simple and complex skills more research on lab-based EF and applied EF Ecologically valid measures
Thank you for your attention! Many thanks to Krishna Kulkarni, Hayley Gains, and Catherine McHenry for supporting data collection and video coding.
References Blair, C. (2002). Early intervention for low birth weight, preterm infants: The role of negative emotionality in the specification of effects. Development and Psychopathology, 14(2), 311 332. Blair, C., & Razza, R. P. (2007). Relating effortful control, executive function, and false belief understanding to emerging math and literacy ability in kindergarten. Child Development, 78(2), 647 663. Diamond, A. (2013). Executive functions. Annual Review of Psychology, 64(1), 135 168. Diamond, A., Prevor, M. B., Callender, G., & DruinMonographs Of The Society For Research In Child Development, 62, D. P. (1997). Prefrontal cortex cognitive deficits in children treated early and continuously for PKU. (4), 1-205. doi:10.2307/1166208 Gerstadt, C. L., Hong, Y. J., & Diamond, A. (1994). The relationship between cognition and action: Performance of children 3 1/2-7 years old on a Stroop-like day-night test. Cognition, 53(2), 129 153 Hughes, C. (1998). Executive function in preschoolers: Links with theory of mind and verbal ability. British Journal of Developmental Psychology, 16(2), 233 253. Jones, S. M., Bailey, R., Barnes, S. P., & Partee, A. (2016). Executive Function Mapping Project: Untangling the Terms and Skills Related to Executive Function and Self-Regulation in Early Childhood. OPRE Report # 2016-88, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Korkman, Kirk, & Kemp (2007). NEPSY-II (2 nd edition). San Antonio: Pearson. McCoy, D. C., Jones, S., Leong, D., Bodrova, E., Koepp, A., & Hemenway, A. (2017). The Regulation-Related Skills Measure. Cambridge, MA: Harvard University. Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., & Howerter, A. (2000). The unity and diversity of executive functions and their contributions to complex frontal lobe tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49 100. Raven, J., Raven, J. C., & Court, J. H. (1998). Manual for Raven s Progressive Matrices and Vocabulary Scales, Section 2: The Coloured Progressive Matrices. San Antonio, TX: Harcourt Assessment. Riggs, N. R., Greenberg, M. T., Kusché, C. A., & Pentz, M. A. (2006). The Mediational Role of Neurocognition in the Behavioral Outcomes of a Social-Emotional Prevention Program in Elementary School Students: Effects of the PATHS Curriculum. Prevention Science, 7(1), 91-102. doi:10.1007/s11121-005-0022-1 Toplak, M. E., West, R. F., & Stanovich, K. E. (2013). Practitioner Review: Do performance-based measures and ratings of executive function assess the same construct? Journal of Child Psychology and Psychiatry, 54(2), 131 143. Wechsler, D. (2013). Wechsler Preschool and Primary Scale of Intelligence (4 th edition). London: Pearson. Welsh, M. C., Pennington, B. F., & Groisser, D. B. (1991). A normative-developmental study of executive function: A window on prefrontal function in children. Developmental Neuropsychology, 7(2), 131 149. Zelazo, P. D. (2006). The Dimensional Change Card Sort (DCCS): a method of assessing executive function in children. Nat. Protocols, 1(1), 297 301.