Executive function measures in early childhood screening: concurrent and predictive validity

Highlights

• Early childhood executive function (EF) predicts learning during kindergarten.

• Early childhood executive function indicates kindergarten readiness.

• Direct assessment of executive function is feasible in early childhood screening.

• Developmental extensions of the NIH Toolbox EF tasks lower the floor of the measures.

Abstract

Executive function (EF) has a critical role in school success, both academic and social-emotional. Nevertheless, there is a lack of evidence on developmentally sensitive measures of EF for young children and on the feasibility of assessing EF for early childhood screening. This study assessed the screening potential of EF tasks, including the National Institutes of Health (NIH) Toolbox measures of EF, Flanker and Dimensional Change Card Sort (DCCS) with developmental extensions (Dext versions). EF measures were added to routine screening of children 3-6 years old in a large urban school district, providing promising data on the validity of the EF measures and their added value beyond traditional screening measures utilized by this district. EF uniquely predicted growth in learning across the kindergarten year. Results also provided evidence that Dext versions of the NIH Toolbox EF measures effectively lower the floor of the Flanker and DCCS tasks, which is essential for their utility in early childhood screening. Findings indicate that EF measures capture a wide range of variability, can be administered during routine screening, show construct validity, and provide unique predictive value for school outcomes.

Introduction

The kindergarten year represents a significant transition into a formal learning environment, with learning and behavioral expectations that are difficult for some children (Rimm-Kaufman & Pianta, 2000). As a result, measures of children's preparedness for school entry, or school readiness, are important for identifying children in need of services (Blair, 2002; Eisenberg, Valient, & Eggum, 2010). The difficulty that some children experience has prompted conceptualizations of school readiness that extend beyond general knowledge and conduct to include processes and skills fundamental to learning in a school context, including self-regulation and executive function (Blair, 2002; Blair & Raver, 2015; Eisenberg et al., 2010).

Executive function (EF) refers to a set of neurocognitive processes and skills involved in the control of attention, action, emotion, and thought, essential to goal-directed behavior (Blair & Ravier, 2015; Zelazo, 2020). These skills include inhibitory control, which is the intentional suppression of attention to a stimuli and subsequent inhibition of impulsive responding; cognitive flexibility, which involves viewing a situation in multiple ways or from multiple perspectives; and working memory, which is holding information in mind for further use or problem solving (Zelazo, Blair, & Willoughby, 2016).

There is growing evidence that EF skills are foundational for learning and that assessments of EF may provide unique insights into school readiness (Allan, Hume, Allan, Farrington, & Lonigan, 2014; Zelazo et al., 2016). Difficulties with EF during the preschool years are negatively associated with academic skills in kindergarten (Willoughby et al., 2017). Concomitantly, evidence also has accumulated that EF skills are malleable, particularly in the preschool years, when early childhood programs are focused on school readiness (Zelazo et al., 2016). These findings have given rise to interest in the feasibility and value of directly assessing EF in early childhood screening. To date, however, there is little data on the potential value of EF assessments for early childhood screening purposes. The goal of this study was to evaluate the potential of EF assessment for inclusion in early childhood screening. This study addressed whether EF tasks are practical for early childhood screening in terms of usability. It further examined the convergent validity of the National Institutes of Health (NIH) Toolbox measures of EF and assessed the value added of incorporating a brief battery of EF tasks over more traditional screening assessments.

The following sections of the introduction highlight key findings from the literature on EF pertinent to this study, including group differences, the relationship with school success, and the amenability to preventative intervention. Subsequently, the literature on developmental screening is discussed with specific attention to the potential for including EF measures.

EF continues to develop steadily across childhood and adolescence but develops most rapidly during the preschool years (e.g., Carlson, Zelazo, & Faja, 2013; Garon, Bryson, & Smith, 2008), underscoring the need for sensitive measures during this time (Carlson, 2005). These rapid changes are thought to be linked to the development of cortical networks involving prefrontal cortex (PFC; Luria, 1973) and increased reliance on anterior regions of PFC (Zelazo, Carlson, & Kesek, 2008 for review). As reviewed by Garon and colleagues (2008), the ability to sustain attention for longer periods of time becomes more internal and voluntary across infancy and into the preschool period. As such, children become better able to sustain their attention, think flexibly, and follow rules instead of acting on urges and natural tendencies.

Although studies consistently show EF improves with age, findings regarding sex differences have been mixed (Zelazo et al., 2016). One early review found sex differences favoring females in more “social” inhibitory control tasks (e.g., emotional control) but not more “cognitive” inhibitory control tasks (e.g., selective attention in a Stroop task; Bjorklund & Kipp, 1996). Carlson and Moses (2001) found that preschool girls outperformed boys on cognitive measures of inhibitory control in a lab setting. When sex differences are found, they are often small in magnitude but typically favor females (Zelazo et al., 2016).

In recent years, research has shown that EF skills predict school-related outcomes, including early learning (Zelazo et al., 2016 for review). For example, Willoughby and colleagues (2017) found that children who persistently performed poorly on measures of EF during the preschool years were at risk for learning-related problems. Further, stronger EF skills during the preschool years provided an immediate advantage in reading and math in kindergarten that persisted into the school-age years (Bull, Espy, & Wiebe, 2008).

In addition to observed short-term benefits of well-developed early EF skills, there are potential long-term implications of problems with these skills. McClelland, Acock, and Morrison (2006) demonstrated that EF difficulties were linked with worse academic performance in kindergarten, a gap that widened until second grade and then persisted from third through sixth grade. In contrast, strong early EF skills may help children initially behind academically to close the gap with same-age peers across the school years (Ribner et al., 2017). Research on individual differences in self-regulatory behaviors relate to indicators of later school success in the expected directions, such as completing high school and college (Duckworth and Carlson, 2013).

Self-regulatory skills may promote children's learning in part through positive influences on peer and teacher relationships. Children with better EF skills have better developed theory of mind skills, which involve being able to take another's perspective (e.g., Carlson & Moses, 2001; Hughes & Ensor, 2007; Müller, Zelazo, & Imrisek, 2005). Further, children with better EF skills show fewer behavioral problems and better social emotional competence (Bierman, Nix, Greenberg, Blair, & Domitrovich, 2008; Masten et al., 2012; Obradović, 2010). They are also better able to inhibit reactive aggressive tendencies (Cole, Usher, & Cargo, 1993) and are more likely to be well liked by classmates and cope after experiencing a social failure (Wilson, 2003).

Interest in measuring EF as part of early childhood screening stems in part from evidence that EF skills are malleable through interventions in early childhood (Zelazo et al., 2016). For example, recent studies suggest that parenting practices forecast EF and academic skills in young children (e.g., Bernier, Carlson, & Whipple, 2010; Meuwissen & Carlson, 2018; Prime et al., 2021). In families experiencing homelessness, Herbers, Cutuli, Supkoff, Narayan, & Masten, (2014) found positive co-regulation by parents related to better EF skills in children entering kindergarten or first grade, suggesting that parenting interventions may be an avenue for boosting EF in early childhood.

There also is evidence that while EF development may be compromised in children who have adverse experiences such as homelessness, poverty, early institutional care, and maltreatment (e.g., Cowell, Cicchetti, Rogosch, & Toth, 2015; Hostinar, Stellern, Schaefer, Carlson, & Gunnar, 2012; Masten et al., 2012; Schmitt, Finders, & McClelland, 2015), strong EF skills may help children at risk for academic difficulties. Masten et al. (2012), for example, found that EF assessed prior to the school year was a unique predictor of academic success, beyond general intellectual functioning and parenting quality, among kindergarten and first-grade children experiencing homelessness. Children with strong EF skills did well in school despite academic risks associated with homelessness. Further, early achievement gaps between homeless students and more advantaged peers persist or even widen over the years (Cutuli et al., 2013; Herbers et al., 2011), making early identification and intervention even more pressing.

School readiness assessment, focused on specific skills believed to be necessary for success in the initial years of primary school, is related to developmental screening, which is focused on identifying deviations from normative development (physical, cognitive, social, or emotional) that may require intervention. Developmental screening is routine in pediatric well-child visits and a standard part of early childhood screening by school districts, with the goal of ascertaining the need for further evaluation and services (Boan, Aydlett, & Multunas, 2007; Brassard & Boehm, 2007). Developmental screening typically includes brief assessments of norm-referenced general knowledge, language, physical health, and social-emotional functioning. Although self-regulation may be assessed through caregiver report or behavioral observations at the time of screening, direct EF assessments have not typically been included in developmental screening. Given the robust associations cited earlier between EF and school success, however, it is important to investigate the relation between EF and developmental readiness screeners, as well as their shared and unique prediction of academic competence and behavior in school settings.

The current study is part of a collaborative, longitudinal project with the Minneapolis Public School (MPS) district. This district has high proportions of disadvantaged children at risk for difficulties with school achievement and adjustment. Minnesota State Law requires that children be screened prior to kindergarten. To address this need, MPS offers free early childhood screening for 3-6-year-old children to identify concerns related to development and school readiness. Children who receive a comparable developmental screening (e.g., by a health care provider) may be exempt, while children who have not been screened prior to the start of kindergarten are assessed at the beginning of that school year.

The MPS district utilizes a locally normed screener developed by their research department and widely utilized by multiple districts in the region (Minneapolis Preschool Readiness Instrument, Revised; Minneapolis Public Schools, 2007, 2016). This screener aims to measure key aspects of development that relate to early school functioning, termed “developmental readiness,” and to identify children at-risk for school problems.

This study had 4 primary aims. The first aim was to test whether newly modified EF measures from the NIH Toolbox that were intended to lower the floor of 2 EF tests in the Cognitive Battery provide a wider range of valid scores for preschool-aged children, which is essential for utility in early childhood screening. The developmental extension (Dext) versions of 2 EF tasks now included in the NIH Toolbox were developed by a collaborative team with funding from the National Children's Study (Carlson et al., 2011; Masten et al., 2011). The Dimensional Change Card Sort with Developmental Extension (DCCS-Dext; Carlson et al., 2021) and the Flanker – Developmental Extension (Flanker-Dext; Anderson et al., 2021) were designed to address the need for developmentally sensitive EF measures that were usable and valid with more diverse preschool-age children. Previous research with the standard versions of NIH Toolbox DCCS and Flanker established validity and reliability but indicated floor effects among preschoolers, particularly those from disadvantaged homes (Akshoomoff et al., 2014). It was hypothesized that a substantial number of children in the diverse urban school district studied here would benefit from the Dext versions of the tests to capture a greater range of performance.

The second aim was to evaluate the convergent validity of the DCCS-Dext and Flanker-Dext measures with respect to a third, widely used measure of EF in the preschool period (Peg Tapping), as well as concurrent validity with respect to demographic variables known to be related to EF. Specifically, it was hypothesized that performance on the EF tasks would be positively related to one another and increase as a function of age and SES. Sex differences were explored but given the mixed findings in the literature, no differences were predicted.

The third aim was to examine the validity of EF as an indicator of school readiness as indexed by the Minneapolis Preschool Screening Instrument-Revised (MPSI-R) and to assess the predictive value of EF for kindergarten academic outcomes and learning across the kindergarten year. Related to this aim, the added value of the EF assessment beyond the MPSI-R was assessed. It was hypothesized that EF scores would be associated with developmental readiness and that EF would predict achievement and learning in kindergarten above and beyond the MPSI-R.

The final study aim was to evaluate the validity of EF as an indicator of social-emotional readiness in relation to the MPSI-R and its predictive value for kindergarten classroom behavior. It was hypothesized that EF scores would be negatively related to social-emotional problems at the time of screening and that stronger EF at the time of screening would predict fewer behavior problems in the classroom.