Measurement of aggressive behavior in early childhood: A critical analysis using five informants

https://doi.org/10.1016/j.jecp.2021.105180Get rights and content

Highlights

  • A two-factor model of relational and physical aggression fit the data well.

  • Five different methods had weak to strong correspondence in early childhood.

  • Classroom clustering was greater for relational compared to physical aggression.

  • There were within and between group gender differences in aggression.

Abstract

Measurement of aggressive behavior in early childhood is unique given that relational aggression is just developing, physical aggression is still prevalent, and both forms of aggression are relatively overt or direct. The current study had three aims. The first aim was to examine the internal reliability, validity, and correspondence of five different assessments of aggressive behavior in early childhood: parent report, teacher report, observer report, child report, and naturalistic school-based observations. The second aim was to test a one- and two-factor model of early childhood aggression using confirmatory factor analysis. The final aim of the study was to investigate gender differences among different reports of aggression. Observations, teacher report, and observer (research assistant) report were collected in the children’s school, and parent report and child report were collected in a lab session at one time point (N = 300; 56% male; Mage = 44.86 months, SD = 5.55). Observations were collected using a focal child sampling with continuous recording approach, and previously validated measures were used for the remaining four informants. Results demonstrated that all measures were reliable with the exception of child report of relational aggression, and there was small to strong correspondence among the various informants. In addition, a two-factor structure of aggression provided the best fit to the data, providing evidence for divergence among relational and physical aggression. Finally, there were robust gender differences in physical aggression, but gender differences in relational aggression varied by method. The implications of different types of measurement are discussed.

Introduction

The study of aggression has a rich history, including distinguishing different forms and functions of aggression and identifying developmental changes in aggression (e.g., Hartup, 1974). Aggression, defined as behaviors intended to hurt or harm an individual, can be enacted using distinct means, including via physical harm (e.g., verbal threats of harm, hitting, kicking) and through damage to relationships (e.g., exclusion, gossiping, verbal threats of withdrawing) (Crick and Grotpeter, 1995, Eisner and Malti, 2015). The majority of early research on aggressive behaviors in childhood focused on elementary school children, with less research among early childhood samples (Landy & Peters, 1992). In addition, studies of aggression in early childhood tended to focus on physical forms of aggressive behavior across a variety of assessment methods, including peer nomination, self-report, and teacher ratings (e.g., Behar and Stringfield, 1974, Johnston et al., 1977) and, perhaps most commonly, observations (Johnston et al., 1977). Use of these various methods often leads to divergent conclusions about the independence among subtypes of aggressive behavior, gender differences in aggressive behavior, and the various contexts in which aggressive behavior occurs (Card, Stucky, Sawalani, & Little, 2008). The current study evaluated the internal reliability and validity of five different measures of relationally and physically aggressive behavior in early childhood. The ability to distinguish among different informants of aggressive behavior, draw conclusions about the methodological rigor of these various methods, and evaluate the structure of aggressive behavior using multiple methods is critical to accurately conceptualizing aggression in early childhood.

In the research literature, there has been a focus on physical and relational forms of aggression in part because of the historical interest in between- and within-group gender differences in aggression and gender differences in longitudinal studies linking aggression and social-psychological adjustment outcomes (e.g., Crick and Grotpeter, 1995, Crick et al., 2006, Lagerspetz et al., 1988, Murray-Close et al., 2008). In fact, in early research on the development of aggression in early childhood, the focus on physical and verbal forms of aggression led researchers to conclude that boys were more likely than girls to exhibit aggressive behaviors (e.g., Behar and Stringfield, 1974, Hartup, 1974). However, this conclusion was challenged by research conducted during the 1980s and 1990s, when the study of nonphysical forms of aggression, such as indirect aggression (Lagerspetz et al., 1988), social aggression (Cairns et al., 1989, Galen and Underwood, 1997), and relational aggression (Crick & Grotpeter, 1995) gained prominence.

The developmental period of early childhood (3–5 years) serves as a critical period for studying the development of physical and relational aggressive subtypes for several reasons. First, forms of aggression may have different correlates and prevalence rates across development (Sijtsema & Ojanen, 2018), highlighting the need for research focused on distinct developmental periods, including early childhood. Second, although both physical and relational aggression are commonly expressed during this period, key developmental changes occur among typically developing children. For instance, children exhibit an increase in physical aggression after the onset of expressive language and then rapid declines as they transition to kindergarten (National Institute of Child Health and Human Development Early Child Care and Research Network, 2004, Tremblay et al., 2005). As this developmental shift occurs, relational aggression appears to increase (see Fite & Pederson, 2018).

Third, relational aggression appears to be qualitatively different in early childhood compared to other periods of development (see Casas & Bower, 2018). Specifically, relational aggression in early childhood tends to be direct, the identity of the perpetrator is known, and such aggression is based on the “here and now” and typically does not reflect retaliation for a prior hostile exchange that took place days or weeks earlier (Casas & Bower, 2018). For these reasons, the focus of the current study was on physical and relational aggression during the developmentally salient early childhood period. Given previous research and conceptualizations of physical and relational aggression as related but distinct, it was hypothesized that a two-factor (i.e., physical and relational) model of aggressive behavior with a moderate correlation between the two factors would fit the data better than a one-factor aggression model.

A critical question for researchers investigating forms of aggression in early childhood is what methods are best suited to capturing these behaviors and whether results differ when different methods are adopted. The most common measures of physical and relational aggression in early childhood include observations, self-report, and other report (e.g., teacher, peer, parent, and observer reports). Importantly, children may behave differently within the various settings they encounter; therefore, assessing behavior in a single setting (e.g., school) might not capture a child’s overall aggressive tendencies.

One of the most rigorous methods for studying aggression is using behavioral observations. Typically, observations in early childhood are conducted within schools because preschool-age children most frequently interact with peers in this setting. Children have low reactivity to observers, particularly when observers are in the classroom for a period of time before the observations (i.e., reactivity period; Ostrov & Keating, 2004) and are trained to be minimally reactive to the children. In the current study, we used a focal child sampling with continuous recording observation procedure (Juliano et al., 2006, Ostrov and Keating, 2004), in which a trained observer conducts an observation on a selected focal child and notes each interaction that the focal child has with other children. Observations are recorded over a set period of time, and the trained observer records interactions within a short distance of the child but does not interact with the child (Juliano et al., 2006, Ostrov and Keating, 2004). Close proximity between observers and focal children allows for the trained observer to differentiate between physical aggression and rough-and-tumble play (Pellegrini, 1989). Observations using the focal child sampling approach have been shown to have good reliability (e.g., Ostrov, Ries, Stauffacher, Godleski, & Mullins, 2008), providing support for the utility of behavioral observations of aggression in early childhood.

Child reports are designed to collect children’s unique perspective about their own aggressive behaviors across multiple contexts (Godleski and Ostrov, 2020, Phares et al., 1989) and may be administered within an interview format for young children (e.g., Godleski & Ostrov, 2020). However, there has been some debate about whether young children are able to reliably and validly report on their own behavior (McLeod, Southam-Gerow, & Kendall, 2017). Specifically, they might not have insight into their own behavior, may respond with extremes, or might not report on behavior that is socially undesirable because they want to please adults (Chambers and Craig, 1998, Chambers and Johnston, 2002). Notably, not much work has used child report in early childhood. Previous work examining concordance rates during later developmental periods has found that there are often disagreements among informants. For example, reports from one study found that cross-rater correlations ranged from .10 to .65, with the largest disagreements being between peer assessments and self-reports (Pakaslahti & Keltikangas-Jarvinen, 2000).

Given the limited use of child self-report, particularly in early childhood, further investigation into this methodology is important to evaluate the utility of young children’s perspective on their aggressive behavior. Prior work in early childhood using the same child report method used in the current study has found support for the reliability and validity of this method (Godleski & Ostrov, 2020). Specifically, when using a developmentally appropriate child interview, there was acceptable reliability for relational and physical aggression (Cronbach’s α > .68) and evidence of validity, such that child reports of relational and physical aggression were correlated (Godleski & Ostrov, 2020).

Given the challenges of relying on young children as reporters of aggressive behavior and the time-intensive nature of observations, many researchers have used other reporters as informants of a child’s aggressive behavior. These include parent, teacher, peer, and observer informants.

Parent report has historically been a commonly used index of children’s behavior problems, including aggressive behavior (Achenbach & Edelbrock, 1978). Parent reports of their children’s relational and physical aggression, rated on a Likert-type scale (Casas et al., 2006, Ostrov and Bishop, 2008), may provide details on children’s aggressive behavior in a novel context that teachers and observers are not privy to (e.g., with peers on play dates). In addition, parents may be able to recognize more private displays of aggressive behavior, which are harder for teachers and observers to notice (Ostrov and Bishop, 2008, Pellegrini and Bartini, 2000). However, parents might not be as objective as teachers and other observers when reporting on their own children’s behavior with peers. In previous studies, parent measures have shown acceptable internal consistency and, in some work, moderate correlations with teacher reports of physical and relational aggression (Ostrov & Bishop, 2008). However, in other studies, parent report was not significantly associated with teacher or observer report, potentially indicating that parents are reporting on aggression outside of the classroom setting (Ostrov, Gentile, & Mullins, 2013).

Teacher reports are used to examine the frequency of aggressive behavior displayed within the classroom. Many researchers have used teacher reports of children’s aggression (e.g., Johnson and Foster, 2005, McEvoy et al., 2003, McNeilly-Choque et al., 1996). Previous studies have found teachers to be reliable reporters of relational aggression (Estrem, 2005, Juliano et al., 2006), with weak to strong overlap with observer reports, observations, and peer reports (Crick et al., 1997, Crick et al., 2006, Johnson and Foster, 2005). Teachers are also reliable reporters of physical aggression (Crick et al., 1997, Hawley, 2003), with weak to strong correlations with observer reports, behavioral observations, and peer reports (Casas et al., 2006, Estrem, 2005, Ostrov et al., 2008).

One limitation to the use of teacher reports is that they reflect children’s behavior in the school environment. In addition, teachers do not always witness the behaviors that occur during peer-to-peer interactions. Despite these limitations, teacher reports of aggressive behavior are one of the most commonly used methods for assessing young children’s aggression. Teachers witness more daily peer-to-peer interactions than parents, which may make them better informants of peer-directed aggression than parents. In addition, teachers are generally experienced with children’s peer interactions and therefore have an idea of what “typical” behavior is for preschool children. Finally, teachers have consistently been reliable and valid informants of children’s aggressive behavior (Estrem, 2005, Johnson and Foster, 2005, Juliano et al., 2006), providing support for the utility of this method.

Other reporters have also been used to examine aggressive behavior in children such as camp counselors and observers (Murray-Close et al., 2008, Ostrov, 2008). For instance, after completing the observations in a classroom where observers are around children for an extended period of time (e.g., 2 or 3 months; Ostrov & Keating, 2004), observers may be randomly assigned to complete “teacher report” measures for students within that classroom (Ostrov, 2008). This method has been reliable in several studies for both physical and relational aggression and has shown small to strong correlations with teacher reports and naturalistic school-based observations of aggressive behavior (e.g., Murray-Close and Ostrov, 2009, Ostrov, 2008). Observer reports of aggressive behavior allow for an objective reporter trained to efficiently identify specific acts of aggression to assess children’s aggressive behavior in a classroom context.

A large extant literature focuses on identifying informant discrepancies, why they exist, and the meaning of such discrepancies (e.g., Achenbach et al., 1987, De Los Reyes, 2013). Recent meta-analytic work has demonstrated that informant agreement is generally larger for more observable behavior when informants are reporting within the same context (e.g., mothers and fathers) and when using continuous measurements (see De Los Reyes et al., 2015). One theoretical model that addresses the interpretation of informant discrepancies is the operations triad model (OTM; De Los Reyes, 2013). The OTM is composed of three parts: converging operations, where stronger confidence in results is drawn when there are similarities in relations between target constructs and outcomes, diverging operations, where differences in outcomes based on informants are theorized to be meaningful, and compensating operations, where differences in informants are due to measurement errors or structural measurement differences (De Los Reyes, 2013). The current study is a precursor to an OTM study with a focus on similarities and differences across informants rather than associations with outcomes.

Specifically, the first aim of the study was to examine the internal reliability and validity of five different methods of aggressive behavior in early childhood: child report, naturalistic observations, observer (research assistant) report, parent report, and teacher report. Based on previous research, it was expected that cross-informant agreement (e.g., converging operations) would be stronger for physical aggression compared with relational aggression because physical aggression is more overt, and therefore more observable, among informants. In addition, it was expected that there would be more agreement among informants in the school context (i.e., teacher report, observations, and observer report) compared with the home context (i.e., parent report) and self-report. Consistent with compensating operations, the reliability was examined for each measure of aggressive behavior and was expected to vary by informant. There was expected to be more convergence for informants using the same measure (i.e., teacher report and observer report) compared with informants using different measures or assessments (i.e., parents vs. teachers, parents vs. observations). Additionally, the second aim of the current study was to evaluate whether a one or two-factor model would fit the data better. Given previous research that has found that the two forms of aggression are related but distinct, it was hypothesized that a two-factor model of aggressive behavior would fit the data best.

The third aim of the study was to evaluate the role of gender in assessments of aggression. In contrast to the common conception of girls as nonaggressive, meta-analytic findings and cross-cultural research suggest that boys and girls engage in similar levels of relational aggression (Card et al., 2008, Lansford et al., 2012). However, gender effects vary by informant; for instance, studies using parent and teacher reports are more likely to report that girls exhibit more relational aggression than boys (Card et al., 2008). Consistent with the extant literature (Card et al., 2008), we hypothesized that between-group analyses would demonstrate higher physical aggression scores among boys than among girls across all measures of aggressive behavior. Prior to evaluating mean gender differences in latent variables of physical and relational aggression, the measurement invariance of the model was tested to evaluate whether the assessments functioned differently for boys and girls in predicting latent factors. Prior research has found mixed evidence for the measurement invariance of different measures across gender, with items functioning differently for boys and girls when using general measures of aggressive behavior (Kim et al., 2010) but with items functioning similarly when evaluating a bifactor model of aggressive behavior (Perry and Ostrov, 2018). Therefore, it was hypothesized that a two-factor confirmatory factor analysis (CFA) of physical and relational aggression would demonstrate strong factorial invariance across gender. In addition, we expected no gender difference in the relational aggression factor but potential gender differences on individual measures of relational aggression such as parent and teacher reports. We further expected that within-group analyses would demonstrate higher relational aggression scores than physical aggression scores among girls across all measures of aggressive behavior. For boys, it was hypothesized that there would be no difference in physical and relational aggression scores or that physical aggression scores would be higher.

The current research drew on a large multicohort study (N = 300) to examine the reliability and inter-method correspondence for five different methods of assessing aggressive behavior in early childhood: child report, naturalistic observations, observer report, parent report, and teacher report. Classroom-level nesting of different measures of aggressive behavior was also examined. The second goal was to evaluate the structure of aggressive behavior using these different methods. A CFA was conducted to test the factor structure of aggressive behavior (i.e., physical and relational aggression). It was hypothesized that a two-factor structure, with a moderate correlation between the two factors, would fit the data better than a one-factor model. The third aim of the study was to examine the role of gender in aggression by considering the measurement invariance of the final model across gender, between- and within-group gender differences in different measures of aggression, and between-group gender differences in the latent factors of aggression. We hypothesized that there would be within- and between-group gender differences in physical and relational aggression.

Section snippets

Participants

A total of 300 children (44.0% girls; Mage = 44.86 months, SD = 5.55) from four cohorts participated in the current study, which is part of a larger study. The sample was somewhat diverse (3.0% African American/Black, 7.6% Asian/Asian American/Pacific Islander, 1.0% Hispanic/Latinx, 11.3% multiracial, 62.1% White, and 15.0% missing/unknown). Parental occupation was gathered at enrollment and was coded using Hollingshead’s (1975) four-factor index 9-point scoring system (e.g., 9 = executives and

Inter-method correspondence

In regard to relational aggression, observations, observer reports, and teacher reports were positively correlated (rs = .21–.40, p < .01) (see Table 1). Parent and child reports of relational aggression were not significantly correlated with the other informants but were significantly correlated with each other (r = .31, p < .001). In regard to physical aggression, observations, observer reports, and teacher reports were positively correlated (rs = .40–.41, p < .001). Parent reports were

Discussion

There were three aims of the current study. The first aim was to examine the internal measurement characteristics of different measures of aggressive behavior. Results demonstrated that all methods were reliable with the exception of child report of relational aggression. In addition, as expected, inter-rater correspondence of aggression was weak to strong, with greater correspondence when using physical aggression and the weakest correspondence between parents and other raters. The second aim

Acknowledgments

We thank the PEERS project staff and the participating families, teachers, and schools for their contributions to and support of this project. We thank Dr. Kimberly Kamper-DeMarco, Lauren Mutignani, Sarah Probst, Samantha Kesselring, and many research assistants for data collection and coordination. In addition, we acknowledge Erin Dougherty for reference checking. Research reported in this publication was supported by a National Science Foundation (NSF) grant (BCS-1450777) to the second and

References (62)

  • N.E. Werner et al.

    Mothers’ responses to preschoolers’ relational and physical aggression

    Journal of Applied Developmental Psychology

    (2006)
  • T.M. Achenbach et al.

    The classification of child psychopathology: A review and analysis of empirical efforts

    Psychological Bulletin

    (1978)
  • T.M. Achenbach et al.

    Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity

    Psychological Bulletin

    (1987)
  • J. Archer et al.

    An integrated review of indirect, relational, and social aggression

    Personality and Social Psychology Review

    (2005)
  • L. Behar et al.

    A behavior rating scale for the preschool child

    Developmental Psychology

    (1974)
  • R.B. Cairns et al.

    Growth and aggression: I. Childhood to early adolescence

    Developmental Psychology

    (1989)
  • N.A. Card et al.

    Direct and indirect aggression during childhood and adolescence: A meta-analytic review of gender differences, intercorrelations, and relations to maladjustment

    Child Development

    (2008)
  • J.F. Casas et al.

    Developmental manifestations of relational aggression

  • C.T. Chambers et al.

    Developmental differences in children’s use of rating scales

    Journal of Pediatric Psychology

    (2002)
  • N.R. Crick et al.

    Relational and overt aggression in preschool

    Developmental Psychology

    (1997)
  • N.R. Crick et al.

    Relational aggression, gender, and social-psychological adjustment

    Child Development

    (1995)
  • A. De Los Reyes

    Strategic objectives for improving understanding of informant discrepancies in developmental psychopathology research

    Development and Psychopathology

    (2013)
  • A. De Los Reyes et al.

    The validity of the multi-informant approach to assessing child and adolescent mental health

    Psychological Bulletin

    (2015)
  • A. De Los Reyes et al.

    Linking informant discrepancies to observed variations in young children’s disruptive behavior

    Journal of Abnormal Child Psychology

    (2009)
  • Eisner, M. P., & Malti, T. (2015). Aggressive and violent behavior. In M. E. Lamb & R. M. Lerner (Eds.), Handbook of...
  • T.L. Estrem

    Relational and physical aggression among preschoolers: The effect of language skills and gender

    Early Education & Development

    (2005)
  • Fite, P. J., & Pederson, C. A. (2018). Developmental trajectories of relational aggression. In S. M. Coyne & J. M....
  • B.R. Galen et al.

    A developmental investigation of social aggression among children

    Developmental Psychology

    (1997)
  • W.W. Hartup

    Aggression in childhood: Developmental perspectives

    American Psychologist

    (1974)
  • Hollingshead, A. B. (1975). Four factor index of social status. Unpublished working paper, Department of Sociology,...
  • L.-T. Hu et al.

    Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives

    Structural Equation Modeling

    (1999)
  • Cited by (13)

    • Early life exposure to lead and its association with parent-reported aggression and conduct problems during childhood and adolescence

      2022, NeuroToxicology
      Citation Excerpt :

      It has been observed that measures of child behavior and psychopathology obtained from different informants (e.g., parents, teachers, child self-report) do not always have good correspondence, which might reflect at least different contexts in which children are observed (e.g., school vs. home), different definitions of abnormal behavior by different informants, and different sources and magnitude of measurement error for different informants (De Los Reyes, 2013). Although parent-reported aggression in early childhood has been found to be reliable in some contexts, (Perry et al., 2021) the possibility remains that our results were affected by measurement error of the outcome. If this error was nondifferential in terms of the levels of the exposure (e.g., parents attenuated their child’s level of aggression/conduct problems independently of the level of Pb exposure), it is likely that this error would lead to bias towards the null value in our association estimates, potentially obscuring existent associations where we observed none (early childhood and periadolescent models).

    View all citing articles on Scopus
    View full text