Predicting pathways to optional summer science experiences by socioeconomic status and the impact on science attitudes and skills

Abstract

Background

Large achievement and motivation gaps exist in science between students from higher and lower socioeconomic status (SES) backgrounds. Middle and high school are an important time to address these disparities, as science motivation typically declines for all students at this time, leading to particularly low science interest and achievement for lower SES students on average when the gaps are left unaddressed. Students’ control over their free time also increases at this time, providing opportunities for optional science experiences that may improve science attitudes and skills to combat these achievement and motivation gaps. Using a longitudinal dataset of 2252 middle and high school students from two regions in the USA, we investigate (1) disparities between higher and lower SES students in participation in optional summer science experiences and post-summer science attitudes and skills; (2) whether the child and family characteristics that predict participation in home-related, nature-related, and STEM camp experiences in the summer differ for higher and lower SES students; and (3) how participation in these types of optional summer science experiences contribute to post-summer science attitudes and skills when controlling for self-selection biases.

Results

Higher SES students reported greater participation in optional summer science experiences and higher post-summer science attitudes and sensemaking skills. Fascination for science was more important for participation in home-related and nature-related experiences for higher SES participants, whereas science competency beliefs were more important for lower SES participants. For STEM camp experiences, higher SES participants with higher competency beliefs and lower SES participants with lower scientific sensemaking skills were more likely to participate. After controlling for self-selection biases that may influence participation in these experiences, we found that home-related and nature-related experiences had a positive impact on students’ attitudes toward science.

Conclusions

Our findings suggest two pathways for increasing participation in optional summer science experiences for higher SES and lower SES students. Specifically, it may be helpful to support interest in science for higher SES students and competency beliefs for lower SES students. Greater participation in home-related and nature-related summer science experiences can also increase science attitudes during middle and high school.

Introduction

Large science achievement gaps exist in US schools based on socioeconomic status (SES). In 2015, the U.S. Department of Education (2015) found that only 18% of 8th grade students who were eligible for the National School Lunch Program (NSLP) reached proficiency in science, whereas 49% of students who were not NSLP-eligible reached the same levels. Data from the Early Childhood Longitudinal Study have similarly found moderate differences in science achievement related to household income and parental education (Betancur, Votruba-Drzal, & Schunn, 2018). The trajectory of science achievement between students from lower SES families compared to higher SES families suggests that these difficulties are cumulative over time (Morgan, Farkas, Hillemeier, & Maczuga, 2016), making the socioeconomic gap particularly persistent.

The socioeconomic gap extends into students’ motivation for science. Lower SES kindergarteners report less motivation to learn science compared to their higher SES peers (Saçkes, Trundle, & Bell, 2013), and these differences continue through middle and high school (e.g., Morgan et al., 2016; Saçkes et al., 2013; Zhang, Hu, Ren, & Zhang, 2019). There are several mechanisms through which SES is theorized to influence science learning and motivation. Some theories posit that income has a direct influence on the learning resources available for families and students (e.g., Huang, Guo, Kim, & Sherraden, 2009). For example, wealthier families may be able to buy more physical learning resources to use at home (Bradley, Corwyn, McAdoo, & Coll, 2001) or afford access to schools with more experienced teachers and advanced class offerings (Clotfelter, Ladd, & Vigdor, 2006), which can lead to better educational outcomes for their children. Higher SES may also allow parents more time to spend directly supporting their children’s learning (Betancur et al., 2018). Another theory suggests that SES can constrain students’ educational expectations and goals. Families from lower SES backgrounds may make decisions about available educational options based on the costs of further education, the expected probability of success in education, or the expected benefits for their family, more so than higher SES families (Breen & Goldthorpe, 1997). This may manifest as the value that students place on science or the factors that students consider most when making decisions about science (e.g., participating in optional science activities, taking science classes). For example, a longitudinal study following students in England from high school to university found evidence that students with less educated parents were more likely to consider extrinsic benefits of a degree (e.g., financial returns) when choosing their majors, whereas students with more educated parents were more likely to decide based on intrinsic benefits (e.g., whether they personally enjoyed or thought they were good at the subject), even when controlling for students’ and parents’ enjoyment or beliefs about their abilities in science and math (Mcmaster, 2019). Traditionally underrepresented students may also feel as though they do not fit with traditional ideas of who a scientist is, based on historical contexts of science that have privileged those with higher SES backgrounds (Archer, Dewitt, & Osborne, 2015); indeed, when Visintainer (2016) interviewed high school students of color, some students envisioned scientists as people who were already smart and high-status and felt there were limited options for who could do and be good at science, suggesting that dominant ideas of scientists can influence their attitudes toward science and skills.

These disparities compound the steady decline in science interest that is typically seen for all students around this period (e.g., Potvin & Hasni, 2014; Vedder-Weiss & Fortus, 2010), leading to particularly low levels of science motivation for low SES adolescents (Aschbacher, Ing, & Tsai, 2014). Because greater interest in science has been shown to predict science identity (Maltese & Tai, 2010) and STEM career choices (Maltese & Tai, 2011), this can have significant implications for low SES students’ persistence in STEM. The gaps in science achievement and science motivation also sustain each other; children with higher achievement are more motivated to learn science (Saçkes et al., 2013), and children with higher science motivation are more likely to pursue opportunities that lead to greater science achievement (Alexander, Johnson, & Kelley, 2012; Bryan, Glynn, & Kittleson, 2011). Because this developmental period is critical for the development of science motivation, investigating ways to decrease the educational debt accumulated by lower SES students in science achievement and motivation at this age may be particularly effective for improving science outcomes.

Starting around middle school, children have a growing amount of control over the types of activities in which they engage (Dierking & Falk, 2003). Students spend most of their waking hours outside of formal school activities (National Center for Education Statistics, 2008), and they have options to participate in informal experiences outside of school that can contribute to their knowledge and interest in science (Dierking, Falk, Rennie, Anderson, & Ellenbogen, 2003; National Research Council, 2009, 2015). This free time increases during the summer; as such, informal summer experiences may be especially fruitful for diminishing initial science achievement and motivation gaps seen at the start of the school year. However, socioeconomic status still appears to play a role in summer learning opportunities. Many studies show that higher and lower SES students show similar academic growth during the school year, but higher SES students outpace lower SES students during the summer, also known as summer learning loss (Alexander, Entwisle, & Olson, 2007; Coley, Kruzik, & Votruba-Drzal, 2019). These differential summer changes are a large contributor to the achievement gap between low-income students and their middle- and upper-class peers, with students at the 10th percentile of the income distribution scoring 1.3 standard deviations lower on average than students at the 90th percentile of the income distribution in math and reading tests (Reardon, 2011). Understanding differences in participation in summer science experiences between higher and lower SES students can help in addressing the summer learning loss phenomenon and lead to higher academic achievement (McCombs et al., 2011).

Informal learning experiences come in many types, which may not be equally accessible or effective for students. The After-School Corporation (2014) estimated that a 6th grader from a lower SES family will have spent approximately 3000 fewer hours in after-school or extracurricular activities, 1000 fewer hours in summer programs, and 250 fewer hours visiting zoos, museums, and other similar locations than their higher SES peers. Hill, McQuillan, Hebets, Spiegel, and Diamond (2018) similarly found that lower SES students were less likely to visit public libraries and museums, watch nature shows, engage in science-related scouting activities, and indicate interest in joining an after-school science club than higher SES students. Financial and time costs may impact the extent of participation in optional science experiences; for example, lower SES middle and high school students appear to engage in more everyday science learning experiences (e.g., watching science TV programs, visiting science websites) compared to informal (e.g., visiting zoos and museums, taking nature walks) or school-led (e.g., taking science-related school trips, attending talks about science) experiences, which may be due to the need for more planning and expenses for the latter two experience types (DeWitt & Archer, 2017). Families also report feeling that some informal activities are structured in ways that privilege dominant communities’ conceptions of science and can feel exclusionary to lower-income families (Dawson, 2014a, 2014b). More information is needed about the factors that can support participation in these optional summer experiences, and whether high and low SES students take different factors into account when deciding whether to participate.

In the current study, we investigate whether there are gaps between high and low SES students in participation in optional summer science experiences and post-summer science motivation and skill, which student- and family-level factors predict participation in three types of optional summer science experiences for high SES and low SES students, and how participation in these experiences impact science attitudes and skill.

Optional science experiences, motivation, and achievement

Prior studies show that optional science experiences positively affect science interest and learning (e.g., Fredericks & Simpkins, 2012; McCombs et al., 2011; Quigly, Pongsanon, & Akerson, 2010; Sahin, 2013; Sha, Schunn, & Bathgate, 2015). We investigate three common types of summer science experiences: home-related experiences that take place around students’ homes or with their families (e.g., watching science TV programs, going to science museums), nature-related experiences (e.g., exploring and collecting items from nature, gardening), and STEM camps or programs (Lin & Schunn, 2016). Each of these experience types has been associated with benefits in attitudes, as reviewed below. However, their individual connections to summer changes have not been well examined, especially with respect to heterogeneous experience by SES or with respect to science skills.

Home-related experiences have often been associated with positive effects on science attitudes across a wide range of ages (e.g., Dabney et al., 2012; Henriksen, Jensen, & Sjaastad, 2015; Simpkins, Davis-Kean, & Eccles, 2006; Uitto, Jutti, Lavonen, & Meisalo, 2006). For example, children’s participation in science activities in 5th grade was positively associated with their ratings of science importance, interest, and self-concept a year later (Simpkins et al., 2006). Similarly, a study involving high school students found that more experiences with science and technology-related activities highly correlated with an interest in biology (Uitto et al., 2006). College students in STEM disciplines also report that experiences with reading or watching popular science and science fiction and with science hobbies played a significant role in their decisions to enter STEM programs (Henriksen et al., 2015), and other studies have also found that participation in these out-of-school experiences is strongly associated with university students’ STEM career interests (Dabney et al., 2012).

Similar to home-related experiences, nature-related experiences have also been associated with greater science interest, positive affects toward science, and stronger science identities (Uitto et al., 2006). College STEM students have also cited nature-related experiences as an inspiration for pursuing STEM careers, though to a lesser extent than home-related experiences (Henriksen et al., 2015). More structured nature-related experiences, including urban community gardening projects, school-based gardening projects, and outdoor classrooms, have also been used to foster positive attitudes toward science and, in some cases, higher science achievement (Berezowitz, Bontrager, & Schoeller, 2015; Education & Roundtable, 2000; Fusco, 2001; Klemmer, Waliczek, & Zajicek, 2005; Mittelstaedt, Sanker, & VanderVeer, 1999; Williams, Brule, Kelley, & Skinner, 2018).

Some STEM camps have also been found to improve both science attitudes and achievement (Bischoff, Castendyk, Gallagher, Schaumloffel, & Labroo, 2008; Gibson & Chase, 2002; Hayden, Ouyang, Scinski, Olszewski, & Bielefeldt, 2011; Knox, Moynihan, & Markowitz, 2003; Levine, Serio, Radaram, Chaudhuri, & Talbert, 2015; Markowitz, 2004). For example, students enrolled in the Summer Science Academy program at the University of Rochester reported more confidence in their science abilities, higher motivation to pursue a science career, and better performance in their science classes (Knox et al., 2003). These improvements were also long-term; a follow-up study found that these students were more likely to participate in other science programs and performed better in later advanced science courses (Markowitz, 2004).

The positive effects associated with summer science experiences suggest they could increase start-of-year science motivation and achievement, but their relative contributions to science attitudes and skill have not been examined. Understanding the contributions of different types of experiences can determine the experiences that are particularly helpful for science achievement and motivation. Critically, it is important to consider typical programs to which families will have access, not just the best programs that receive special research attention. Prior studies on science attitudes and achievement have focused on summer programs that were specifically designed to promote positive feelings toward science or prepare students to clear academic hurdles (Potvin & Hasni, 2014), which may not be representative of the average summer science-related program (Cooper, Charlton, Valentine, & Muhlenbruck, 2000). Indeed, some recent studies using regional and international datasets have found that higher participation in out-of-school experiences is negatively correlated with science achievement scores (Lin & Schunn, 2016; Suter, 2016). Prior work also suggests that the average learner that self-selects into optional science experiences already has higher science interest and self-efficacy (Alexander et al., 2012; Vallett, Lamb, & Annetta, 2018). Thus, it is important to investigate these questions not only in the context of individual experiences but also in larger, more representative datasets, and to control for any initial differences in students who attend these experiences versus those who do not. It is also important to investigate whether the characteristics of typical participants differ based on SES to understand whether support can be provided to help students participate in new experiences that may be beneficial for their achievement and motivation.

The current study

In the current study, we investigate summer changes in science attitudes and skill, as well as the influence of out-of-school summer experiences in a regional, rather than program-specific, dataset. Specifically, we aim to answer three research questions:

• 1. Does participation in home-related, nature-related, and STEM camps science experiences and post-summer science attitudes and skills differ based on SES?

• 2. What child and family characteristics are associated with participation in home-related, nature-related, and STEM camps science experiences for higher SES and lower SES students?

• 3. What are the effects of home-related, nature-related, and STEM camps science experiences on science attitudes and skill, controlling for self-selection biases?

Theoretical/conceptual framework

The study uses longitudinal data from the ALES14 and ALES15 datasets (Activated Learning Enables Success 2014/2015) to investigate middle and high school students’ pre- and post-summer scientific sensemaking skills and science attitudes. These constructs are conceptualized using definitions from the Science Learning Activation Lab, a research and design initiative to improve science achievement that characterizes learners based on science practices, knowledge, and malleable dispositions (Dorph, Cannady, & Schunn, 2016).

Current conceptions of science learning argue that students should not only learn scientific practices that align with skills that scientists use in real-world contexts; they should also develop and use these practices to make sense of scientific content knowledge (e.g., National Research Council, 2012; OECD, 2017). Scientific sensemaking practices include asking empirical questions, designing experiments, and finding mechanistic explanations for scientific phenomena (Apedoe & Ford, 2010; Lehrer, Schauble, & Petrosino, 2001). These skills provide the foundation for scientific thinking that helps learners purposefully build and connect knowledge across scientific contexts (Berland et al., 2015; Kapon, 2016). Because these skills are not content-specific, they support learners in engaging with novel contexts and understanding the content within them (Bathgate, Crowell, Cannady, Dorph, & Schunn, 2015; Cannady, Vincent-Ruz, Chung, & Schunn, 2019). Based on the Activation Lab framework, an “activated science learner” is someone who can apply scientific practices and existing content knowledge to understand novel scientific situations. As such, in the current study, we measure the application of general scientific sensemaking skills across various natural science contexts and separately measure specific scientific content knowledge. Because we are investigating optional summer science experiences that may vary widely in the content that they teach, we only examine pre- and post-summer changes in scientific sensemaking and use science content knowledge from the previous school year as a control measure of prior knowledge.

Learner dispositions are defined as the attitudes and beliefs the learner holds about themselves with regard to learning science content and doing science. We measure these on three dimensions, based on Expectancy–Value theory and social cognitive theory: students’ science competency beliefs (defined as one’s belief in their ability to do science successfully; Bandura, 2010; Wigfield & Eccles, 2000); students’ perceptions of science’s personal or societal value (Wigfield & Eccles, 2000); and fascination, a construct comprising aspects from curiosity (Loewenstein, 1994), interest in science in and out of school (Hidi & Renninger, 2006; Osborne, Simon, & Collins, 2010), and mastery goals for science content (Ames, 1992). An “activated science learner” will see the value in science, believe they can do science, and be fascinated by science. Together with science practices and knowledge, these characteristics should enable choices, motivation, and learning that support future science success (Dorph et al., 2016).

We operationalize SES as a composite measure of financial resources and how these resources are used to provide physical resorces and intangible support for students, based on the theorized mechanisms through which SES is thought to convey advantages for learning and motivation (detailed further in the “Analysis plan” section). We also use pre-summer dispositional measures and demographics information to determine the factors that predict participation in optional summer science experiences for high and low SES students, as well as post-summer dispositional and knowledge measures to see how summer experiences change attitudinal and knowledge outcomes. Finally, we also use the pre-summer demographics data for propensity score matching (PSM), a statistical method that allows us to calculate the likelihood that a student would have attended each type of optional summer learning experience (home-related, nature-related, STEM camps) and then match students with similar likelihoods to create comparison groups (i.e., students who participated in many summer experiences vs. those who attended few or no experiences) that are approximately equal in expectation before summer, allowing us to more closely estimate the causal relationship between participation in these experiences and post-summer science attitudes and skills.

Method

Participants

The ALES14 and ALES15 datasets included 2252 6th to 9th grade middle and high school students as a source of data. Beginning with a large dataset is critical for both the OLS and PSM analyses reported in the current study since these analyses can exclude many participants, either because participants are missing data on some of the variables used in the multiple regression models or because they do not have a matched propensity participant. Missing data were largely from demographics information (which were replaced when possible with data from other timepoints) or scientific sensemaking scores because these were collected on a different day than the other measures. Further, most data loss in the current analyses was due to the lack of a matched propensity participant rather than missing data. Students were recruited from 19 public schools (23 7th grade, and 32 9th grade classes) in Pittsburgh, Pennsylvania and four public schools (22 7th grade and 5 9th grade classes) from the Bay Area of California. Urban schools in a variety of school configurations with a broad range of races, ethnicities, and socioeconomic statuses were selected for recruitment. Parents provided written consent for their children to participate.

The sample at the post-summer data collection consisted of 29% 6th graders (mean age = 11.6, SD = 0.73), 40% 7th graders (mean age = 12.4, SD = 0.62), and 24% 9th graders (mean age = 14.3, SD = 0.46). Some students were absent on testing days or unable to complete all measures, so sample size varies across analyses and therefore are specified for each analysis in the Results section. Demographics were collected at both pre-summer and post-summer times; when a student’s reported information did not match between the time points, then the post-summer information was used. Forty-nine percent of students were White, 35% were Black or African-American, 7% were Asian, 5% were Indian or Middle Eastern, 6% were Native American or Pacific Islander, 10% were Hispanic, Latinx, or Mexican, and 15% did not report their race or ethnicity. Based on self-reported gender, 46% were female, 48% were male, 3% preferred not to answer, and 4% did not report their gender.

Materials

Measures included in the dataset have been extensively validated with iterative evaluation and improvements using a combination of cognitive interviews, exploratory and confirmatory factor analyses, and expert review. Additionally, Item-Response Theory analyses have also been conducted to ensure that scale means could be used and that items were equally valid across participants’ genders, race/ethnicity, and parental education levels, which is particularly important for analyses of differential change by SES. Basic psychometric information for the sample are presented as part of each measure description; more detailed factor analysis and IRT reports for each measure are available at http://activationlab.org/tools and in previous publications (Bathgate et al., 2015; Bathgate & Schunn, 2017; Dorph et al., 2016; Lin & Schunn, 2016; Sha et al., 2015; Sha, Schunn, Bathgate, & Ben-Eliyahu, 2016; Vincent-Ruz & Schunn, 2017). Because the Optional Summer Science Experiences survey is at the center of the current study and has not been described in prior publications, additional psychometric details for its three subscales are presented.

As an overview, the measures can be conceptually grouped into three treatment variables related to summer science learning experiences (home-related experiences, nature-related experiences, and STEM camp experiences), three attitude outcome measures (post fascination, post values, post competency beliefs), one science skill outcome measure (post scientific sensemaking), eight child-characteristic covariates used in the linear regressions and propensity score models (pre fascination, pre values, pre competency beliefs, pre science content knowledge, pre scientific sensemaking, grade, gender, minority status), and three additional context covariates used in the propensity score models (home support, family support, and teacher).

Optional summer science experiences

This survey asked about the frequency of participation in the immediately prior summer for three groups of common optional summer science-related experiences (see Table 1), building on results found with a similar informal science experiences survey that examined cumulative rather than recent experiences (Lin & Schunn, 2016). Six home-related items (α = .80) asked about experiences that took place in or near the home (e.g., “I did science experiments or activities at home”). Three nature-related items (α = .61) asked about activities that occurred outside in nature (e.g., “I collected rocks, butterflies, bugs, or other things in nature”). Three STEM camp items (α = .81) asked about the frequency of attending summer science, making/engineering, or computer programming camps (e.g., “I went to a science camp”). Students recorded the extent of their participation in each activity using a 4-point Likert scale (Never, Once, A Few Times, Many Times for the Home- and Nature-Related items, Never, One Day, A Few Days, Many Days for the STEM Camp items). The home-related, nature-related, and STEM camp items were averaged separately to create a home-related, nature-related, and STEM camps score for each participant.

Table 1 Survey items within each type of optional summer science experience

We conducted an exploratory principal components factor analysis with a Promax rotation on the survey items to confirm that the items were categorized correctly as home-related, nature-related, and STEM camps. The factor analysis found a two-factor solution in which all home-related and nature-related items loaded onto one factor, and all STEM camps items loaded onto a second factor. When the factor analysis was conducted with three fixed factors, then the items separated cleanly into home-related, nature-related, and STEM camps factors, and the third factor had a loading very close to the recommended 1.0 eigenvalue cutoff (see Table 2). Although the home-related and nature-related items appear similar based on the exploratory factor analysis, the three-factor analysis loading suggests that there may be some differences between the two categories, as expected theoretically; thus, our results use the three original categories of home-related, nature-related, and STEM camps.

Table 2 Obliquely rotated component loadings for the optional summer science experiences survey (three-factor analysis)

Fascination for science

The fascination survey measured students’ intrinsic attachment to science (Bathgate & Schunn, 2016, 2017; Dorph et al., 2016), which involves interest toward science (e.g., “After a really interesting science activity is over, I look for more information about it: YES!/yes/no/NO!”), affective disposition related to science (e.g., “In general, when I work on science I: love it/like it/don’t like it/hate it”), and desire to master science (e.g., “I want to know everything about science: YES!/yes/no/NO!”). The eight items (α = .86) were answered using a 4-item Likert scale, with responses depending on the question, as shown in the examples above. A mean fascination score was calculated for each participant at each time point.

Valuing science

The values survey measured the extrinsic importance that students place on science (Bathgate & Schunn, 2017; Dorph et al., 2016), which involved attitudes regarding science’s importance for their personal goals (e.g., “Knowing science helps me understand how the world works: All the time/Most of the time/Sometimes/Never”) and for the broader world (e.g., “I think scientists are the most important people in the world: YES!/yes/no/NO!”). Eight items (α = .82) were answered using a 4-item Likert scale, with responses depending on the item, and a mean values score was calculated for each participant at each time point.

Competency beliefs in science

The competency beliefs survey measured students’ beliefs about their ability to perform age-relevant science tasks (Dorph et al., 2016; Vincent-Ruz & Schunn, 2017). Items included competency beliefs about general scientific skills (e.g., “I think I am very good at doing experiments: YES!/yes/no/NO!”), completing class-related activities (e.g., “I can do the science activities I get in class: All the time/Most of the time/Half of the time/Rarely”), and completing activities not related to class (e.g., “If I went to a science museum, I could figure out what is being shown in: all areas/most areas/a few areas/none of it”). Students responded to eight items (α = .80) with a 4-point Likert scale, with responses varying based on the item. A mean competency beliefs score was calculated for each participant at each time point.

Scientific sensemaking

The scientific sensemaking (SSM) assessment measured students’ ability to apply scientific practices to make sense of natural phenomena. The same attitudinal measures can be meaningfully used multiple times in a longitudinal study, but skill measures typically require that alternative forms of the assessment be given at different time points to avoid large test-retest effects; indeed, pilot testing revealed poor psychometric performance at the second time point when the same sensemaking assessment was used in a pre-post design. Thus, several forms of a multiple-choice assessment were created that measured students’ ability to use scientific practices. Each assessment involved a different phenomenon to investigate but had closely matched questions to capture the same scientific practices. The specific scientific practices being assessed included generating scientific questions, designing relevant experiments to test a question, weighing potential mechanisms for a phenomenon, justifying or refuting arguments using evidence, analyzing evidence, understanding information from a scientific model, and understanding the nature of science (e.g., recognizing why a scientist may change their explanation for a scientific phenomenon). Examples of each type of question are shown in Table 3 (see Vincent-Ruz & Schunn, 2017 for measure details).

Table 3 Examples of each scientific practice type in the scientific sensemaking assessments

The current dataset involved three different sensemaking assessments, each based in a different phenomenon: (1) understanding monkey behavior to improve animal conservation efforts (α = .75), (2) understanding eagle behavior (α = .82), and (3) understanding lung cancer and assessing potential treatments (α = .78). These topics were chosen to be compelling and approachable (based on topic-interest survey data and pilot testing), so that interest in the topic or with science more generally was not the primary driver of performance (Bathgate et al., 2015). Sufficient information about each topic was also provided within the assessments so that students with different levels of prior knowledge on the topic could still succeed on the assessments. Providing strong evidence of validity, scores on these measures were found to be highly predictive of how much science content students learned during a semester of science class regardless of grade-level (6th, 8th, or 9th grades), disciplines (life sciences or physical sciences), or type of instruction (textbook vs. hands-on instruction; teacher-centric vs. student-centric classroom discourse) (Cannady et al., 2019; Vincent-Ruz & Schunn, 2017). Further, similar student estimates were obtained from pure multiple-choice assessments compared to assessments involving open-ended responses scored with rubrics (Bathgate et al., 2015). The proportion of correct responses were recorded for each participant at each time point. Since we focus on relative post-summer performance controlling for pre-summer performance, an exact equivalence of scores across assessment forms was not needed for analysis.

Science content knowledge

A content knowledge test, measuring how much the students had learned by the end of the previous school year, was used to characterize relative levels of students’ prior science knowledge within each classroom for propensity score matching purposes. Because curricula necessarily differed across grades and often differed between teachers and schools, each classroom was given an individual assessment that aligned with their specific curriculum. The tests focused on big ideas rather than basic facts. Each test consisted of 18 multiple choice questions (mean α = .70) drawn from various research assessment banks, such as TIMSS (Mullis, Martin, Gonzalez, & Chrostowski, 2004), AAAS (Laugksch & Spargo, 1996), and MOSART (Sadler et al., 2009) (e.g., “What is the primary energy source that drives all weather events, including precipitation, hurricanes, and tornadoes? (a) the Sun, (b) the Moon, (c) Earth’s gravity, or (d) Earth’s rotation”). Test bank items were individually chosen for each class to match the class’s covered content, based on a topics survey completed by the teacher at the beginning of the semester. The teachers also later verified that the sampled questions covered the content that had been taught in their curriculum. Test scores were standardized at each time point by subtracting the relevant test form mean and dividing by the relevant test form standard deviation to create comparable scores across classrooms and grades.

Home resources for science learning

The seven-item survey (α = .73) measured students’ access to resources at home that can support science learning (Bathgate et al., 2015; Dorph et al., 2016; Vincent-Ruz & Schunn, 2017) and can be taken as an indicator of socio-economic status of particular relevance to science learning (i.e., family financial resources may be of little value to student learning if not applied to support learning). Students were asked how often various items were available to them using a 4-point Likert scale (Always, Most of the time, Rarely, Never). These items included access to different technologies (e.g., access to calculators, computers, internet, or E-readers), books (e.g., dictionaries or science books), and study locations (e.g., a study or homework area). A mean Home Resources score was calculated for each participant.

Family support for learning

The five-item survey (α = .78) measured the level of support for learning that students received from their families (Sha et al., 2016; Vincent-Ruz & Schunn, 2017). Students rated their agreement to statements about their family members’ attitudes toward learning (e.g., “My learning in school is important to someone in my family”), knowledge (e.g., “When I work on homework at home, I have someone who can help me with it if I need help”), and involvement in learning activities (e.g., “Someone in my family takes me to places where I can learn new things”). Ratings used a 4-point Likert scale (YES!, yes, no, NO!). A mean Family Support score was calculated for each participant.

Demographics

The survey asked participants about their grade, gender, race/ethnicity, mother’s level of education, mother’s current occupation, father’s level of education, and father’s current occupation. From the race and ethnicity data, a binary Minority Status variable was calculated (coded as 0 for White and Asian students, 1 for Black/African-American, Indian/Middle-Eastern, Native American/Pacific Islander, and Hispanic/Latinx/Mexican). Parental education level (did not graduate high school, graduated high school, went to but did not graduate from college, graduated from college, post-college education) and occupation (unemployed, non-STEM job, STEM-related job, health-related mid or tech job, STEM mid or tech job, health professional, non-health STEM professional) were used as additional indicators of socio-economic status (see the “Analysis plan” section for details).

Procedure

Surveys were administered to students during the school year prior to the summer experiences (pre-test measures and covariates) and during the school year after the summer experiences (post-test measures). As part of the larger longitudinal ALES2014/ALES2015 study, surveys were administered in small blocks. Attitudinal and skill measures were collected multiple times (we use the measures closest in time to the summer), and summer experience measures were collected only once. Demographic information was collected before and after summer (prioritizing after summer information if data were not consistent between time points, such as changes in parental occupation) at the end of a block to avoid negative stereotype threat effects on attitudes or ability measures. Figure 1 presents the variables used and obtained from pre and post summer time points.

Fig. 1

Timeline of pre and post measure data collection. Italics denote post measures

Analysis plan

Our analyses focused on three research questions: whether there were SES differences in participation in optional science experiences during the summer and science attitudes and skills after summer; which child and family characteristics influenced participation in these optional experiences for low and high SES students; and how these optional experiences affected post-summer science attitudes and skills, controlling for self-selection biases. We investigated these questions using a combination of ANCOVAs, ANOVAs, linear regressions, and propensity score matching. An overview of analyses is presented here, and more specific details for each analysis are provided in the “Results” section below.

Because we were interested in differences based on SES, an SES score was calculated for each student by combining the measures of Home Resources, Family Support, Parental Education (measured as the highest level of education between the student’s mother and father), and Parental Occupation (measured as the highest job level between the student’s mother and father). Income was not included as part of our SES measure because we were primarily interested in the ways in which families spent their income to support learning and motivation, which common measures of income (e.g., exact income, National School Lunch Program eligibility) would not be able to determine. Further, our participants were sampled across areas with considerable differences in cost of living, so a pure measure of income would not produce a directly comparable estimate of wealth across our participants. Instead, we included measures of common mechanisms through which higher income is thought to convey advantages—by providing physical resources used for learning activities and more time for family to be directly involved with children’s learning activities. We also include the non-economic aspects of SES: access to the knowledge that may be needed to directly help students with learning via parental education and occupation. The use of a composite SES measure also increased the chances that our SES categorization would be accurate, as some students may not have known their parents’ exact level of education or occupation but would still know their perceived level of home resources and family support.

We calculated a z-score for each of the Home Resources and Family Support measures collected before and after summer and the Parental Education and Parental Occupation measures after summer for six total z-scores. These z-scores were averaged together to create a mean SES score. A median split was then used to categorize participants into higher SES and lower SES groups and retain power for analyses. Mean raw scores for Home Resources and Family Support pre- and post-summer and Parental Education and Parental Occupation post-summer for each SES group are shown in the left half of Table 4. In general, there were large mean differences between the two groups on the SES measures. The largest differences occurred for parental education level; on average, the higher SES group reported parental education levels between college and post-graduate degrees, whereas the lower SES group reported levels between finishing high school or some college. For parental occupation, the majority of participants reported that their parents held non-STEM jobs, but differences were still seen between the higher SES and lower SES groups in average parental occupation. The lower SES group and higher SES group were similar in valence for Home Resources and Family Support but differed in strength (with the lower SES group responding “high” on average compared to the higher SES group’s “very high” responses). Although our groups appear comparable on several SES measures to what is typically defined as low-mid vs. high SES backgrounds, particularly for parental education, we consider our groups to be relative measures of SES and as such refer to our groups as “lower SES” and “higher SES.”

Table 4 Means and standard deviations for the home resources, family support, parent education, parent occupation, and optional summer science experiences surveys for the higher SES and lower SES groups, and Cohen’s d for the differences between higher and lower SES groups

To address the first research question on differential participation in optional summer science experiences and post-summer attitudes and skills based on SES, we ran one-way ANOVAs on reported participation in home-related experiences, nature-related experiences, and STEM camps, and one-way ANCOVAs on each post-summer attitudinal and skill measure, controlling for each measure’s respective pre-summer score. The median split of higher and lower SES was used as a between-subjects variable.

To investigate how child and family characteristics predicted participation in optional summer science experiences and whether predictors differed by SES, we conducted linear regression analyses separately for higher SES and lower SES participants. Because the study focuses on out-of-school activities during the summer, it is unlikely that the students’ classes and schools had a strong effect on any of the study’s outcome measures. It is possible that peers or friends may attend similar camps or out of school activities, but the effect would likely be across individuals rather than across classrooms or schools, which we cannot account for with the available data. Thus, hierarchical models nesting by classroom or school were not used and these nuisance effects were not considered in the analyses. Instead, we used multiple linear regression to predict participation in each type of summer experience (measured as a continuous variable) based on the child and family characteristic measures. We also tested whether the relative size of the coefficients predicting participation in summer experiences was statistically significantly different between the higher SES and lower SES groups by calculating a z-score to represent the difference between low SES and high SES coefficients, where $z=\frac{B_1-{B}_2}{\sqrt{SE{\left({B}_1\right)}^2+SE{\left({B}_2\right)}^2}}$ and z-scores above 1.96 or below − 1.96 were considered statistically significant at the level of p < .05 (Clogg, Petkova, & Haritou, 1995).

To address the third research question regarding the impact of the three types of optional summer science experiences on students’ attitudes and abilities, we first conducted multiple linear regressions to predict science attitudes and skills after the summer experiences, controlling for pre-summer levels (i.e., assessing the change) and student demographics. Each regression included the main predictors of participation in home-related experiences, nature-related experiences, and STEM camps experiences. Regressions were also done on the full sample, collapsed across the two SES groups to maintain power for propensity score matching and because we did not expect significant differences in the impact of optional summer science experiences based only on SES. Mean SES scores were instead included as a covariate in the OLS and propensity score matching models.

Because the investigated summer experiences were optional, participation likely involved some self-selection biases. These self-selection biases may be imperfectly adjusted with regular regression techniques, so we also conducted propensity score matching analyses to determine whether the OLS relationships remained after more carefully controlling for confounds associated with the likelihood of experiencing each type of informal learning in the summer. A propensity score, representing the likelihood that the participant would participate in optional summer experiences (i.e., be categorized as a High-Participation student vs. a Low-Participation student), was calculated for each participant. Separate propensity scores were calculated for each type of experience (described in detail in the next paragraph). Next, each High-Participation participant was matched to a Low-Participation participant with a similar propensity score, producing sets of matched High-Participation/Low-Participation pairs for each of the three types of experiences. These matched participants were then used in a second set of analyses to test the effects of participation that fully controlled for likelihood of participating (i.e., participants without a match were excluded).

Propensity scores for each type of summer experience were calculated using logistic regression models. We transformed participation in summer experiences into a binary variable by categorizing participants as either High-Participation (i.e., those with mean participation scores of 2.5 or higher) or Low-Participation (i.e., those with mean participation scores of less than 2.5) for each type of summer experience. We then predicted High- vs. Low-Participation in each summer experience using three logistic regressions that included 11 covariates: two school-level covariates (the student’s grade and teacher during the spring semester) to account for possible teacher or peer influences; two family-level covariates (family support and home resources for learning) to account for the encouragement students may receive from their families to participate in optional science experiences; and seven student-level covariates (gender, minority status, and pre-test values in fascination, values, competency beliefs, science content knowledge, and scientific sensemaking) to account for students’ interest and ability to participate in optional summer experiences. This led to the following propensity score models (one for each type of experience), where RE indicates whether a student would be categorized as High-Participation (coded as 1) or Low-Participation (coded as 0) on a recent summer experience dimension:

Prob(RE) = β₀ + β₁(grade) + β₂(teacher) + β₃(family support) + β₄(home resources) + β₅(Gender) + β₆(minority status) + β₇(pre science fascination) + β₈(pre science values) + β₉(pre science competency beliefs) + β₁₀(pre science content knowledge) + β₁₁(pre scientific sensemaking ability)

Because propensity score matching requires participants to be matched to each other, there is a potential for a greater loss of data (and subsequently power) compared to the OLS regressions if many participants cannot be matched to another participant. Thus, the combination of linear regressions and propensity score matching allows us to test the robustness of results in case of power loss. In addition, it also allows us to determine which optional summer experiences may be most affected by self-selection biases (if there are large differences in findings between the linear regressions and propensity score matching), providing insight into possible factors that influence participation in optional summer science experiences.

Results

Does participation in optional summer science experiences and post-summer science attitudes and skills differ based on SES?

We ran three one-way ANOVAs on participation in home-related experiences, nature-related experiences, and STEM camps with SES (higher vs. lower) as a between-subjects variable. Consistent with prior studies, higher SES students reported participating in significantly more home-related experiences, F(1, 1176) = 21.7, p < .001, d = 0.27, nature-related experiences, F(1, 1219) = 31.6, p < .001, d = 0.32, and STEM camps, F(1, 1219) = 11.7, p = .001, d = 0.20, though each effect was small in size. For home-related and nature-related experiences, higher SES students on average reported attending some or many of these experiences, whereas lower SES students reported attending few of these. For STEM camps, both SES groups reported attending few camps, though it was slightly higher for Higher SES students (see right half of Table 4). Correlations between the continuous mean SES score and participation in each type of summer experience are also shown in Table 5 and correspond with the SES group results, with mean SES positively correlating with higher participation in all three types of optional science experiences.

Table 5 Correlations between mean SES, science attitudes, science skill, and reported participation in optional science experiences

ANCOVAs were also run on post-summer fascination, values, competency beliefs, and SSM scores, controlling for each measure’s respective pre-summer score, using SES (higher vs. lower) as a between-subjects variable. Mean scores and standard errors for each SES group are shown in Fig. 2. As expected, higher SES students showed significantly higher fascination, F(1, 748) = 14.0, p < .001, d = 0.36, values, F(1, 748) = 22.5, p < .001, d = 0.41, competency beliefs, F(1, 748) = 58.5, p < .001, d = 0.64, and SSM scores, F(1, 591) = 6.20, p = .01, d = 0.50, after summer than lower SES students, with the largest effect sizes for competency beliefs and SSM scores. Consistent with the SES group results, there were also significant positive correlations between mean SES score and all attitude and skill measures. Thus, higher SES participants appear to attend slightly more optional summer science experiences than lower SES participants and begin the school year with higher science motivation and sensemaking skills.

Fig. 2

End of summer means, standard errors, and Cohen’s d for science fascination, values, competency beliefs, and scientific sensemaking scores for the higher SES and lower SES groups, controlling for pre-summer differences

What child and family characteristics are associated with participation in optional summer science experiences for higher SES and lower SES students?

To build a model of factors that drive participation, we investigated how pre-fascination, pre-values, pre-competency beliefs, pre-scientific sensemaking, gender, grade, and minority status predicted participation in optional summer science experiences for higher SES and lower SES participants. We inputted the above factors into a multiple linear regression to predict participation in each type of summer experience (with participation measured as a continuous variable). Table 6 shows the standardized betas for the multiple linear regressions separately for Higher SES and Lower SES participants.

Table 6 Beta coefficients of the multiple linear regressions predicting participation in optional summer science experiences for higher SES and lower SES participants

Results are summarized in the left half of Figs 3 and 4 for higher SES and lower SES participants, respectively. For higher SES participants, the multiple linear regression models predicted a significant amount of variance in participation for home-related experiences, F(7, 331) = 12.8, p < .001, R² = .21, nature-related experiences, F(87, 348) = 8.48, p < .001, R² = .15, and STEM camps, F(7, 347) = 3.87, p < .001, R² = .07. The multiple linear regression models also predicted a significant amount of variance for lower SES participants for home-related experiences, F(7, 251) = 5.30, p < .001, R² = .13, nature-related experiences, F(7, 261) = 2.80, p = .008, R² = .07, and STEM camps, F(7, 264) = 2.11, p = .04, R² = .05.

Fig. 3

Summary of characteristics that influence optional summer science experience participation (from the OLS models of participation) and how participation influences post-summer outcomes (from the PSM models of impact) for Higher SES participants

Fig. 4

Summary of characteristics that influence optional summer science experience participation (from the OLS models of participation) and how participation influences post-summer outcomes (from the PSM models of impact) for Lower SES participants

For home-related experiences, higher pre-fascination and being male significantly predicted participation for Higher SES participants, whereas higher pre-competency beliefs and being male predicted participation for lower SES participants. For nature-related experiences, higher pre-Fascination, being female, and being a non-minority were significant predictors of participation for higher SES participants. Higher pre-competency beliefs were again a significant predictor of participation for lower SES participants. For STEM camps, higher pre-competency beliefs and being male were predictive of participation for higher SES participants. Interestingly, lower pre-scientific sensemaking was a significant predictor for lower SES participants. Significant between-SES differences only emerged for pre-fascination for home-related and nature-related experiences; prior interest in science appeared significantly more important in determining participation in these activities for higher SES participants compared to lower SES participants.

What are the effects of different types of optional summer science experiences on science attitudes and skill, controlling for self-selection biases?

OLS regressions

To determine the impact of the three types of optional summer science experiences on students’ attitudes and abilities, we conducted four regressions to predict the outcomes of fascination, values, competency beliefs, and scientific sensemaking after the summer experiences, controlling for pre-summer levels. Participation in home-related experiences, nature-related experiences, and STEM camps experiences (as continuous variables) were used to predict outcomes, with gender, grade, minority status, and pre-scientific sensemaking included as demographics and ability covariates. The regression model for the outcome of scientific sensemaking also included a second scientific sensemaking pre-score from earlier in the prior year to further reduce noise; however, similar effects are observed with and without this addition.

Standardized betas for regressions are shown in the “Full OLS” rows of Table 7. Home-related experiences during the summer significantly and positively predicted post-summer Fascination, β = .24, t(583) = 5.69, p < .001, values, β = .13, t(583) = 3.13, p = .002, and competency beliefs, β = .17, t(583) = 4.05, p < .001. There was no effect on scientific sensemaking, β = − .01, t(310) = − .24, p = .81. In contrast, STEM camps showed no influence on any of the attitudinal measures, but significantly and negatively predicted later scientific sensemaking, β = − .15, t(310) = − 3.33, p = .001. Nature-related experiences did not significantly predict any outcome measure. Thus, home-related science experiences appear to have the most positive impact on how interested students are in science, how important they find science, and how well they think they are able to do science, whereas STEM camps appear to have a significant negative impact on students’ general scientific abilities.

Table 7 Beta coefficients for the full OLS and matching regression models (without and then with the addition of the covariates)

Propensity score matching

The investigated summer experiences were optional and participation involved some self-selection biases, as shown by our results on the characteristics predicting participation in optional summer experiences. Thus, we also conducted propensity score matching analyses (using the procedures described in the “Analysis plan” section) to control for the likelihood of experiencing each type of informal learning in the summer.

Participants who were missing data for any of the covariates were removed through list-wise deletion from the sample to ensure maximal correspondence in the matching procedure. The High-Participation group and Low-Participation group were then matched using nearest neighbor matching without replacement and a caliper of 0.2; that is, High-Participation participants were matched with the Low-Participation participant who had the closest propensity score to them, with a maximum allowed difference of 0.2 between matched propensity scores, and each participant could only be matched once. After list-wise deletion and matching, there were 108 High-Participation and 108 Low-Participation students for home-related experiences, 118 High-Participation and 118 Low-Participation students for nature-related experiences, and 50 High-Participation and 50 Low-Participation students for STEM camp experiences. Power analyses show that these sample sizes can detect minimum effect sizes that are low to medium in size at .80 power: .38 for home-related experiences, .37 for nature-related experiences, and .57 for STEM camps experiences.

Table 8 shows the mean differences between the High-Participation and Low-Participation groups for each covariate before and after matching, with the test for significant differences based on independent samples t tests. The table also shows the mean propensity score for the High-Participation and Low-Participation groups after matching. The two groups differed significantly along many dimensions before matching (especially on pre-summer science attitudes, gender, and our SES proxies), suggesting high levels of self-selection into groups, but were closely equal on all dimensions after matching (even though the matching procedure only attempts to match on overall propensity). Because a few small (but all non-significant) differences remained, analyses were conducted on the matched pairs with and without additional regressors.

Table 8 Mean differences between matching model covariates before and after matching

Results are conceptually summarized in the right half of Figs. 3 and 4 for higher and lower SES participants, respectively. Table 7 (“Match” row) shows standardized betas for each outcome measure from the matching models. Figure 5a–c shows the detailed effects on post-test values of fascination, values, competency beliefs, and scientific sensemaking for High-Participation and Low-Participation students in the matched participation set, based on the home-related experiences model, the nature-related experiences model, and the STEM camps model. Consistent with the OLS model, home-related summer experiences were significantly related to post-test fascination, values, and competency beliefs, with no relationship to post-test scientific sensemaking, as was see in the OLS model. This suggests that the positive effects shown by home-related experiences were robust even after carefully controlling for initial differences between participants. Interestingly, nature-related experiences now were positively related to fascination, competency beliefs, and scientific sensemaking, whereas STEM camps were no longer related to post-scientific sensemaking (even though the relationship was still negative, it did not reach statistical significance). Complex self-selection biases may have been strongest for nature-related and STEM camp experiences; when these biases were taken into account, nature-related experiences were revealed to positively predict students’ attitudes toward science, whereas STEM camp experiences lost their predictive power.

Fig. 5

For fascination, values, competency beliefs, and scientific sensemaking, post summer scores separately for high-participation vs. low-participation matched participants in a home-related experiences, b nature-related experiences, and c STEM camp experiences

To further test the robustness of the findings, we also ran another regression on the matched sample with the propensity model covariates (i.e., grade, teacher, family support, home resources, gender, minority status, pre fascination, pre values, pre competency beliefs, pre science content knowledge, and pre scientific sensemaking) included in the regression. Table 7 (“Plus covariates” row) shows the standardized betas of this regression model for each type of summer experience. As with the Match sample results, home-related experiences positively predicted fascination, values, and competency beliefs, and nature-related experiences positively predicted fascination, competency beliefs, and scientific sensemaking. The new model for STEM camps again differed from the OLS and Match models, with STEM camp participation now positively predicting fascination, values, and competency beliefs. The differences seen between the three STEM camp models suggest that these results may be particularly unstable and prone to biases.

Discussion

In the current study, we investigated whether SES disparities were present in participation in optional science experiences during the summer and in science attitudes and skill at the start of the school year, whether different child and family characteristics predicted participation in these summer experiences based on SES, and the impact on post-summer science attitudes and skills after participating in such experiences. On average, lower SES students reported attending slightly fewer optional summer science experiences. When choosing whether to attend these summer experiences, lower and higher SES students appeared to take different factors into account. Fascination with science showed especially large and consistent differences in predicting participation in home-related and nature-related experiences, being significantly more predictive of participation for higher SES students than lower SES students. In contrast, competency beliefs in science were more important for predicting participation in home-related and nature-related experiences for lower SES students. STEM camps diverged from this pattern, such that higher SES students with higher competency beliefs were more likely to participate, whereas lower SES students with lower scientific sensemaking skills were more likely to attend these experiences. Gender also played a consistent role in predicting participation: boys participated more in home-related experiences overall, higher SES girls participated in more nature-related experiences, and higher SES boys participated in more STEM camps.

Higher participation in most of these optional experiences also had positive effects on science attitudes and skills: higher participation in home-related experiences predicted higher fascination, values, and competency beliefs after summer, and higher participation in nature-related experiences predicted higher fascination, competency beliefs, and scientific sensemaking after controlling for self-selection biases. However, higher participation in STEM camps inconsistently predicted higher science attitudes and lower sensemaking after summer. These findings provide details about potential pathways utilizing informal summer science experiences for improving specific science attitudes and general science skill in both lower and higher SES students.

Characteristics influencing participation in optional summer science experiences

Our results support prior findings showing an association between science attitudes and science participation. A greater interest in science can predict students’ preferences to participate in optional science experiences (Bathgate & Schunn, 2016), and students with higher science competency beliefs take more science courses in high school and are more likely to pursue science (Simpkins et al., 2006). Interest and competence beliefs can also support plans to engage in science activities over other academic subjects in the present and future (Sha et al., 2016). We similarly find that interest (measured as Fascination) and competency beliefs are most important in predicting participation in science experiences.

Our findings extend upon prior studies to suggest that different factors may be prioritized by high SES and low SES students when deciding whether to participate in these optional science experiences; in particular, fascination with science is a larger factor for high SES students, whereas competency beliefs may be more important for low SES students. These findings reflect the importance of competency beliefs seen within underrepresented groups in STEM (e.g., Alhaddab & Alnatheer, 2015; Cousins, 2007; Vincent-Ruz & Schunn, 2017). For example, Wiederkehr, Darnon, Chazal, Guimond, and Martinot (2015) found that lower SES 8th and 9th grade students had lower self-efficacy than their higher SES counterparts, and that self-efficacy mediated the relationship between SES and school grades. High competency beliefs also appear to be an additional requirement for middle school girls to achieve equal science content learning gains from science activities compared to boys (Vincent-Ruz & Schunn, 2017).

We further show that these relationships may depend on the type of optional summer science activity, as these relationships hold most true for home-related and nature-related experiences, and less consistently for STEM camps. This differential pattern by experience type may suggest that students are more likely to seek out independently-motivated experiences that would be found at home or in nature when they are initially interested in or perceive themselves as capable in science, compared to STEM camps that are more structured and where parent involvement may drive participation. High and low SES students may also differently perceive STEM camps. Higher SES students with high competency beliefs were more likely to participate in STEM camps, suggesting that they may see camps as an enrichment opportunity on top of their usual science experiences. Meanwhile, lower SSM scores predicted participation for lower SES students, meaning that they may see camps as more of a remedial solution or alternative learning opportunity for students who struggle with science during school. Competency beliefs may also be important for participation, especially for low SES students, because students are more likely to take part in independent experiences without adult supervision (so belief that they have the knowledge to successfully complete these experiences may be particularly important), or because students’ beliefs about their abilities may influence whether parents or other supervisors are willing to invest in activities that need their involvement (e.g., their time, money, or approval). Our findings provide information about potential pathways to increase participation in optional science experiences and subsequent science attitudes and skills for lower SES students (beyond increasing socioeconomic resources).

Many studies have also found gender differences in favor of boys for participating in optional science experiences, pursuing further science education, and aspiring toward science careers (e.g., Dasgupta & Stout, 2014; Lane, Goh, & Driver-Linn, 2012; Nosek et al., 2009). Some of these differences can be explained by people’s implicit gender stereotypes that associate science with males more than females (Lane et al., 2012; Nosek et al., 2009), which may also contribute to women receiving less support from parents and peers and having fewer female mentors available in science fields (Bottia, Stearns, Mickelson, Moller, & Valentino, 2015; Dasgupta & Stout, 2014). Our findings suggest that gender differences are not uniform across all types of experiences, as being male significantly predicted participation in home-related and STEM camp experiences, whereas being female predicted participation in nature-related experiences; further, this sometimes depended on SES, such that the bias toward boys was seen across SES for home-related experiences, but the bias toward girls for nature-related experiences and boys for STEM camps was only seen in high SES participants. It may be that home-related and STEM camp experiences are more likely to involve adults or peers who display gender stereotypes that turn girls away from participating in such experiences. Meanwhile, less pressure may exist in nature-related experiences, as students can take part in these experiences on their own, making them particularly appealing to those who may feel excluded from other types of activities. These stereotypes may also be stronger in high SES communities.

Effects of participation on science attitudes and skill

Many of our results on the effects of participation in optional summer science experiences are congruent with prior findings on the effects of science participation on science attitudes. Many studies have shown that optional home-related science experiences (e.g., Dabney et al., 2012; Henriksen et al., 2015; Simpkins et al., 2006; Uitto et al., 2006), and nature-related experiences (e.g., Henriksen et al., 2015; Uitto et al., 2006) can positively affect science interest, consistent with the higher fascination, values, and competency beliefs shown by high-participation home-related students and the higher fascination and competency beliefs shown by high-participation nature-related students in the current study.

Our findings that higher participation in nature-related summer experiences predicted higher scientific sensemaking also adds support to prior studies in which nature programs led to higher science achievement (e.g., Berezowitz et al., 2015; Education & Roundtable, 2000; Klemmer et al., 2005). Interviews also reveal that scientists often consider early experiences with nature to be an initial step toward pursuing science careers (Sadler, Burgin, McKinney, & Ponjuan, 2010), perhaps suggesting that nature-related experiences encourage the development of skills that are aligned with actual scientific practice. In the past, higher science achievement has generally been seen after specific structured nature-related experiences (e.g., community or school-based projects). The current study investigated nature-related activities more broadly, including questions about being in nature, collecting items from nature, and gardening. This suggests that independent nature activities may also be able to contribute to science achievement, though more research is needed into the extent to which structure is needed.

Surprisingly, there was some evidence that STEM camp experiences may lead to lower scientific sensemaking, though it was inconsistent across models. This contradicts some studies showing improvements in attitudes and achievement after particular camps that were investigated in-depth (e.g., Bischoff et al., 2008; Gibson & Chase, 2002; Knox et al., 2003; Markowitz, 2004). However, broader investigations of a wider range of out-of-school experiences have found that attending a greater number of optional science experiences, including science camps, can lead to lower science motivation (Fortus & Vedder-Weiss, 2014) and science scores (e.g., Lin & Schunn, 2016; Suter, 2016), and that these negative effects appear to be strongest for more structured experiences that take place in or near schools, though they were also found for home-related experiences after controlling for differences in the likelihood to attend experiences (Liu & Schunn, 2018). STEM camp experiences may share more factors with these structured experiences than either home-related or nature-related experiences, leading to similar negative effects on scientific sensemaking. These results could also indicate that previously studied camps were particularly high in quality, whereas the current study’s sample is more varied and representative of the average camp experience. Regardless, more research is needed into why these experiences are more likely to be detrimental for science skills, especially given findings that lower SES students who enter with lower sensemaking skills are also more likely to attend these experiences.

Limitations of the current study

The current study furthers our understanding of how characteristics affecting participation depend on SES and the type of optional summer science experience, as well as how different types of summer experiences have unique effects on science attitudes and skill. However, it was limited in its ability to describe the trajectory of change of science attitudes and skill before, during, and after the summer. A prior study on optional science experiences during the school year suggests that high participation in these experiences can both increase and maintain positive attitudes toward science but can also actively decrease or prevent growth in science knowledge and skills (Liu & Schunn, 2018). Optional summer science experiences may show similar patterns of change, but it is difficult to determine with our current data; although the study was longitudinal, students’ attitudes and skill scores were lower immediately before summer compared to prior time points throughout the school year, which may make any changes during and after summer experiences look particularly pronounced. More longitudinal data are needed on science participation and its relationship to attitudes and skill to determine whether these lower pre-summer scores are an artifact of the current study’s sample or a robust pattern and how summer activities impact existing trajectories of science attitudes and skill.

We chose to measure SES without a direct measure of income, which may limit our ability to accurately measure the SES of our participants. However, as discussed in our analysis sample, the effectiveness of income as an SES measure was reduced in our sample because our participants were from areas with significantly different costs of living, and we were primarily interested in how income was used to support science learning. Thus, we combined conventional measures of SES (parental education, occupation) with measures of the way income was spent (home resources, family support). Further, we chose to analyze the factors predicting participation for lower SES and higher SES participants by using a median split of our SES measure. Although the median split allowed us to simplify our analyses and was appropriate for our relative measure of SES, the median split of our participants into two categories may have also failed to capture the full variability of SES in our sample and prevent us from directly comparing our results to studies that use absolute measures of SES. Future studies may want to investigate these questions using a continuous measure of SES that includes exact income to get a more fine-grained look at how factors predicting participation change based on SES.

Our study measured a relatively broad range of optional summer science activities, comprised of various activities at home, in nature, and at camps, and found that lower SES students generally participated in fewer optional summer experiences compared to higher SES students. However, this may reflect limitations in our choices of informal science activities. Our measures may primarily represent activities that are found in more Western-centric communities and ignore other informal activities that may be more common in non-dominant communities and still meaningfully contribute to science attitudes and skill. For example, storytelling at dinnertime within the home may be a home-related activity that helps children think about the world and develop theory-building skills involved in scientific sensemaking (Ochs, Taylor, Rudolph, & Smith, 1992). Nature-related experiences could also be expanded to include other ways of thinking about nature, such as those found in indigenous communities (Bang & Marin, 2015). Our sample is also taken from two largely urban areas, and there may be differences seen in science participation and predictors for more rural communities (Hill et al., 2018). Future studies will want to consider a wider range of activities and settings to gain a more representative look at the participation and impact of such activities.

Our findings suggest one potential pathway to increase participation for higher and lower SES students through fascination and competency beliefs, respectively. However, we note that our models predicting participation in optional summer science experiences for STEM camps and lower SES participants explained a relatively small amount of variance. This suggests that other unmeasured variables likely factor into participation for STEM camps and lower SES students’ decisions to attend these experiences. Participation in nature-related and STEM camp experiences may be further complicated by issues of self-selection that may not be solved only by increasing student attitudes toward science. This is evidenced by the differing results when analyzing nature-related and STEM samp effects with OLS vs. PSM, meaning that differences in people’s initial propensity to participate were particularly large and not fully accounted for by the child and parent characteristics in the OLS models. Future studies will want to include a wider variety of predictors to investigate other pathways toward science participation in more detail. One example may be other child characteristics, such as students’ science identity, which has been found to be conceptually distinct from the constructs of fascination, values, and competency beliefs used in the current study, and to be a strong predictor of students’ science-related choices (Vincent-Ruz & Schunn, 2018). External factors, such as general access (e.g., greater time commitment, more parental involvement needed, cost), may also be a concern especially for STEM camps, as both higher and lower SES students reported going to an average of only few or no STEM Camp experiences.

We also acknowledge limitations in the response options for our measures of gender, parental education, and parental occupation which may have influenced our findings. Our gender and parental survey items assumed that students fit within a gender binary (boy or girl) and had a household with one woman and one man (parent or guardian), which excluded any students who are non-binary, have non-binary parents or guardians, or are from same-gendered households. Non-binary participants or participants with non-binary or same-gendered parents or guardians may have been a non-trivial part of our sample; however, because they were not given responses to represent themselves, we would not have accurate gender or parental data on these participants, which would influence our SES calculations and regression models. Further, our lack of representative options may have indirectly influenced these participants’ science attitudes if our options implicitly made them feel that they did not belong in science. We strongly recommend that future studies include more inclusive demographics options when investigating these topics.

Improving access to all optional summer science experiences may not be useful without understanding the specific factors that contribute to increases in attitudes, as opposed to decreases in science skill. That is, while increasing access to home-related or nature-related experiences may be positive for students, increasing participation in STEM camps may be counterproductive given the lower SSM scores of High-Participation STEM camp students. The current study uses a coarse measurement of optional summer science experiences, asking about general participation so that students are not asked to recall details of distant events that are likely subject to memory biases. However, this also limits our ability to explore specific features of programs and whether certain features within each type of summer science experience relate to more impactful changes in science attitudes and skills. Future studies should investigate individual programs using more immediate methods (e.g., experience sampling methods) to determine the characteristics associated with home-related, nature-related, and STEM camp experiences that increase or decrease students’ attitudes and skill.

Conclusion

In conclusion, different factors appear to contribute to participation in optional summer science experiences for lower SES and higher SES students. Higher SES students were more likely to attend home-related and nature-related experiences when they were interested in science, whereas lower SES students were more likely to participate in these same experiences when they believed they could succeed in science. Our work provides a look at potential pathways through science fascination and competency beliefs that can be used to increase participation in optional science activities that may combat gaps seen in science attitudes and skill during middle and high school. More work is needed to determine other predictive factors of participation in a greater range of optional summer science activities, as well as the specific program features that benefit students’ attitudes, as opposed to hurting their science skills, to ensure that students of all SES backgrounds can succeed in science.