- Research
- Open access
- Published:
Classroom, media and church: explaining the achievement differences in civic knowledge in the bilingual school system of Estonia
Large-scale Assessments in Education volume 9, Article number: 3 (2021)
Abstract
This study investigates civic and citizenship education in a unique post-Communist context–in the bilingual education system of Estonia. Estonia continues to have a bilingual school system where there are Estonian and Russian language schools in parallel. While Estonian language school students are ranked very high in international comparisons, there is a significant difference between the achievement of Estonian and Russian language school students. We claim that this minority achievement gap in the performance of civic and citizenship knowledge is in addition to family background characteristics explained by behavioral and attitudinal factors that are moderated by the school language. Behavioral and attitudinal independent variables that we consider relevant in our analysis are classroom climate, trust in various media channels, and students’ beliefs in the influence of religion. We rely on hierarchical modeling to capture the embedded data and aim to explain how the different layers (school- and student level) interact and impact civic knowledge. We show that an open classroom is beneficial to students and part of the gap can be explained by Russian school students’ lower involvement in such practices. The strength of the belief in the influence of religion, on the contrary, is hurting students, despite that the negative effect is smaller for minority students there is a higher aggregate negative effect of it and therefore it also contributes to the minority achievement gap. Media trust indicators explain the gap marginally while the high trust of social media hurts students’ civic knowledge scores–still more Russian school students trust social media more than Estonian school students.
Introduction
Contemporary society is especially demanding in terms of both education in general and education for social and political participation in particular. This study aims to investigate the opportunities and challenges of civics and citizenship education to provide the knowledge and practices for political participation in the bilingual education system of Estonia. Our ambition is a two-fold contribution to the literature. First, as Knowles and McCafferty-Wright (2015) have identified, there is a need for additional research focusing on contextualised notions of citizenship studies and civic and citizenship education in unique post-Communist context, often under-represented in related scholars. In exploring whether students from different ethnic groups in linguistically divided school system experience the potential influencers of civic knowledge differently and how such differences relate to civic knowledge outcome would be an enhancement in this line. Second, measures of student religiosity and its associations with civic and citizenship education have remained mostly unexplored (ibid. 22).
This study investigates civic and citizenship education in the research gap described above in a unique post-Communist context in the bilingual education system of Estonia. We aim to show how in addition to student background characteristics the differences in classroom climate, trust in media and the belief in the influence of religion interplay with the school level characteristic such as the language of school and how this interplay is associated with the minority achievement gap in civic and citizenship knowledge (CCK). Estonia became a host of a sizeable Russian minority after World War II. Since then, a large proportion of the population (1/3) is Russian speaking, and the schools with Estonian or Russian as their respective languages of instruction still coexist in the contemporary education system of Estonia. There have been attempts to integrate these two educational streams (e.g., regulation requiring 60 per cent Estonian-language use at the gymnasium level in Russian language schools since 2011 and various language immersion programs). However, the differences between the performance of children studying in schools offering Estonian-language instruction and those offering Russian-language instruction (hereinafter Russian and Estonian schools) are significant. Estonian school students, for instance, are one of the top performers in international comparisons (e.g., OECD’s Programme for International Student Assessment (PISA, Schleicher 2019) and International Civic and Citizenship Education Study (ICCS, Schultz et al. 2018a). However, a significant gap between the achievement scores of Estonian and Russian schools exist (hereinafter minority achievement gap). While lower achievement scores of immigrant background students are often associated with the additional difficulties of coping with the language, in Estonia (as in many other post-Soviet countries), the situation is different—the population in question are so-called historical immigrants; students and often their parents are born in countries of residence and study in their mother tongue. Besides, civic and citizenship education is provided as one of the general competences and as a separate subject in both language schools, and there are differences neither in curriculum nor the governance of these schools.
This dichotomy poses the question of why students in schools with different languages of instruction score so differently despite a uniform national curriculum? We aim to show that after controlling for individual-level background characteristics the minority achievement gap in CCK in Estonia is, to a large extent, moderated by cultural-ideological factors such as perceived school climate, trust in different media channels, and the role of religiosity in everyday decisions that, first, have remarkable between-group differences in Russian and Estonian language schools; second, these cultural-ideological effects have specificities stemming from the school language.
Our data originate from International Association for the Evaluation of Educational Achievement (IEA) second cycle of the Schulz et al. (2018a). Estonia was one of 24 participating countries. The data structure contains the student-level data–the CCK test, student questionnaire based assessment of background, attitudes and involvement; for the school-level we use school language to distinguish between Estonian and Russian schools. In Estonia, 3,255 students participated in the second cycle of ICCS, from both Estonian and Russian language schools (2,391 and 864, respectively). For the empirical technique, we employ hierarchical modelling, which handles well the embedded data structures such as educational data. However, modelling specificities due to missing data, scales of measurement and robustness allow multiple approaches that we discuss in the paper.
To accomplish our aim, we start with the short case description and posing the literature ‘enlightened’ hypothesis, illustrated by stylised facts from exploratory data analysis. Then we describe the data, impute missing data and give a discussion over statistical tools for the confirmatory analysis. After that, we continue with school random effect hierarchical modelling and end with a discussion.
Case specificities, exploratory analysis, and hypotheses
The education system in Estonia is characterised by comprehensive schooling; most schools are owned and operated by municipalities. Besides, the education system is single track, i.e., children often stay the same school for 12 years, schools follow the same curriculum, and transfer of students between schools is not common nor encouraged. Even in the Soviet period, Estonian schools were able to keep Estonian as a language of instruction. Russian schools were introduced to serve the educational needs of first-generation immigrants. Following the restoration of independence, the language-based segregation of the school system remained, and the schools with Estonian or Russian as their respective languages of instruction still coexist in today’s education system. Estonian government’s approach to Russian schools has been characterised as laissez-faire (Khavenson and Carnoy 2016), indicating the degree that Russian schools continued teaching mainly, if not entirely, in Russian. Also, reform initiatives to increase the share of the Estonian language classes in Russian schools remained slow. Not until 2007 did a secondary-school language ruling require 60 per cent of the curriculum of Russian upper secondary schools to be taught in Estonian by 2011 (Siiner 2014). The activity of reform was arguably accelerated by the increased attention brought by PISA tests (Khavenson and Carnoy 2016), in 2006 that revealed Estonian schools high scores in PISA but a considerable minority achievement gap between Estonian and Russian schools. The 60–40 requirement was implemented with a certain degree of flexibility and school-by-school, avoiding the resistance to Latvia’s similar 2004 law (Siiner 2014). The issue of Russian school progress has remained salient in parliamentary elections since 2015, and all mainstream parties have promised to improve the situation with Russian schools. However, despite the recent political emphasis, a concrete plan is still not yet agreed by today’s incumbency, and the minority achievement gap in Estonia revealed by several international comparisons is still worrisome.
The usual suspects in explaining differences in educational outcomes are differences in students’ socio-economic characteristics such as immigrant status, parental education, occupations, and the number of books at home. Students in Russian schools in Estonia do originate more frequently from immigrant families, and they have fewer books in their homes (Põder et al. 2017). However, the share of the whole gap these differences can explain is small (ibid.). Furthermore, pointed out above, in Estonian schools, the language spoken at home and in school by students with a migrant background generally differs, although, in the case of schools in which the language of instruction is Russian, immigrants primarily learn in their mother tongue. Thus, the language barrier that might explain the gap associated with immigrant status is not a relevant explanation for Russian schools in Estonia. Also, Lindemann (2013) has found certain differences in the professional positions of the parents of students in Estonian and Russian schools and Mikk et al. (2012) found a variance in the teaching methods at schools and the learning skills of students. However, again, these explain only a very small part of all the differences in academic results. Therefore, while in admitting the relevance of parental background in explaining the minority achievement gap in Estonia, in posing our hypotheses, we rely on behavioural and attitudinal explanations.
In exploring literature and our data, we illuminate specific patterns in the data that allow us to pose three hypotheses that might explain the achievement gap in CCK of Estonian and Russian school students.
Hypothesis 1: Classroom climate differs
Previous research on classroom climate has revealed a positive association between civic knowledge and openness to discussion and plurality of opinions (Campbell 2008; Knowles and McCafferty-Wright 2015). The assumption behind the association between political socialisation and participation is that a good citizen does not necessarily appear spontaneously, but rather, early socialisation within the family, and particularly at school, should aim to provide the tools that allow future citizens to become successfully involved in political life and to show good civic knowledge (Castillo et al. 2015). A more recent line of research in this field has highlighted that not only’what’ is transmitted by the schools (knowledge) but also’how’ this is transmitted, arguing that exposure to a democratic environment in the school can be as significant (or even more) as civic knowledge (ibid.). This focus has emphasised the role of the classroom climate as an environment that could have an impact on students’ political outcomes and knowledge (Alivernini and Manganelli 2011; Quintelier 2010). While the strength of an open classroom climate to support the civic development of youth has been documented in dozens of studies spanning decades (see overview in Knowles et al. 2018), many schools are often unable to provide such spaces for all students (Castro and Knowles 2017; Levinson 2010). While many factors contribute to this deficiency, the political and ideological nature of the schooling experience likely plays a role.
While schools in Estonia are considered amongst the most autonomous, there is still limited horizontal diversity; schools rely on the same central curriculum and schools are not very diverse by the teaching methods (OECD 2016). Whereas the rhetoric of curriculum reforms has been de-centralist, it has not always meant a real shift in power nor professional autonomy of teachers (Erss 2015). Recent reform agendas have tried to minimise the standardisation of the education system and to emphasise the child-centric, renewed and more active approach of teaching. However, the level and speed of these initiatives are often dependent on the motivation and the resources of the school heads, and there are some concerns (Klaas-Lang et al. 2014; Loogma et al. 2009) that Russian schools have had problems revising their educational practices due to language deficiencies and inadequate preparation of teachers for bilingual classrooms. Furthermore, Kiilo and Kutsar (2012) have shown that state language policy have had unintended consequences in the form of the feeling of powerlessness of the Russian-speaking educators, which in its turn restricts the balanced approach towards legitimization of the state language and causes deficiencies in self-efficacy and disempowerment. Thus, we may speculate that Russian schools have been less prone to innovate and remained more traditional in their approaches, having also impact on classroom climate. Therefore, in posing our hypothesis based on the students’ assessment of an open classroom climate, we treat it as a proxy of school developments in terms of the changing paradigm of education.
H1: Lack of school resources and low self-efficacy to adapt to a new educational paradigm has resulted in a more traditional, and less-open classroom climate in Russian schools that has negative consequences on their students’ CCK scores and increases the minority achievement gap.
We have one composite indicator from the ICCS international database to measure classroom climate–OPDISC, which indicates students’ perception of openness in classroom discussions. Higher values indicate perceptions of more openness. OPDISC is derived through the scaling of items (currently six categorical variables) by using item response modelling (Schulz et al. 2018a). We use this index in a standardised format. This index reflects answers to questions such as whether teachers encourage students to make up their minds and express their opinions (even if they differ from the class opinion), but also whether students bring up current political events for discussion in class and teachers present several sides of the issues when explaining them in class. Explorative insights from ICCS data show that in case of Russian-language schools the group mean is lower than zero (− 0.126), and the standard deviation is higher than one (1.041), indicating that Russian school students have relatively lower values for the OPDISC and more variance than Estonian school students (mean = 0.047 and standard error = 0.981). The overall statistic of the OPDISC variable is indicated in Table 1. Figure 1 reveals the data indicating mean scores by groups (vertical lines) and geometrical distribution of the data. We see group difference as Estonian school students have relatively more observations with the upper right part of Fig. 1
Hypothesis 2: Trust in media has non-linear effects
Previous research has shown that civic engagement and knowledge is beside to individual level characteristics such as social demographics variables strongly associated also with media use habits (Quintelier and Hooghe 2011; Romer et al. 2009; Pasek et al. 2006). More recently, social media has received attention and interest as people are making active use of the Internet in ways that may or may not contribute to civic engagement (Martens and Hobbs 2015). Whereas some scholars believe that civic engagement arises naturally from digital media use (Kahne et al. 2014), others believe that media literacy education is needed to provide the cognitive and social scaffolding that systematically supports civic engagement (Martens and Hobbs 2015). Part of this literature agrees that media usage and trust in media are important predictors of civic knowledge and engagement, but, first, the effect of traditional media and social media are different in that regard, and, second, the effect of the usage and trust in social media might be adverse. For instance, there is some evidence (Koršňáková and Carstens 2017) that among active social media users there is a group of students who are not very knowledgeable concerning the key dimensions of the ICCS framework (civic society and systems, principles, participation, and identities) and often not very tolerant, raising concerns about’disengaged’ or even’alienated’ youth. Furthermore, Fennema and Tillie (1999) argue that ethnic minorities might have different’ethnic’ social capital. They also enjoy watching their cultural television stations in their languages, and this may influence young people’s ethnic identity, feeling of belonging and internal group cohesion. However, ethnic television-watching can work both ways. On the one hand, as members of a minority group gradually perceive their ethnicity as more salient, one might argue that their political participation will increase in favour of their ethnic group (Miller et al. 1981). On the other hand, the counter-argument could be that ethnic television creates a closed community attitude, with less political participation. Thus, both the non-linearity and the type of media consumed seem to be key in understanding the associations between trust in media and CKK.
We see the question of usage and trust in media to be relevant in analysing the potential explanation behind the minority achievement gap in CCK in our case. In Estonia, the media enjoy a high degree of freedom. For quite a long time, Russian-speakers have been portrayed as quite monolithic and a rather inactive part of Estonia’s society. However, the recent studies (Integration Monitoring … 2017; Tammaru et al. 2017) have revealed that although clustered according to language, this group is indeed very heterogeneous in terms of its ethno-cultural background, citizenship, political activity and preferences, educational and socio-economic parameters, proficiency in the Estonian language, and media consumption. Furthermore, quite a large share of Russian speakers rely on Russian sources in their media consumption which are potential risks in biased news or disinformation (CIMU-SEE, 2018). In general, three main media segments can be distinguished in Estonia: national Estonian-language media, local Russian language media and foreign media (including Russian-language media from Russia). Furthermore, it has been demonstrated that social media networks are a more important source of information for young Russian speakers than for young Estonian speakers (Integration Monitoring … 2017). Given the distinctiveness in media consumption between the Russian-language and Estonian-language cohorts in terms of the type and intensity of usage, we assume the trust in media to be one of the explaining factors behind the gap in civic and citizenship education. However, we admit the potential problem of reverse causality here, i.e. students with a better understanding of how society works trust (social) media less than others. We take this concern into account in interpretation.
H2.1 and H2.2: The association between trust in media and CCK scores has a reversed U-pattern–the ones who are the least and the most trustful, show the lowest result; those with modest trust have the highest results across language groups. While Estonian school students are relatively more trusting on classical media (“completely” 7% vs 4%, “quite a lot” 44% vs 28%), they trust less social media (“not at all” 13% vs 11%, “a little” 57% vs 52%). H2.1 poses the hypothesis regarding traditional media of television, newspapers and radio. H2.2 poses the same hypothesis regarding social media platforms, i.e. Twitter, blogs, and YouTube.
The trust in media variable is based on the question "How much do you trust each of the following groups, institutions or sources of information?”. We use question IS3G26G for operationalizing trust in’traditional’ media (TRUST_C) and IS3G26H (TRUST_SM), which is asking how much you trust social media (Twitter, blogs, YouTube). ICCS data reveals that Russian school students have a much lower trust towards media (12% do not trust media at all; 6% of students in Estonian schools do not), while there is no significant difference in trust to social media (11% vs 13% don’t trust social media at all). As Table 1 indicates, these are four-category variables; ordered categories are "completely, quite a lot, a little, not at all". Figure 2 reveals that there are considerable achievement differences between Russian and Estonian school students regarding trust in media; middle categories among Estonian school students perform considerably better with a small variance compared with almost no such effects in case of Russian school students. Also, as we see in Fig. 3, there is a clear distinction of CCK among Estonian school students who have complete trust compared to all lower trust categories. In the case of Russian school students, there are no significantly different CCK effects over the different categories of trust in social media. Thus, expected non-linearity between trust in media and CKK seems to be present only in case of Estonian students, a phenomenon potentially partly explained by Russian speakers’ distinctive media sources some of which are not necessarily beneficial for civic knowledge and engagement. We hope to cast more light on that in the section of analysis.
Hypothesis 3: Context-dependent effects of influence of religion
Recent scholarship has reinvigorated research of religion’s influence on political life (Van Heuvelen 2014). More broadly, results suggest that religion plays an important role in structuring a citizenry’s view of the moral economy, and thus has real consequences for political outcomes (Edgell 2012; Svallfors 2012). However, there are contradictory views on both–the direction of the relationship and on the causal mechanism. Furthermore, there is evidence that the relationship between religiosity and civic life might be moderated by context, and religion tends to empower (have a positive impact) only in settings where religiosity is a prevalent practice. There are studies (Sobolewska et al. 2015; Djupe and Gilbert 2006) that show that religious attendance has a strong positive effect on non-electoral participation in the case of those religions–Islam and Sikhism–that are politically salient, but not in the case of Hinduism, which is not politicised. Christianity presents a more complex case, where regular attendance bolsters electoral, but not non-electoral participation.
In our analysis, we focus on the importance of religiosity in guiding decisions in civic life instead of the effects of different religious practices. First, Estonia has moderately low diversity in terms of religious practices followed; second, there are considerable differences in the share of believers across the Estonian- and Russian-speaking community in Estonia. While Estonians are one of the least religious nations in Europe (Eichhorn 2012; Inglehart et al. 2014), almost half of Russian speakers claim to regularly practice, mainly Orthodox Christianity, making the latter the most-practised religion in Estonia (Statistics in Estonia, 2013). In Estonia, 66% of Estonians and 30% Russians do not follow any religion (ibid.). Thus, assuming the context dependency of the relationship between religiosity and civic life– religiosity restricts civic life in the contexts where religiosity is rare and empowers where religiosity is prevalent and considering the differences in the religiosity between Russian and Estonian groups, we assume the minority achievement gap to be diminished by religiosity.
H3: We hypothesise the religiosity to be negatively associated with CKK in Estonian schools and positively associated in Russian schools.
We use the independent variable RELINF to capture the impact of religion on CCK. RELINF indicates students’ endorsement of the influence of religiosity in society. Higher values indicate more positive attitudes toward the influence of religion.
RELINF is a composite of six categorical variables. These six variables are statements students can agree with on a scale of four categories ("strongly agree" to "strongly disagree"). The statements are the following: (a) religion is more important to me than what is happening in national politics; (b) religion helps me to decide what is right and what is wrong; (c) religious leaders should have more power in society; (d) religion should influence people’s behaviour towards others; (e) rules of life based on religion are more important than civil laws; (f) religious people are better citizens. So, a general composite score of RELINF is not a measure of religiosity as such; it is a proxy measuring students’ attitudes toward the role of religion in society. Summary statistics given in Table 1 show that we use a standardised score of RELINF. Group summary statistics of RELINF indicate that the mean of a standardised composite score of Russian schools (0.207) is statistically significantly different from Estonian school mean (− 0.075). Also, standard errors differ; in Russians schools there is less variance (s.e = 0.974) compared to Estonian schools (s.e = 1.038). As Fig. 4 explores correlations between civic knowledge and influence of religion, we see in the case of both groups a negative correlation. However, in the case of Russian schools, the slope of a simple linear trend line is smaller than in the case of Estonian schools. Thus we see a larger negative association between the religiosity and civic knowledge in Estonian school, the exploratory finding which contradicts our theory-driven hypothesis (H3), but gives some evidence that trust in religion can be less harmful to CCK in settings where religiosity is a prevalent culture.
Data, methods and statistical techniques
Our data originate from the IEA study ICCS, the cycle of 2016. To repeat, CCK scores between Estonian and Russian language schools differ 57 points (more than 10% of the mean score). We aim to find an association between the factors behind differences in Estonian-Russian school educational achievements in the CCK score and explain these differences. In general, our research design will rely on hierarchical modelling to capture the layered data and aim to explain how the different layers (school- and student-level) interact and impact civic knowledge. We hypothesise that school language is not explaining the minority achievement gap; instead, school language is the proxy for indicating’social space’ which moderates the effects of the independent student level variables.
In general student achievement is mostly explained by parental background characteristics, school-teacher, and peer effects. Regarding school and classroom level, literature has shown that citizenship competencies are not related to the subject matter, teaching time, the teachers’ educational approach, tailoring of teaching to student needs, and other school level variables (Dijkstra et al. 2015). Similarly, for most aspects of school quality, no significant effects were found in multilevel models (see ibid). Thus we rely on student level independent variables mostly and allow the estimation technique take care or school level effects by random slopes and intercepts.
Our data is hierarchical, and we show the data structure in Fig. 5. In general, data allow us to indicate the school language (ITLANG), school-level and multitude of individual-level variables. In setting up our model, we distinguish between following categories of explanatory components (Fig. 5): the language of school (school level); students’ activities in school and outside school including perceptions on classroom climate (student level); the trust of various media channels (student level) and religiosity-related characteristics (student level). As mentioned, we apply hierarchical modelling that allows to handle the unobservable school level heterogeneity without intensive school level controls. So our estimation strategy is not allowing us to directly control for school-level factors, instead school level random intercepts are applied.
We can assume that our data require a random intercept and slope model, where groups are schools. We support our assumptions by rich data structures of ICCS data and theoretical insights (e.g. Clarke et al. 2010) that indicate–when the selection mechanism is fairly well understood, and the researcher has access to rich data, the random effects model should be preferred. Random effects estimators of regression coefficients and shrinkage estimators of school effects are more statistically efficient than those for fixed effects. Barr et al. (2013) recommend in the case of embedded data structures specifying the maximal random effect structure, which means including random intercepts and random slopes for all within-subjects and within-items factors, as well as allowing correlations between within-unit random effects. These models are referred to as maximal LMEMs. Unfortunately, maximal LMEMs are prone to model non-convergence, which is the failure of the model estimation algorithm to reach a solution. In our case we were able to estimate random slopes and random intercepts (RMEL) using R package lmer4 (Bates et al. 2015).
We use the stepwise modelling technique, however in many steps we include the covariates that proxy the same phenomenon (e.g. background characteristics or student involvement factors). Our null model initiated by standard assumptions of hierarchical data structures is a two-level model where level 1: \({y}_{ij}={\pi }_{ij}+{e}_{ij}\) and level 2: \({\pi }_{0j}={\beta }_{00}+{u}_{0j}\). Random coefficients (level 2 predictors) is used to predict student performance in each school, while intercept values for school performance are assumed to be a sample of the intercepts from a larger population of schools. For estimating the model, we use the following specification:
where \(Y\) indicates CCK scores, \(L\) is language at school, \(F\) is the vector of family characteristics of the individual student and \(B\) is the vector of school and classroom quality indicators (measured at student level), \(M\), \(S\) and \(R\) are the trust in media, the trust in social media, and the trust in religion indicators, respectively. Also, we include the interaction between the language of the school L and independent variables \(M\), \(S\) and \(R\) to see how school language (identified as Estonian) moderates the effect of student-level independent variables. In the Schulz et al. (2018a) dataset, there are categorical (ordered) variables and scaled composite variables. A scale is a special type of derived variable that assigns a score value to students based on their responses to the component variables.
In Schulz et al. (2018a), composite variables were typically calculated as weighted likelihood estimates, where scores with a mean of 50 and a standard deviation of 10 for equally weighted countries. We turn all scaled variables into standardised variables with country mean zero and standard deviation equals to one. The description of transformed data (see Table 1) contains 2,857 observations; we assume that all missing values have been randomly allocated and thus, we applied imputation of missings (see Appendix 2). Imputations are important to make all our models comparable while we use maximum likelihood estimations as a method to fit the model. We have 2857 observations from 164 schools, while 75% of observations originate from Estonian language schools. Table 1 reveals the descriptive statistics of the imputed dataset (descriptives about raw ICCS data are indicated in Appendix 1). Many of our variables have self-explanatory abbreviations. In Student background characteristics, HISEI is a composite indicator indicating the highest parental occupational status, and HISCED is categorical highest parental educational level. Latter uses standard scales from "did not complete ISCED level 2" to "ISCED level 6, 7 or 8".
As indicated, in addition to standardised scored variables we have many categorical variables from survey design. Due to four (or less) category property of latters we will not treat any of the categorical variables as linear (numeric or continuous) and thus we estimate each category separately.
So to sum up our design: data require us to use hierarchical modelling, we opt for random effect model that has school random slopes and intercepts. In addition we imputed missing observations using Multivariate Imputation via Chained Equations and standardised all composite variables. All categorical variables are used as factor variables estimating the effects for each category. For model estimations we use RMEL and student total weights.
Results: confirmatory analysis
We apply hierarchical modelling to show the effects of, class culture (OPDISC) trust in media-related variables (TRUST_C and TRUST_SM) and influence of religion (RELINF) on civic knowledge score. We have following models: Model 0: empty Model; Model 1: Model 0 + ITLANG (variable indicating the language used by the student in a test, ITLANG = 1 indicates Estonian); Model 2: Model 1 + student background variables; Models 2A and 2B inflict student involvement factors step by step (first SINT and then SCHPART); Model 3: Model 2B + OPDISC; starting from Model 3 we use stepwise method to show the effect of our main independent variables –classroom climate, trust in traditional and social media, and trust in religion (higher values indicating more positive attitudes toward the influence of religion). Interaction terms between Estonian schools (ITLANG) and independent variables are applied in Model 6 and 7.
Interclass correlations
In each case, we see how much variation of the dependent variable is explained between schools or between individuals (students). To get meaningful interclass correlation coefficients (ICC) also for models with random slopes, we use option "adjusted" available in R icc package (sjstats library). The adjusted ICC uses the mean random effect variance, which is based on the random effect variances for each value of the random slope and random effects (see also Johnson 2014). In general, total variance in data is 16.5% which can be decomposed between schools, school language, and student. We show that language-level variance is larger than school-level variance (see Table 2), while the latter shows the’similarity coefficient’ which indicates the average degree of similarity between students in the same school. It is visible that in the case of Model 1 that a large part of school-level variance can be explained by school language. Also controlling for individual-level characteristics affects school level variability (M2). Controlling for student involvement (M2A and M2B) we see some marginal effect on school level variability. While our variable of interests–OPDISC (M3) has slightly bigger effect. In addition, including trust in media and religion variables (Model 4 and 5) we see a marginal decrease of school-level variability, so we can assume that language explains more the similarity of the students than the religious attitudes and trust in media (including interaction). Still, we almost eliminate the school level similarity between students, by full specification of the model (Model 7).
Confirmation of hypotheses
In Table 3, we report the regression estimates for models 2, 3, 5 and 6. In Model 2 we see that a 57-point CCK gap is decreased to 52 points when we control for student specific characteristics such as gender, language at home, immigrant status, the highest educational level and the highest occupational level of the parents. So as expected, background characteristics explain the gap, but not in full extent. Before estimating Model 3 which includes our independent variable for classroom climate–OPDISC, we estimate Models 2A and 2B which we report only in Table 2.
In Table 3, we report the control of the first hypothesis H1. H1: Russian language school OPDISC effect is larger than average effect and highly significant. While for Estonian school students effect size is smaller (\(Interaction1\) is negative). We see that on average, one standard deviation change of OPDISC has a significant positive effect on student achievements. In model 5, we reveal the effects of the other independent variables–trust in media, trust in social media and attitudes toward the impact of religion in society (RELINF). In the case of the latter, we see as expected significant negative and a large effect–one standard deviation change in the belief of the influence of religion in society will affect student results by 23 points. However, this insight will not yet allow us to confirm H3.
Interpretation of the effects of trust indicators of media and social media are more complex. In general, we see nonlinearity as expected. In case of trust in traditional media, there is a considerable increase in student results while answering "quite a lot" or "a little" compared to "completely". While "not at all" has insignificant effect compared to "completely". In the case of trust in social media, the effect sizes are increasing if the’degree of trust’ is decreasing. We have significant effects in all cases, so we interpret estimates as follows–smaller trust in social media will increase average student scores in civic knowledge.
In Model 7 Table 4, we control for hypothesis 2 (2.1 and 2.2) and 3. We add interactions between ITLANG and independent variables to Model 6. As we see, all media trust variables and ITLANG school-level variables turn insignificant. As now independent student-level variables reflect on Russian school students’ effects, we conclude that in the case of H2.1 (trust in media): Russian schools don’t differ significantly from Estonian schools–the media effect by school language turns insignificant. While in the case of H2.2 (trust in social media): Russian schools differ significantly from Estonian schools–social media effects are insignificant in the case of Russian schools but are significant and with the expected sign in Estonian schools. In Estonian schools, smaller trust to social media will bring along higher students’ scores, especially in case of "not at all" compared to "completely". In addition, as Model 7 captures moderating effects of the school language to independent variables of trust in media, we see that it turns school language variable (ITLANG) itself insignificant, both statistically and in magnitude.
Finally, H3 (attitudes toward influence of religion): we see a negative significant impact in the case of both schools, less belief that religion has an influence in society will bring along higher scores in civic knowledge. However, Estonian schools’ effects are higher, meaning more negative. So, we don’t confirm our H3 that in the case of Russian schools RELINF will have a positive effect on CCK, still it is less negative compared to Estonian school students. See Appendix 3 for the full specifications of the model with controls; moreover, we show model fit statistics in Appendix 3. For the comment, most of the controls have signs and significance as expected, including also student involvement factors. From student level controls–gender (girl), having high HISEI parents, books at home and test language the same as the home language, have considerable positive effects. While HISCED and IMMIG have unexpected insignificant effects. Also, school-level variables such as SINF and SPART have positive and significant effect.
The mechanism behind heterogeneous effects: language of the school or not
In this section, we look at a subsample–non-Estonians in Estonian school. Figure 6 illustrates the distribution of civic education scores of non-Estonians in Estonian schools (136 observations) compared to all Russian school students. This exercise explains whether there is some self-selection to the Estonian language schools based on background characteristics and most of all exhibits a robustness check of our argument that school language is the mediator not the explanation for the achievement gap.
Descriptively in Fig. 6 we see clear differences in distribution, while mean scores of both groups are similar–505.56 for a Russian school and 505.24 for non-Estonians in Estonian schools. The standard error of the subsample of non-Estonians in Estonian schools is smaller, so there is less variance in data.
Hence, we are interested in how our estimates of Russians school students (Models 6 and 7) look like compared to the same specifications of the subsample. We report estimates in Table 5. We see mostly insignificant effects. Due to maximum likelihood property, our models are not directly comparable in Table 3 and Table 4; however, we see a significant similarity in effect sizes and statistical significance.
We make a weak statement that non-Estonians perform similarly (their effect sizes of independent variables are similar) after controlling for student background and involvement in Estonian schools compared to students in Russian schools. So, we speculate that school language is just a’cultural indicator’ that moderates our independent variables and school language itself is not an explanatory cause in driving CCK scores.
Conclusions
Our aim was to find out what is the explanation for the achievement gap (16.5% of total variance) between Estonian and Russian school students in Estonia. The empirical strategy for this was to minimise (explain) between-school and -student variance. We showed that the gap between schools (16.5%) could be decomposed between school variance and between student variance. In Model 1, we saw that between school variance drops considerably if we test for school language. In Model 2, we indicated that individual background characteristics explain most of the between-school variance. Finally, after controlling for behavioural and attitudinal independent variables such as perceived openness of classroom climate, belief in the influence of religion, trust in traditional and social media, CCK between schools is less than 1.5%. We use this model as a final specification.
We hypothesised that open classroom climate, trust in social and classical media, and the belief in the influence of religion, explain the CCK gap between Russian and Estonian school students. In general we compiled three hypothesis–H1, H2 (H2.1 and H2.2.) and H3.
First, H1 stated that open classroom climate supports Estonian school students more. We saw the opposite–Russian school students benefit more from an open classroom climate, and this effect is highly significant. However, as in Russian schools on average the level of the classroom climate is less open, thus the gap is still partially explained by this.
Even though trust in classical media channels is low on both sides of the language divide, trust in classical media channels is exceptionally low among Russian school students. We were interested in whether this distinctiveness in trust in media (H2.1) and trust in social media (H2.2) have a significant explanatory power behind the knowledge gap in different language schools. We found that the trust in the classical media has some positive reverse u-shape effect. Thus, students who trust media quite a lot or moderately (compared with completely trusting) have higher achievements in civic knowledge scores. Still, there is no significant difference between Estonian and Russian language school students in that regard. At the same time, social media as an independent variable acts differently–a small amount of trust in social media tend to increase student scores. However, this effect is present only in the case of Estonian school students.
Finally, we hypothesised (H3) that belief in influence of religion in society matters in associations with civic knowledge and this effect is heterogeneous–positive in the case of Russian schools and negative in the case of Estonian school students. Descriptive statistics revealed that Russian school students have higher mean scores and less variance on the composite index measuring’influence of religion’ on everyday life. We showed that belief in influence of religion on civic life matters for both language groups–a lower role of religion in everyday life has an association with higher CCK scores. However, this effect is significantly larger for Estonian school students. So, we found no confirmation of H3–in both schools, the’religion variable’ has negative effects, however different in size.
Moreover, our final specification allowed us to’get rid of’ the Estonian school (language)) effect. We saw that when interactions are brought in, the Estonian school (language) effect turns statistically insignificant and also small in size. Thus, the language of the school influences the educational outcome through the perception on the openness of classroom, trust in media and religion, meaning indirectly.
We confirmed the robustness of our results by using a subsample of non-Estonians in Estonian schools. These students have similar mean scores in civic knowledge compared to Russian school students. We also see a lot of similarities in effect sizes. This demonstrated that the positive effect of’openness of classroom’ and the negative effect of’belief in the impact of religion’ make non-native students worse off independently from school language. Thus, we speculate that the civic knowledge gap of Russian-language school students is in addition to student background characteristics explained by attitudinal and behavioural variables like trust in social media and religiosity a phenomenon revealed also by others (Schulz et al. 2018b; Schulz and Ainley 2018). The explanation of student background characteristics lies in socio-economic characteristics, working through lower parental occupational status (HISEI).
To conclude, our school random effects hierarchical models showed that most student background and involvement characteristics work as expected. Regarding behavioural and attitudinal variables we showed that student-oriented classroom practices benefit Estonian student more than minority school students because these are experienced more often in Estonian schools. Russian school students have lower trust in classical media, and it can be speculated that this reflects their mingling between Estonian and Russian media channels. At the same time we found marginal evidence that high levels of trust (or no trust al all) is hurting students. Estonian students benefit from low trust to social media considerably, while effects are insignificant for Russian school students. In addition, these associations turned out to be more complex than expected and should elaborated further addressing the concerns of reverse causality. Finally, the impact of belief in religion (instead of civil law) is stronger (more negative) for Russian schools and has significant effects–the lower is the role of religion, the better are the results.
What could one learn from this comparative exercise? First, as literature is struggling between input quality (student background) and school quality (cf. Timmermans 2012) in explaining student achievements, we bring in another dimension–how can individual-level attitudinal and behavioural characteristics affect student civic achievements in different cultural settings (operationalized as the language of the school)? So this’social experiment’ in Estonia, where minority- and majority-language students embedded in different cultural settings (school language), can give us some weak evidence that the cultural setting (keeping Russian speaking students in Russian language schools) is moderating the effects of behavioural and attitudinal factors. However, we have no evidence that it can reverse the effects (e.g. belief in religion is still negatively associated with student achievement in civic knowledge,), at least as far as we discuss students largely from a European-Christian background where the cultural split is mostly between Christianity in Orthodox-Protestant dimensions. However, it can be also argued that the moderating effect of cultural setting shows how minority students are detached from civil society in specific case-contexts (see also Masso et al. 2013), and thus our study has limited external validity.
However, given the robustness of the positive association between open classroom climate and CKK revealed, one of the policy implication at best can be related to our first hypotheses – Russian school students can benefit from a changed teaching paradigm, proxied as the openness of classroom climate. It means that freedom to express their opinions, discuss and dispute over politics can benefit minority students to a large extent and policies that could eliminate the barriers that has hold Russian schools back in implementing that change would be an important step in diminishing minority achievement gap in Estonia.
In addition to the external validity concerns stated above, our study has limitations originating from our estimation strategy. A hierarchical model specification using random slopes and intercepts treats well unobservable heterogeneity between schools, so the estimation strategy is designed to adjust for unmeasured covariates, however, this is not enough to call the method a causal estimation technique. Moreover, all effects should be interpreted as associations or partial correlations at best. Our second hypothesis needs specific care. However, we have some theoretical evidence that media consumption is a possible explanatory factor for civic engagement, our estimation strategy, and model specification is not ruling out the problem of reverse causality. Namely, low scores in civic and citizenship education can lead to extreme values on trust in media–either no trust or complete trust. So, we encourage further studies and causal methods to test for the model specification error.
References
Alivernini, F., & Manganelli, S. (2011). Is There a relationship between openness in classroom discussion and students’ knowledge in civic and citizenship education? Procedia - Social and Behavioral Sciences, 15, 3441–3445. https://doi.org/10.1016/j.sbspro.2011.04.315.
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. https://doi.org/10.1016/j.jml.2012.11.001.
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01.
Burgette, L. F., & Reiter, J. P. (2010). Multiple imputation for missing data via sequential regression trees. American Journal of Epidemiology, 172(9), 1070–1076.
Campbell, D. (2008). Voice in the classroom: How an open classroom climate fosters political engagement among adolescents. Political Behavior, 30(4), 437–454. https://doi.org/10.2307/40213327.
Castillo, J. C., Miranda, D., Bonhomme, M., Cox, C., & Bascopé, M. (2015). Mitigating the political participation gap from the school: The roles of civic knowledge and classroom climate. Journal of Youth studies, 18(1), 16–35. https://doi.org/10.1080/13676261.2014.933199.
Castro, A.J., & Knowles, R.T. (2017). Democratic citizenship education. In Manfra and Bolick (Ed.), The Wiley Handbook of Social Studies Research (pp. 287–318). John Wiley & Sons, Inc.
Clarke, P., Crawford, C., Steele, F., & Vignoles, A. (2010). The choice between fixed and random effects models: some considerations for educational research. IZA Discussion Paper 5287. https://www.iza.org/publications/dp/5287/the-choice-between-fixed-and-random-effects-models-some-considerations-for-educational-research.
CIMU-SEE. (2018). Disinformation resilience in Central and Eastern Europe. Coalition of Information and Media Users in South East Europe. http://prismua.org/en/dri-cee/. Accessed 30 September 2019.
Dijkstra, A. B., Geijsel, F., Ledoux, G., & van der Veen & ten Dam, G. . (2015). Effects of school quality, school citizenship policy, and student body composition on the acquisition of citizenship competences in the final year of primary education. School Effectiveness and School Improvement, 26(4), 524–553. https://doi.org/10.1080/09243453.2014.969282.
Djupe, P. A., & Gilbert, C. P. (2006). The resourceful believer: Generating civic skills in church. Journal of Politics, 68(1), 116–127. https://doi.org/10.1111/j.1468-2508.2006.00374.x.
Edgell, P. (2012). A cultural sociology of religion: New directions. Annual Review of Sociology, 28(1), 247–265. https://doi.org/10.1146/annurev-soc-071811-145424.
Eichhorn, J. (2012). Happiness for believers? Contextualizing the effects of religiosity on life-satisfaction. European Sociological Review, 28(5), 583–593. https://doi.org/10.1093/esr/jcr027.
Erss, M. (2015). The politics of teacher autonomy in Estonia, Germany and Finland. Doctoral Dissertation, Tallinn University, Institute of Education Sciences.
Fennema, M., & Tillie, J. (1999). Political participation and political trust in Amsterdam: Civic communities and ethnic networks. Journal of Ethnic and Migration Studies, 25(4), 703–726. https://doi.org/10.1080/1369183X.1999.9976711.
Inglehart, R., C. Haerpfer, A. Moreno, C. Welzel, K. Kizilova, J. Diez-Medrano, M. Lagos, P. Norris, E. Ponarin & B. Puranen et al. (eds.). 2014. World Values Survey: Round Six - Country-Pooled Datafile Version: https://www.worldvaluessurvey.org/WVSDocumentationWV6.jsp. Madrid: JD Systems Institute.
Integration monitoring of the Estonian society 2017. (2017). Institute of Baltic Studies and Praxis Center for Policy Studies. https://www.kul.ee/en/integration-monitoring-estonian-society-2017. Assessed 10 June 2020.
Johnson, P. C. D. (2014). Extension of Nakagawa and Schielzeth’s R2 GLMM to random slopes models. Methods in Ecology and Evolution, 5, 944–946. https://doi.org/10.1111/2041-210X.12225.
Kahne, J., Middaugh, E. & Allen, D. (2014). Youth New Media, and the Rise of Participatory Politics. YPP Research Network Working Paper 1, Youth and Participatory Politics Research Network. Oakland. https://clalliance.org/wp-content/uploads/files/ypp_workinpapers_paper01_1.pdf.
Koršňáková, P., & Carstens, R. (2017). Social media and civic engagement: New developments From IEA’s International Civic and Citizenship Education Study 2016. Childhood Education, 93(6), 511–513. https://doi.org/10.1080/00094056.2017.1398566.
Khavenson, T., & Carnoy, M. (2016). The unintended and intended academic consequences of educational reforms: the cases of post-soviet Estonia, Latvia and Russia. Oxford Review of Education, 42(2), 178–199. https://doi.org/10.1080/03054985.2016.1157063.
Kiilo, T., & Kutsar, D. (2012). When language becomes power: Russian-speaking teachers in the bilingual general education system in Estonia. British Journal of Sociology of Education, 33(2), 245–262. https://doi.org/10.1080/01425692.2011.649834.
Klaas-Lang, B., Praakli, K., Peedisson, A., & Lašmanova, A. (2014). Arvamusi ja hinnaguid riigikeele õppe korraldamise kohta vene õppekeelega koolides [Opinions and assessments of the teaching of national education in Russian language schools]. Tartu University. http://hdl.handle.net/10062/45121.
Knowles, R. T., & McCafferty-Wright, J. (2015). Connecting an open classroom climate to social movement citizenship: A study of 8th graders in Europe using IEA ICCS data. The Journal of Social Studies Research, 39(4), 255–269. https://doi.org/10.1016/j.jssr.2015.03.002.
Knowles, R. T., Torney-Purta, J., & Barber, C. (2018). Enhancing citizenship learning with international comparative research: Analyses of IEA civic education datasets. Citizenship Teaching & Learning, 13(1), 7–30. https://doi.org/10.1386/ctl.13.1.7_1.
Levinson, M. (2010). The Civic Empowerment Gap: Defining the Problem and Locating Solutions. In Sherrod and Torney-Purta (Ed.), Handbook of Research on Civic Engagement, (pp. 331–361). Hoboken, NJ: John Wiley & Sons.
Lindemann, K. (2013). The school performance of the Russian-speaking minority in linguistically divided educational systems: A comparison of Estonia and Latvia. In: Integration and inequality in educational institutions, (pp 45–69). Springer.
Loogma, K., Ruus, V.R., Talts, L., & Poom-Valickis, K. (2009). Õpetaja professionaalsus ning tõhusama õpetamis- ja õppimiskeskkonna loomine. OECD rahvusvahelise õpetamise ja õppimise uuringu TALIS tulemused [Teachers professional skills in creating efficient teaching and studying environment]. Tallinn University. http://hdl.handle.net/10062/40808.
Masso, J., Espenberg, K., Masso, A., Mierina, I., & Philips, K. (2013). Gini country report: Growing inequalities and their impacts in the Baltic States: Estonia, Latvia, Lithuania. Tech. rep., AIAS, Amsterdam Institute for Advanced Labour Studies. https://ideas.repec.org/p/aia/ginicr/baltics.html.
Martens, H., & Hobbs, R. (2015). How Media Literacy Supports Civic Engagement in a Digital Age. Atlantic Journal of Communication, 23(2), 120–137. https://doi.org/10.1080/15456870.2014.961636.
Mikk, J., Kitsing, M., Must, O., Säälik, Ü., & Täht, K. (2012). Eesti PISA 2009 kontekstis: tugevused ja probleemid. [Estonia in the context of PISA2009: strengths and challenges]. Programmi EDUKO uuringutoetuse kasutamise lepingu aruanne. University of Tartu, Ministry of Education in Estonia. http://hdl.handle.net/10062/40609.
Miller, A. H., Gurin, P., Gurin, G., & Malanchuk, O. (1981). Group consciousness and political participation. American Journal of Political Science, 25(3), 494–511. https://doi.org/10.2307/2110816.
OECD. (2016). PISA 2015 Results (Volume II): Policies and Practices for Successful Schools. OECD Publishing, Paris. https://doi.org/10.1787/9789264267510-en.
Pasek, J., Kenski, K., Romer, D., & Jamieson, K. H. (2006). America’s Youth and Community Engagement How Use of Mass Media Is Related to Civic Activity and Political Awareness in 14- to 22-Year-Olds. Communication Research, 33(3), 115–135. https://doi.org/10.1177/0093650206287073.
Põder, K. Lauri, T., & Rahnu L. (2017). Eesti koolisüsteemi väljakutsed: Õpiedukuse erinevus erikeelsetes koolides ja sisserändajate koolivalikud [challenges of estonian school system: achievement in different language schools and the choices of migrants]. In: Tammaru T, Eamets R, Kallas K (eds). Eesti Inimarengu Aruanne, (pp. 155–162). Eesti Koostöö Kogu. https://www.2017.inimareng.ee/en/immigration-and-integration/challenges-facing-the-estonian-school-system/.
Quintelier, E. (2010). The Effect of Schools on Political Participation: A Multilevel Logistic Analysis. Research Papers in Education, 25(2), 137–154. https://doi.org/10.1080/02671520802524810.
Quintelier, E., & Hooghe, M. (2011). Television and Political Participation among Adolescents: The Impact of Television Viewing. Entertainment and Information Preferences, Mass Communication and Society, 14(5), 620–642. https://doi.org/10.1080/15205436.2010.530383.
Romer, D., Jamieson, K. H., & Pasek, J. (2009). Building social capital in young people: The role of mass media and life outlook. Political Communication, 26(1), 65–83. https://doi.org/10.1080/10584600802622878.
Schleicher, A. (2019). PISA 2018: Insights and Interpretations. https://www.oecd.org/pisa/PISA%202018%20Insights%20and%20Interpretations%20FINAL%20PDF.pdf. Assessed 20 December 2020.
Schlomer, G. L., Bauman, S., & Card, N. A. (2010). Best practices for missing data management in counselling psychology. Journal of Counseling psychology, 57(1), 1–10. https://doi.org/10.1037/a0018082.
Schulz, W., Ainley, J., Fraillon, J., Losito, B., Agrusti, G., & Friedman, T. (2018a). Becoming Citizens in a Changing World IEA International Civic and Citizenship Education Study 2016 International Report. Springer International Publishing.
Schulz, W., Carstens, R., & Losito, B. (2018b). International Civic and Citizenship Education Study 2016. Amsterdam, IEA: Technical Report.
Schulz, W., & Ainley, J. (2018). Students’ attitudes toward equality opportunities, trust in civic institutions and endorsement of religious influence, The Australian Council for Educational Research (ACER), Paper prepared for the Annual Meeting of the American Educational Research Association, New York, 13–17 April 2018: https://iccs.acer.org/files/SchulzAinley_CivicAttitudes%28AERA2018%29.pdf.
Siiner, M. (2014). Decentralisation and language policy: local municipalities’ role in language education policies. Insights from Denmark and Estonia. Journal of Multilingual and Multicultural Development, 35(6), 603–617. https://doi.org/10.1080/01434632.2014.889143.
Sobolewska, M., Fisher, S., Heath, A., & Sanders, D. (2015). Understanding the effects of religious attendance on political participation among ethnic minorities of different religions. European Journal of Political Research, 54(2), 271–287. https://doi.org/10.1111/1475-6765.12081.
Statistics Estonia (2013). PHC 2011: over a quarter of the population are affiliated with a particular religion https://www.stat.ee/en/uudised/news-release-2013-050. Accessed 21 June 2019.
Svallfors, S. (2012). Contested welfare states: Welfare attitudes in Europe and beyond. Stanford University Press.
Tammaru, T., Kallas, K., & Eamets, R. (2017). Estonia at the age of migration. In: Tammaru T, Eamets R, Kallas K (eds) Eesti Inimarengu Aruanne (pp. 3–10). Eesti Koostöö Kogu. https://2017.inimareng.ee/en.
Timmermans, A.C. (2012). Value added in educational accountability: Possible, fair and useful? Doctoral Dissertation. University of Groningen, Faculty of Behavioural and Social Sciences.
Van Heuvelen, T. (2014). The religious context of welfare attitudes. Journal for the Scientific Study of Religion, 53(2), 268–295. https://doi.org/10.1111/jssr.12118.
Van Buuren, S., & Groothuis-Oudshoorn, K. (2011). MICE: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–67. https://doi.org/10.18637/jss.v045.i03.
Acknowledgements
We thank Anu Toots for encouraging us to use the ICCS data and valuable tips and ideas regarding theoretical premises. Also, our gratitude goes to anonymous referees from the Baltic Economic Association conference in Riga, International School Choice and Reform Conference in Lisbon, the 8th IEA International Research Conference (IRC-2019) in Copenhagen and LEER Conference for Education Economics in Leuven for their guidance and constructive criticism. Most of all, thanks for two autonomous referees from the Large Scale Assessments of Education Journal for their feedback and comments.
Funding
This work was supported by the Grant TF514 "IEA International Civic and Citizenship Education Study (Schulz et al. 2018a)" and an EBS base-financing grant.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1
See Table
6.
Appendix 2: Imputation of missing data
In the case of 12 variables, we have missing data. In the case of HISEI, which indicates the highest parental occupational status, we have 81 missing observations. See Table
7, which indicates that disregarding HISEI variables have less than 3% of missing observations. Best practices for missing data management (e.g. Schlomer et al. 2010) including revision of patterns of missing data and description of strategies for dealing with them suggest that mean substitution is a poor method for handling missing data. Our measurement scales of variables are often ordered categorical, but we also have binary or standardised continuous scale variables.
We use Multivariate Imputation via Chained Equations (MICE) package for creating multiple imputations (Van Buuren and Groothuis-Oudshoorn 2011) as compared to a single imputation (such as the mean), which is considered to take care of uncertainty in missing values. MICE assumes that the missing data are Missing at Random (MAR), which is supported by the pattern of missing observations is revealed in Fig.
7. MAR means that the probability that a value is missing depends only on an observed value and can be predicted using them. It imputes data on a variable by variable basis by specifying an imputation model per variable. Multiple imputation is particularly well suited to deal with missing data in large studies (see Burgette and Reiter 2010), including interactions and nonlinear relations. We use classification and regression trees (cart) which is suitable for categorical and continuous variables.
Appendix 3
See Table
8.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Põder, K., Lauri, T. Classroom, media and church: explaining the achievement differences in civic knowledge in the bilingual school system of Estonia. Large-scale Assess Educ 9, 3 (2021). https://doi.org/10.1186/s40536-021-00096-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s40536-021-00096-3