“That’s just, like, your opinion” – European citizens’ ability to distinguish factual information from opinion

1 Introduction

In a 2020 article by the Poynter Institute, Eliana Miller asserts that “(r)eaders often can’t tell the difference” between opinion, news, and editorials (Miller, 2020). In addition to people struggling to identify such genre differences in the context of professional journalistic products, they often have difficulties to differentiate between opinionated and fact-based claims (e. g., Graham and Yair, 2023; Merpert et al., 2018; Mitchell et al., 2018; Walter and Salovich, 2021). This is a particular challenge on social media platforms, where information is shared by both experts and laypersons. Notably, substantial parts of the population rely on social media to inform themselves about current affairs (Newman et al., 2021), and the line between factual information and opinions is increasingly difficult to draw. This may have severe consequences for citizens’ critical thinking and their behavioural responses to falsely categorised information, and it is also relevant when assessing the corrective impact of fact-checking messages: If confronted with opinionated statements (online), citizens may be more likely to take them at face value if they believe they present a fact, and even corrective attempts in the form of fact-checkers may fail to mitigate that initial effect (Walter and Salovich, 2021). We therefore need to understand what makes citizens more (or less) able to differentiate between factual information and opinions.

In this study, we rely on original survey data from ten European countries, collected in the wake of the 2019 European Parliament (EP) elections. We asked respondents to classify six political statements relating to the European Union (EU) as either a factual statement or an opinion, and to subsequently indicate whether they judged them as accurate (if categorised as a factual statement), or whether they agreed with them (if categorised as an opinion). We assess respondents’ ability to correctly differentiate between factual and opinionated statements and investigate which role sociodemographic indicators and relevant attitudes play for correct classification. Furthermore, we aim to understand whether biases in citizens’ categorisation affect the strength of their beliefs in the accuracy of and/or their personal agreement with the statements. Our findings contribute to an understudied area of public opinion research on the underlying causes for political misperceptions and “fake news” susceptibility.

2 The distinction between factual information and opinions

Most attempts to distinguish between factual information and opinions highlight that the former can be “proved or disproved based on objective evidence” (Mitchell et al., 2018, p. 6), while “an opinion can be neither right nor wrong” (Cohen et al., 1989, p. 13). Unlike factual information, opinionated statements – particularly in the context of news media and reporting – are often considered to be “based on the values and beliefs of the journalist or the source making the statement” (Mitchell et al., 2018, p. 6); they “incorporate varying degrees of speculation, confidence, and judgment” (Schell, 1967, p. 5) or “depend on internal, subjective experience and preferences, as in the case of aesthetic judgments and personal taste” (Banerjee et al., 2007, p. 1084). The respective definitions by the Collins dictionary emphasise similar differences, with factual being defined as “concerned with facts or contains facts, rather than giving theories or personal interpretations” (Collins, n.d.-a), and opinions being “a belief not based on absolute certainty or positive knowledge but on what seems true, valid, or probable to one’s own mind” (Collins, n.d.-b). Especially the reference to personal interpretations or one’s own mind when it comes to opinions is crucial compared to more objective factual information. Similarly, Zaller (1991, p. 1215) points out that “every opinion is a marriage of information and values – information to generate a mental picture of what is at stake and values to make a judgment about it.” What these distinctions have in common is an emphasis on evaluative judgments and thus subjective individual assessments in connection with opinions. In general, extant conceptual distinctions do not assume that factual statements cannot be contested; indeed, as discussions across the globe during the Covid-19 pandemic revealed, a statement such as “vaccines significantly reduce the risk of a virus infection” is far from uncontroversial in public opinion. However, for categorisation as factual, what matters is whether a statement can be proven right or wrong, whereas we usually assume that this is not the case for opinions (see also Edelsztein and Vázquez, 2021).

The differentiation between factual information and opinions is also central to journalistic norms and at the core of professional journalistic values: Survey data from the Netherlands, for example, show that audience members find it very important that news media “clearly separate facts and opinions” (van der Wurff and Schönbach, 2014; see also Karlsson and Clerwall, 2019). Usually, traditional media distinguish both by denoting opinions on, for example, designated pages (in newspapers) or announcing commentary in a separate section of an evening news show. However, these clear-cut lines become increasingly blurry in the online world, especially when the source of a story is difficult to identify (e. g., Morrow et al., 2022; Pennycook et al., 2020). Whether or not, for example, a Facebook post on vaccine efforts at the early stages of the Covid-19 pandemic came from a credible source and presented “checkable” factual information was not always easy to determine.

Along similar lines, Walter and Salovich (2021) argue that for political fact-checkers to work as intended (i. e., in correcting misinformation), media users need to be able to distinguish opinions from (verifiable) factual information. Since “[one] of the distinct aspects of fact-checkers is their exclusive focus on checkable political statements or claims that can be factually verified” (Walter and Salovich, 2021, p. 4; original emphasis), political opinions may bypass fact-checking, with potential detrimental consequences for public opinion formation. In their study, US citizens’ ability to differentiate between factual and opinionated statements moderated the effectiveness of a fact-checker: Only those individuals who could correctly identify a large share of checkable statements reported a decrease in the perceived accuracy of false information (Walter and Salovich, 2021). In contrast, lower levels of correct “checkability” classification led to a backfire effect, that is, these citizens perceived a higher accuracy of false information, particularly those who exhibited a political bias towards the initial message. However, Merpert et al. (2018) showed that a simple 15-minute online training exercise including examples and detailed explanations significantly increased participants’ ability to detect checkable facts (see also Edelsztein and Vázquez, 2021; Tully et al., 2020). Yet it is important to acknowledge that the distinction between matters of fact and matters of opinion is not necessarily clear-cut from the start (e. g., Banerjee et al., 2007; Schell, 1967; Walter and Salovich, 2021), and that it becomes particularly challenging to distinguish between both in a social media setting (Morrow et al., 2022).

Previous evidence indicates that media users, both online and offline, struggle to determine whether they are exposed to factual information or opinions: Distinguishing between both can be a major challenge, even for fairly well-educated adults (e. g., Graham and Yair, 2023; Kuhn, 2010; Merpert et al., 2018; Walter and Salovich, 2021). Data from a Pew Research study (Mitchell et al., 2018) reveal that US American adults are better able to identify opinionated (e. g., “Immigrants who are in the U.S. illegally are a very big problem for the country today”) than factual statements (e. g., “President Barack Obama was born in the United States”) relating to politics. While on average each factual and opinion statement was correctly classified by 70 % of respondents, the authors found large differences with regard to respondents’ ability to distinguish between both. Evidence from Argentina supports these results by showing that respondents correctly classified 69 % of statements to contain checkable facts (vs. non-checkable statements), with significant differences in this ability according to respondents’ demographic characteristics (Merpert et al., 2018).

To contribute to this scarce previous evidence, we first wish to establish in how far such differentiating skills are prevalent in Europe as well, and ask:

RQ1: To what extent are people able to correctly distinguish between factual and opinion statements?

Research has shown that citizens’ political knowledge and interest as well as their gender, age, education, and media literacy affect their ability to distinguish the two types of statements. Citizens with a higher level of political knowledge and interest are better able to identify either; similarly, digital savviness (i. e., familiarity with digital media and technology) and higher media trust both increase the chances that citizens correctly classify factual information and opinions (Mitchell et al., 2018; see also Seo et al., 2020). In Argentina, men were more successful in identifying statements than women, and younger participants fared significantly better than their elder counterparts, as did those with a comparably higher formal education (Merpert et al., 2018). To extend this line of enquiry and gain a better understanding of the conditions that enable European citizens to distinguish between the two concepts, we pose three research questions. Even though we discuss why certain variables may influence peoples’ ability to distinguish between factual and opinion statements, we do not have strong theoretical claims for all of them. Further, in line with our research aiming to explain peoples’ distinction skills as comprehensively as possible – that is, identifying all or at least many of the relevant explanatory factors – we thus focus on broader research questions instead of formulating hypotheses for single variables of interest (but not for others).

First, we assess citizens’ trust in the news media and the extent to which they perceive the media to be agents of mis- and/or disinformation (Hameleers et al., 2022). Misinformation perceptions assess opinions about levels of untruthfulness in the media that may result from honest mistakes, whereas senses of disinformation measure notions of deliberate manipulation by journalists and the media. Either could be relevant predictors for citizens’ reliance on the news media to provide perceived factual knowledge. Along similar lines, Mitchell and colleagues (2018) found that US citizens with higher levels of media trust were more likely to correctly identify factual and opinionated statements.

We further ask whether the degree to which citizens rely on informal sources on social media may relate to their ability to differentiate factual information from opinions. The social media environment makes it increasingly difficult to identify false claims and fake news (Morrow et al., 2022; for a conceptual discussion of the “fake news” term see, e. g., Egelhofer and Lecheler, 2019). Findings by Luo et al. (2022) show that Facebook news headlines with more “likes” are perceived as more credible, and that study participants fare less well in detecting a fake news headline if it has a lot of likes. But it is particularly vulnerable populations (i. e., elderly and lower-educated citizens) that rely on informal contacts and judge them to be reliable sources of information on social media platforms (Seo et al., 2020), which shows that social endorsements (who of my friends likes/shares news online?) can play an important role (see also Turcotte et al., 2015). To take this information environment into account, we assess the relevance of informal social media sources (i. e., not professional news media) for respondents’ exposure to political news such as, for example, family and friends, celebrities, or YouTubers. Additionally, we expect levels of media literacy to determine citizens’ ability to distinguish factual information from opinion (e. g., Merpert et al., 2018; Seo et al., 2020; Tully et al., 2020). Summing up these potentially influential factors, we ask:

RQ2a: Which factors related to citizens’ use and perceptions of news media are associated with their ability to correctly distinguish factual from opinion statements?

Second, we wish to determine whether relevant political background variables might play a role and assess the influence of political knowledge, interest, efficacy, and respondents’ political orientation. Findings reported by Mitchell et al. (2018) indicate that higher levels of both political knowledge and political interest increase the likelihood that citizens in the US correctly classify either type of statements. The same positive relationship might be found for higher levels of internal political efficacy. Finally, we assess the role of respondents’ self-placement on the left-right political scale to account for potential biases as a result of political orientation, for instance to classify opinions that appeal to one’s political point of view as factual (see e. g., Mitchell et al., 2018; Taber and Lodge, 2006; Walter and Salovich, 2021). In line with this, Graham and Yair (2023) find evidence that partisanship affects citizens’ ability to distinguish factual and opinion statements. The authors determine that partisan differences in this area align with study respondents’ wish to express a response that is in line with their political orientation in the form of expressive responding. However, Graham and Yair’s studies are situated in a US and Israeli context, and it is unclear whether ideological leaning plays a similarly important role in the European context. Hence, we ask:

RQ2b: Which factors related to citizens’ political background are associated with their ability to correctly distinguish factual from opinion statements?

Lastly, we assess the influence of individuals’ sociodemographic backgrounds: Going beyond the common consideration as mere controls, various extant studies show the relevance of sociodemographic factors in the context of correctly identifying types of information. For instance, evidence by Merpert et al. (2021) highlights the importance of gender, age, and formal education, with both men, younger individuals, and those with higher levels of education scoring higher on the identification of checkable statements in Argentina (see Seo et al., 2020, for findings from the US). Whether these factors are similarly influential in the European context is again an empirical question, and we thus ask:

RQ2c: Which factors related to citizens’ sociodemographic background are associated with their ability to correctly distinguish factual from opinion statements?

3 Accuracy beliefs and personal agreement

In a next step, we aim to understand whether respondents’ classifications of factual information and opinions also affect the strength of their beliefs towards the accuracy of and/or their personal agreement with the respective statements – or, to be more precise, whether this process works the other way around. We assume that European citizens who categorise a statement as factual also believe it to be accurate or true, whereas they are more likely to dismiss presumed opinionated statements because they do not share the expressed viewpoint (see also Graham and Yair, 2023). In other words, people’s bias in favour of or against a view expressed in the statement under question will influence their perception of whether it constitutes factual information or an opinion. These assumptions align with the findings by Mitchell and colleagues (2018, p. 28): “Overall, Americans overwhelmingly tie the idea of news statements being factual with them also being accurate,” while “(…) seeing factual statements as opinions largely coincides with disagreeing with them.”

Take the aforementioned example of the sentence “vaccines significantly reduce the risk of a virus infection”: As public debates have shown, people may differ in evaluating the truthfulness of the statement and may not believe in its accuracy. However, independent of whether the statement is true or not, it can be checked and verified and would thus, following the definitions outlined above, constitute a factual statement. We posit that citizens who agree with the claim made in the sentence (i. e., find it to be accurate) will be more likely to categorise it as factual, whereas people who disagree with it will judge it to be an opinion. Following Taber and colleagues (2009) and Walter and Salovich (2021), we base these assumptions on the theory of motivated reasoning, which assumes that citizens’ pre-existing opinions will guide their processing and understanding of new incoming information if they follow partisan (as opposed to accuracy) goals in their political judgements (Taber and Lodge, 2006). Prior beliefs are then defended to the extent that new information will be dismissed (avoided, derogated, rejected; see Walter and Salovich, 2021) if it does not align with these beliefs (see also Gleadon, 1988). Furthermore, attitude-inconsistent new information is held to higher standards than congenial content, insofar as individuals will actively engage in more cognitive effort to disconfirm it (Taber et al., 2009). Along these lines, factual information about some political topics is more relevant for specific parts of the electorate than others, and political ideology can play a powerful role for motivated processing and reasoning of (political) arguments (Graham and Yair, 2023; Kuklinski et al., 1998; Taber and Lodge 2006).

We therefore assume that citizens will reject factual statements as “mere” opinion if they do not agree with them because it allows them to brush aside their importance and confirm their own personal existing beliefs (“That’s just, like, your opinion!”). In contrast, accepting an opinion statement as factual lends added weight to its importance and can reinforce the certainty in one’s own belief. As outlined above, these processes play a particularly crucial role when individuals are provided with information by the news media (Pennycook et al., 2020), which is why lower levels of media trust and heightened perceptions of media mis- and disinformation may be influential in this context: Individual biases in conjunction with scepticism towards journalistic professionals may lead citizens to brush aside information that does not align with their views. Based on the background of political motivated reasoning, we formulate the following hypotheses:

H1a: People who wrongly classify opinion statements as factual information are more likely to judge them as accurate compared to respondents who correctly classify factual statements.

H1b: People who wrongly classify factual statements as opinions are more likely to disagree with them compared to respondents who correctly classify opinion statements.

4 Data and methods

Factual vs. opinion statements

For the purpose of this study, we designed six statements for which the respondents were asked: “You will now see a series of statements that have been taken from news stories. Regardless of how knowledgeable you are about the topic, would you consider this statement to be a factual statement (whether you think it is accurate or not) or an opinion statement (whether you agree with it or not)?” (see also Mitchell et al., 2018; for an alternative approach see Graham and Yair, 2023). By adding the explanations in parentheses, we provided a minimal description as to how one may think about the difference between factual and opinion statements without providing a full-fledged definition or even concrete examples of the distinction. By not explicitly providing help, we aimed to uncover people’s real and unbiased ability to distinguish the two types of statements. Three of the statements each were factual and opinion statements (presented in random order):

The opinion statements were:

1. “Some member states contribute too little to the EU budget;”

2. “Climate change is one of the biggest challenges for the EU today;”

3. “A minimum wage of 10.50 Euro/hour across the EU is essential for the economy.”

The factual statements were:

1. “The amount of refugees coming to the EU has decreased significantly since 2016;”

2. “Turnout at the 2019 EP elections was the highest in 20 years;”

3. “Spending on agricultural subsidies makes up the largest portion of the EU’s budget.”

We chose the statements to cover common topics and information that appeared in the context of the 2019 European parliamentary elections and related political debates. Following established definitions (e. g., Cohen et al., 1989), the factual statements include information that one can check and provide concrete evidence for. All three factual statements represent true facts, that is, we simplified respondents’ task somewhat by not presenting both true and untrue factual statements, which could have resulted in confusion among respondents.^[5] The opinion statements represent viewpoints that cannot be proven by hard evidence, but one can certainly agree with the formulated opinions (see also Merpert et al., 2018). In line with Walter and Salovich’s (2021) recommendation, the six statements cover a variety of topics to arrive at more generalisable results not related to respondents’ knowledge of one specific topic only. However, all statements related to knowledge about and attitudes towards the European Union, which allowed for comparability between the ten countries. Furthermore, we did not provide any information about the source of the statements in order to exclude potential effects due to source credibility. The differentiation between the two types of statements is not equally clear-cut for all six statements, which adds to the ecological validity of our research, considering the complexity of news and information in the regular media environment. Our design hence presents a trade-off in external and internal validity, and we get back to this point in the discussion. For our main regression analyses, we calculate the sum of correctly classified statements (0–6 scale).

Depending on whether respondents classified a statement as an opinion or as factual, they were further asked: “To the best of your knowledge, do you think the factual statement is accurate or inaccurate?” (for statements identified as factual), or “Do you tend to agree or disagree with the opinion statement?” (for statements identified as opinion). The respective answers offered a binary choice between inaccurate (0) and accurate (1) and between disagree (0) and agree (1), respectively. This setup asking for either accuracy or agreement based on identification by respondents and not on the real classification of a statement follows the design of the existing Pew Research study (Mitchell et al., 2018; see also Graham and Yair, 2023). We decided to not also ask for the “correct” follow-up questions – accuracy for actual factual statements and agreement for actual opinions – in order to not confuse respondents during the survey (i. e., to not indicate that they may have been wrong in their classification), which may have influenced their answering behaviour in the remaining classification tasks. It would further have been counterintuitive for respondents to be asked to rate agreement for a self-declared factual statement (and accuracy for self-declared opinions), and the resulting perceived logical flaws in the survey may ultimately have resulted in survey break-off. As a result, and following extant research, we consistently asked only one follow-up question after respondents’ assessment of each statement. Figure 1 presents an overview of the survey process in the form of a flow chart, visualising how respondents were guided through the questionnaire based on their responses.

Figure 1:

Overview of classification tasks in the survey.

5 Results

Table 1 presents the proportion of respondents who correctly classified each statement. As the numbers illustrate, respondents struggled to correctly distinguish factual information from opinions, with several statements being correctly identified by only half of the respondents or even less. Only the statements about minimum wage (opinion) and turnout (factual) were correctly identified by more than two thirds of respondents. In comparison to the Pew research setup in the US (Mitchell et al., 2018), the number of correctly classified statements is (much) lower in our sample.

Turning to the sum scores of correctly identified statements in Figure 2, we can see that the largest proportions of respondents correctly identified three or four statements (M = 3.39, SD = 1.41).^[6] Around 7.5 % were able to correctly identify all six statements, while only 1.5 % did not correctly classify a single statement. Comparing the number of correct classifications of the three opinion statements (M = 1.63, SD = .90) with the correct classification of the three factual statements (M = 1.76, SD = 1.02) shows a significant difference, with respondents being better able to correctly classify the factual statements (t(6642) = 8.05, p < .001); yet the overall ability is still low.

Figure 2:

Number of correctly classified statements.

Next, we turn to the regression model results, displayed in Table 2, to analyse associations of media-related, political, and sociodemographic variables with the ability to correctly distinguish opinions from factual information. The first model focusses on the combined 0–6 scale of correctly classified statements as dependent variable. For the media variables (RQ2a), we see that higher perceived disinformation in the media and the use of a larger number of informal social media sources is negatively related to the ability to correctly distinguish factual information from opinions. We equally observe two significant relationships for the political variables (RQ2b), with higher political efficacy and greater political knowledge positively linked to the ability to correctly identify factual and opinion statements. Looking at the sociodemographics (RQ2c), we find a positive association for higher educated respondents, and negative coefficients for female and older respondents, all in line with Merpert et al.’s findings (2018) in the Argentinian context.

Some further interesting results appear in models 2 and 3 when separating the dependent variable into identifying exclusively opinion or factual statements. First, we largely observe the same significant associations in these two sub-models compared to the combined model. Second, we see a couple of additional significant coefficients that run in opposite directions between the two sub-models, which were hidden in the combined model. From the media variables, we now see opposing relationships for media trust, with higher levels of trust being associated with fewer correctly classified opinion statements, but more correctly classified factual statements. In other words, people with higher media trust tend to classify both types of statements more often as factual than people with lower media trust.

We similarly see opposing coefficients for political interest, with more interested respondents correctly identifying fewer of the opinion statements, but more of the factual statements. Again, this means that respondents with a higher level of political interest tend to classify our statements as factual more often. In contrast to these relationships, we do not observe any significant coefficients for self-perceived media literacy, misinformation perceptions, or left-right ideology (and extremity).

Our robustness checks confirm these patterns. First, Table A2 in the Appendix displays the same regression outcomes when using the alternative ordinal probit regression setup. Second, in the separate probit regression models (one for each statement) displayed in Table A3, we see the described patterns in several models, indicating that the relationships do not depend on the content of a single statement but are more universal. Still, some of the relationships are stronger (or weaker) for some statements than for others. The important aspect, though, is that the coefficients of the combined model(s) are not driven by only one of our six statements.

Finally, turning to the perceived accuracy of (self-identified) factual statements and agreement with (self-identified) opinion statements, Table 3 displays the respective values across the six statements (the values represent the proportion of respondents who perceived the statements as accurate or agreed with them). We expected that respondents would be more likely to believe in the accuracy of wrongly classified factual statements – that is, the three actual opinion statements (H1a) – and more likely to disagree with the wrongly classified opinion statements – that is, the three actual factual statements (H1b). Taking the average of accuracy and agreement across the three statements each (numbers printed in bold) supports our expectations. First, respondents who wrongly classified opinion statements as factual are more likely to believe them (M = .70) than respondents believe the correctly classified factual statements (M = .66). The difference is much larger for the agreement measures: Respondents who wrongly classified factual statements as opinions tend to agree less with the statements (M = .25) than respondents agree with the correctly classified opinions (M = .54). Adding to this, the separate agreement with any of the (wrongly classified) factual statements is much lower (.19–.32) than with any of the correctly classified opinion statements (.48–.58). To be clear, due to the design of the classification tasks and related different sample compositions for the follow-up questions (as either agreement or accuracy was asked depending on the self-classification of the statements by respondents; see Figure 1), we cannot run common statistical measures such as t-tests which would provide clear cut answers for our hypotheses. Still, keeping this limitation in mind, in particular the sizeable differences in agreement support our expectations, especially H1b.

6 Discussion

In this study we examined European citizens’ ability to distinguish between factual and opinionated statements – as observable in news stories – and asked which factors make individuals more (or less) susceptible to attempts to present opinionated content as “factual news.” Departing from the argument that it is becoming increasingly difficult to tell the difference between either form of information, and pointing to the blurry boundaries between factual information and opinions in the online and social media environment (e. g., Morrow et al., 2022), we discussed the risks of citizens falling victim to “fake news” content, particularly if they struggle to recognise opinionated statements in the first place. Indeed, our findings show that such a differentiation presents a challenge to European citizens, especially for lower-educated, female, and elder individuals, which aligns with prior research from other countries (Graham and Yair, 2023; Merpert et al., 2018; Mitchell et al., 2018; Walter and Salovich; 2021; Seo et al., 2020). In contrast, political efficacy and knowledge increase the likelihood that citizens correctly classify statements, and these effects hold beyond general education levels. In addition, we find that perceptions of disinformation (Hameleers et al., 2022) and a stronger reliance on informal social media sources (see also Luo et al., 2022) make the differentiation more difficult as well.

One may ask whether these results point to a larger pattern in the form of a general problem, where citizens are increasingly skeptical of professional journalistic content on social media. It is worth investigating whether previously learned media literacy skills are difficult to apply in the online environment, also because the latter does not follow the same rules and norms that apply for offline outlets. However, it is also important to point out that in our study a higher level of self-reported media literacy does not relate to the number of correctly classified statements. On the one hand, this seems surprising, given the effectiveness of even short training sessions to raise media literacy skills (Merpert et al., 2018; but see Tully et al., 2020). On the other hand, we assessed media literacy based on only one item asking about respondents’ self-reported ability to distinguish between accurate and false information, instead of more objective measures of media literacy. The self-reported measurement might be less suited to capture actual skills or may suffer from respondents’ over/misestimation of their own literacy level, an assumption that is further confirmed by the comparably low correlation (Pearson’s r = .110) between respondents’ media literacy and general education levels (see Table A4 in the Appendix). Clearly, more research is needed to assess which skills are decisive in this context, and how they can best be addressed in literacy training.

Neither perceptions of misinformation nor respondents’ position on the left-right ideological scale affect their ability to differentiate between factual information and opinions. The latter finding does not align with research on motivated reasoning and cognitive biases (see also Graham and Yair, 2023), yet caution is warranted when interpreting our statements as decidedly in favour of or opposing the EU. Similarly, research has shown that Eurosceptic attitudes can be located on both the left and right sides of the political spectrum (van Elsas and van der Brug, 2015), which makes it difficult to connect a clear ideological leaning to presumably politically motivated biases in our data. Finally, a large variety of political systems, including different competition structures between parties, were included in our pooled country sample; a simple distinction between left- and right-leaning citizens might be insufficient to capture this diversity and might be complemented by more detailed measures of party identification and related strength in future research.

At the same time, several significant relationships between our predictors and the identification between both statement types only became visible when separating the correct cataloguing of factual information from opinions: Higher levels of trust in the news media, for example, make it more likely that European citizens correctly identify factual statements, and less likely that they recognise opinionated sentences (see Figure A2 in the Appendix). The results for political interest point in the same direction, indicating that respondents who trust the news media more and consider themselves more politically interested categorise more statements as factual in general. These findings correspond with Pennycook and colleagues’ (2020) “implied truth effect,” which shows that in the absence of contrary cues, people tend to take information provided by the media as true. Along these lines, we argue that opinions need to be labelled as such; otherwise, at least some citizens tend to automatically take them to be factual. These results indicate that a more skeptical news consumer may be better equipped to question the verifiability of information in the media, but also show that politically interested citizens might have a disadvantage when it comes to the same task.

We also tested whether citizens’ bias in favour of or against the view expressed in the statement in question influences their perception of whether or not it constitutes factual information or an opinion (Walter and Salovich, 2021). Our results show that respondents are slightly more likely to believe statements when they wrongly classified them as factual, in contrast to believing the correctly classified factual statements. This bias is even larger for agreement with opinions, that is, respondents agree much less with (factual) statements they wrongly classified as opinions compared to agreeing with correctly classified opinions. These findings are in line with the theory of motivated reasoning, in the sense that pre-existing opinions and judgements about certain topics influence the categorisation of information into factual information or opinions, especially the dismissal of factual information as “mere” opinion (Gleadon, 1988; Graham and Yair, 2023; Taber and Lodge, 2006; Taber et al., 2009).

Our study does not come without limitations. The selection of statements is linked to the EU and EP elections 2019, which increased comparability across countries. However, low levels of specific EU knowledge (e. g., Hobolt, 2007) may have made it more difficult to classify statements as factual or opinions. Thus, future studies should extend the investigation to test different and a larger variety of statements; the latter should ideally be part of an overall larger number of statements including other topics as well. This may also enable a more detailed within-category analysis, following our results of correct classification differences of 17 and 30 percentage points for factual and opinion statements, respectively. One may also test statements embedded in whole news stories, instead of as isolated sentences. Such future endeavours may increase both the reliability of our results and the external validity of a related study’s design.

Second, although our study goes beyond extant single-country studies and investigates citizens’ ability to distinguish factual information from opinions across a larger set of countries, we did not explore country-specific differences in more detail. Yet, mean differences between countries in Figure A1 and the significant country fixed effects in our regression model (Table 2) highlight that it may be worthwhile to investigate such differences further. For instance, future research may explore whether country-specific settings related to the media system, the level of social media use, or the political context, e. g., regarding polarisation, may be influential. Ideally, such research would also cover a larger and more heterogeneous set of countries to represent the different parts and regional differences across the world. Similarly, the explorative nature of our investigation into citizens’ ability to distinguish factual from opinion statements aimed at establishing the magnitude of the problem first, alongside examining some explanatory factors. We encourage future work to also consider outcomes of these processes, for example by assessing whether miscategorisations of opinions as factual information (and vice versa) affect political attitudes, voting behaviour, or citizens’ perceptions of their political institutions’ legitimacy. Finally, our explanations for the correct identification of statements as factual or opinions worked better for the factual statements (see lower R² for the separate opinion model in Table A2), although we observed no discernible differences in terms of the significance of the included predictors. There may be other relevant factors that are crucial to explain the ability to correctly classify statements which we missed in this study.

Notwithstanding these limitations and recommendations for future research, our study provides an important addition to the extant literature by highlighting the limited ability of citizens to distinguish factual information from opinions across various countries and relying on larger survey samples (representative according to key sociodemographic variables) instead of commonly smaller experimental samples and/or more biased convenience samples. Awareness of these challenges among citizens may be especially relevant for efforts to combat misinformation using fact-checkers, as at least part of the related problems are relevant in the early stages of exposure to information. Our findings show that citizens may classify popular opinions as factual without them ever being checked for accuracy, while opinions are by definition impossible or at least hard to correct and thus escape the radar of fact-checkers.