Introduction

The information overflow characterizing the digital age poses great challenges to learners in terms of selecting relevant information and integrating that information into a coherent representation of a topic, issue, or phenomenon (Barzilai et al. 2018; Bawden and Robinson 2009; Bråten et al. 2020). Accordingly, making sense of diverse documents on a particular issue can be assumed to require considerable strategic effort on the part of learners (Cho and Afflerbach 2017; Cho et al. 2018; List 2020). Of course, digital devices such as computers and tablets not only support navigation among documents but also provide functions for highlighting, extracting, and organizing information from multiple documents. Still, when working with multiple documents, the affordances of typical document reading applications for digital devices (e.g., Adobe Reader) are limited in the sense that they do not explicitly support multiple document tasks, with the display of only one document at a time making it harder to compare and contrast information across documents.

Digital reading technologies may also include applications designed to facilitate working with multiple documents, for example by providing specific functions for accessing multiple documents simultaneously, selecting and extracting information from those documents, and organizing and integrating information across documents. In the current study, we tested the effects of using one such application, LiquidText, on multiple document comprehension when students worked with multiple documents on a large (12.9") iPad Pro tablet with a stylus (see “Device and applications” section). Innovative digital technologies may be more or less accepted by users, however, with acceptance presumably contributing to students’ adoption and subsequent use of such applications in their study work (Alexandre et al. 2018). In the current work, we therefore focused on students’ acceptance of tablets as a multiple document learning tool as a result of using the innovative multiple document reading application, in addition to the effects of using this application on comprehension performance. Finally, because students may need specific guidance to fully exploit the affordances of new technological tools for learning, and because multiple document comprehension is a challenging cognitive activity requiring strategic effort (Cho and Afflerbach 2017; Cho et al. 2018; List 2020), we investigated whether strategy guidance might influence the effectiveness and efficiency with which students used the application in question. However, before we further specify the hypotheses that guided our work, we briefly discuss the theoretical grounding of our study and relevant prior research.

Theoretical and empirical background

Multiple document comprehension

When working with multiple documents on a particular issue, students commonly try to make sense of the information presented across documents and create an external product (e.g., an essay) that can be used to assess how well they have met the task demands (e.g., to complete a writing assignment on the consequences of global warming). In this process, students not only have to select accurate and reliable information from each document but also to restructure this information into a meaningful whole that captures both converging and diverging contents across documents. Within the area of “multiple document literacy” (Bråten and Strømsø 2010), a prominent framework describing the steps in working with multiple documents is the Multiple-Document Task-based Relevance Assessment and Content Extraction (MD-TRACE) model proposed by Rouet and Britt (2011).

This model consists of five steps that characterize working with multiple documents to complete a particular task. Drawing on external (e.g., a task instruction and a document set) and internal (e.g., prior knowledge and reading skills) resources, learners first interpret the task instruction, set goals in accordance with their interpretation, and plan how to proceed in order to accomplish their goals. Then, in the second step, learners judge to what extent external information resources, such as documents, are needed to accomplish their goals, and in the third step, learners select the external information resources deemed necessary and process them. According to the MD-TRACE, the third step is a complex one that includes selecting, extracting, and processing relevant (i.e., accurate and reliable) information from single documents, as well as linking information across documents to construct an integrated understanding of the issue mentioned in the task instruction. Then, in the fourth step, learners create a product that documents their understanding of the issue and addresses the assigned task, with the alignment between this task product and the requirements of the task evaluated by the learners themselves in the final, fifth step of the MD-TRACE. The cyclical nature of the model involves that learners can cycle back to previous steps as they see needed. For example, after creating their task product, learners may find that they have not accomplished their task goals and therefore return to step three in order to select, extract, and process additional information to cover the topic more broadly. When successfully executed, the steps of the MD-TRACE model support the creation of mental representations unique to multiple document comprehension. These include a mental model that integrates semantic content across documents and an intertext model that represents information about the sources of semantic content (Britt and Rouet 2012).

In the present study, we were particularly interested in the third and fourth steps of the MD-TRACE. That is, the strategy guidance that we provided for some of the participants using the multiple document reading application LiquidText targeted processes taking place in the third step, and the effects of this guidance were evaluated in terms of the understanding reflected in their written task products, located in the fourth step of the MD-TRACE. As external resources, we provided all participants with a specific task instruction, a digital device (an iPad Pro), a set of five documents dealing with the issue of sun exposure and health, and an application for working with these documents. In addition, we assessed participants’ internal resources in terms of their prior knowledge, reading skills, and prior beliefs about the issue, to ensure that these individual differences did not bias the results of our experimental manipulations (see “Method” below).

Strategies and strategy guidance in multiple document tasks

The most comprehensive discussion of the role of strategies in comprehending multiple documents was provided by Cho and Afflerbach (2017; see also, Cho et al. 2018). Thus, in their taxonomy of “constructively responsive reading comprehension strategies for multiple digital texts” (Cho and Afflerbach 2017, p. 120), these authors described particular strategies involved in (1) identifying intertextual links and making meaning across texts, (2) monitoring the construction of intertextual relationships, and (3) evaluating and sourcing multiple digital texts. For example, their first category included strategies such as taking notes to record information from different texts, comparing and contrasting information from related texts, and organizing information across texts. Their second category included strategies such as regulating meaning construction strategies in accordance with the task and goal, and noting that multiple texts on the same issue can present conflicting information, complementary information, or both. Finally, in their third category, Cho and Afflerbach (2017) included strategies such as judging the usefulness of information from one text in relation to information from other texts, and assessing the contribution of one or more texts to the task goal. Of note is that Cho and Afflerbach (2017) constructed their taxonomy on the basis of a thorough review of the literature on learning from and comprehending multiple digital documents. It goes without saying that many students may need guidance in acquiring and executing effective strategies in multiple document task contexts (Anmarkrud et al. 2014; Britt and Rouet 2012).

Regarding the strategic guidance provided to students, Barzilai et al. (2018), in a recent systematic review, found that integration of information across multiple documents can be promoted by “teaching students how to integrate by providing them with various forms of strategy instruction” (pp. 985–986). These forms of instruction ranged from guiding students in annotating and summarizing single texts (e.g., Hagerman 2017; Monte-Sano 2011) to organizing information from different texts graphically or spatially (e.g., Hilbert and Renkl 2008; Lundstrom et al. 2015), with a large proportion of effective interventions also providing prompts or explicit instruction that encouraged or directed students to integrate information across documents (e.g., González-Lamas et al. 2016; Maier and Richter 2014). According to Barzilai et al. (2018), the most common ways to assess intertextual integration have been through written products (e.g., essays) and open-ended questions tasks.

In the same vein, Brand-Gruwel and van Strien (2018) reviewed interventions targeting information problem solving, which involves selecting relevant information, comparing and contrasting information across sources, synthesizing information, and structuring and formatting a task product. This review also highlighted the need for strategy guidance to promote such information problem solving skills in students. In the current study, we built on these insights from prior research and designed an intervention that guided students in selecting, organizing, and integrating information from multiple documents when using the multiple document reading application LiquidText.

Specifically, the strategy guidance that we implemented built on the simple, yet powerful Select-Organize-Integrate (SOI) information-processing model of meaningful learning by Mayer (e.g., Mayer 1988, 1996; Mayer and Wittrock 2006). According to the SOI model, strategy guidance needs to target three essential cognitive processes involved in meaningful learning: selecting relevant information, organizing the selected information, and building a coherent mental representation that can be stored in long-term memory. In adapting the SOI model to the current multiple document task context, we therefore guided students in highlighting content in single documents during reading; selecting, extracting, and organizing task-relevant content based on the highlighted information; and integrating information across documents to build a coherent understanding of the issue (see “Method” below). As noted previously, this strategy guidance also aligns with the processes involved in the third step of the MD-TRACE model (Rouet and Britt 2011).

Finally, it should be noted that such strategy guidance, in addition to being effective in promoting students’ comprehension performance, may influence the efficiency with which they perform multiple document tasks. In particular, students provided with strategy guidance based on the SOI model may spend proportionally more time selecting relevant information and organizing and integrating that information across documents and proportionally less time constructing their task products than students not provided with such guidance. This is consistent with much prior research indicating that efficient problem solvers devote a higher proportion of the time they spend on a task to planning in terms of analyzing patterns and constructing mental models and a lower proportion of the time to completing the task or solving the problem (Glaser and Chi 1988). This proposed parallel between performing multiple document tasks and solving problems also seems to be in line with Mayer’s (1996; Mayer and Wittrock 2006) discussion of the SOI model as applicable across different types of problem solving, including making sense of expository text and writing an essay on an unsettled issue. Accordingly, an assignment to write an essay on a particular issue from multiple documents could be described as an ill-defined problem “because the allowable operators are not clear, and to some extent, the goal state is not clear (Mayer and Wittrock 2006, p. 288). This description of multiple document tasks as a form of problem solving is also consistent with the model of reading as problem solving (RESOLV) proposed by Britt, Rouet, and colleagues (Britt et al. 2018; Rouet et al. 2017).

Acceptance of tablets as a learning tool

Introducing innovative technological tools in work and learning contexts does not guarantee that users accept them. Students’ acceptance of new tools may contribute to the implementation of those tools and support their use in completing learning tasks. Therefore, trying to identify the likelihood that students will integrate and appropriate particular technological learning tools is important because it can facilitate the adaptation of the technology to students’ needs. Acceptance of technologies is usually assessed before their implementation in ecological contexts, with acceptance judged on the basis of the perceptions of future target users (Alexandre et al. 2018). That is, acceptance measures try to capture individuals’ perceptions of a particular technology before or after brief use of the technology. Accompanying research on the acceptance of innovative tools in teaching and learning contexts, different theoretical frameworks have been developed to describe perceptions of technologies that may determine users’ adoption of and intention to use the technologies (for an overview, see Gunasinghe et al. 2019). Regarding acceptance of educational technologies, the most prominent theoretical frameworks are the Technology Acceptance Model (TAM; Davis et al. 1989) and the Unified Theory of Acceptance and Use of Technology (UTAUT; Dwivedi et al. 2019; Venkatesh et al. 2003). Although these frameworks differ somewhat with respect to their structure and the psychological and contextual factors that are considered to impact users’ perceptions and intentions, they both highlight the importance of perceptions concerning instrumental qualities, specifically concerning the ease of use and the usefulness of technologies. While perceived ease of use refers to perceptions regarding the effort required to use a technology (e.g., difficulty and cost of use), perceived usefulness refers to perceptions regarding the benefits of using a technology for task performance.

Recent reviews of the acceptance of mobile learning tools in general and tablets in particular, have found good acceptance among both college (Al-Emran et al. 2016; Nguyen et al. 2015) and primary and high school students (Mulet et al. 2019). Still, Mulet et al. (2019) emphasized the need to consider specific learning contexts and tasks when assessing the acceptance of tablets. Some previous questionnaire studies that examined the acceptance of tablets without targeting specific contexts or tasks found high levels of perceived ease of use, perceived usefulness, and intention to use a tablet (Cacciamani et al. 2018; Dündar and Akçayir 2014; Ifenthaler and Schweinbenz 2016; Lin 2014; van Deursen et al. 2016). However, other studies, which used interviews focusing on particular contexts and tasks, found that students expressed negative perceptions in some instances (Duran and Aytaç 2016; Montrieux et al. 2014). For example, students reported that tablets could cause problems by diverting attention (Duran and Aytaç 2016; Kontkanen et al. 2017) or that using tablets could be problematic when the task involved taking notes and producing text using a virtual keyboard (Dündar and Akçayir 2014; Soffer and Yaron 2017).

In the present study, we were interested in students’ acceptance of tablets as a multiple document learning tool rather than their acceptance of tablets as a tool for learning more generally. Amadieu et al. (2016) found that after brief use of a tablet, students’ perceptions changed depending on the specific task they performed and the compatibility of that task with the multi-touch function of the tablet. Thus, after reading a multimedia document, their perceived usefulness and intention to use the tablet increased, but when performing a writing task using a virtual keyboard, their perceived usefulness and intention to use the tablet decreased. This finding suggests that the extent to which the functions of an application support a particular learning task may influence students’ acceptance of the technology in that task context. Accordingly, Jeno et al. (2019) found that an application adapted to a specific task led to higher acceptance as well as better performance. Those authors compared a traditional textbook, a digital textbook, and a mobile-learning application that provided multimedia information and facilitated access to information as tools for learning particular biology contents. The results indicated that the innovative application led to not only higher performance but also higher acceptance in terms of perceptions of ease of use and usefulness. Perceptions related to acceptance may thus differ dependent on the suitability and benefits of particular tools and applications for performing particular learning tasks.

The Present Study

Given this background analysis, we investigated the effects of a multiple document reading application for tablets, LiquidText, on students’ multiple document comprehension and their acceptance of the tablet as a tool for working with multiple documents. In addition, we investigated the effects of guiding students in using the functions of the application on their multiple document comprehension and acceptance of the tablet as a multiple document learning tool. To achieve this aim, we compared students working with multiple documents on an iPad Pro tablet using the LiquidText application with control students working with the same documents on an iPad Pro using the Adobe Reader application in terms of comprehension performance and acceptance of the tablet. However, only one group of students using LiquidText were given explicit guidance in using all the functions of the application, that is, functions for accessing more than one document simultaneously, as well as for selecting, extracting, and reorganizing information from those documents in order to construct an integrated understanding of the document contents. We termed this group the LiquidText Guidance group. One other group of students using LiquidText were only able to access documents simultaneously, without having the other functions available, and yet another group of students were free to use all the functions of LiquidText but were not given any explicit guidance in using them. We termed these two groups the LiquidText Access group and the LiquidText Non-Guidance group, respectively. Finally, because we also collected process data in terms of the time used for processing the documents (cf., step 3 of the MD-TRACE) and creating the task product (cf., step 4 of the MD-TRACE), we were also able to investigate potential differences between the groups in how they distributed their time across these two phases of the multiple document task,

The hypotheses tested in this study were:

H1a

Students using the LiquidText application were expected to outperform students using the Adobe Reader with respect to multiple document comprehension.

H1b

Among the students using the LiquidText, students in the LiquidText Guidance group were expected to perform better than were students in the two other groups (i.e., the LiquidText Access group and the LiquidText Non-Guidance group) with respect to multiple document comprehension.

H2a

Acceptance of tablets as a multiple document learning tool was expected to differ between students using the LiquidText application and students using the Adobe Reader, with students using LiquidText reporting greater acceptance of tablets in this regard.

H2b

Among the students using the LiquidText, students in the LiquidText Guidance group were expected to report greater acceptance of tablets as a learning tool than were students in the two other groups (i.e., the LiquidText Access group and the LiquidText Non-Guidance group).

H3

The students in the LiquidText Guidance group were expected to use longer time to process the documents and shorter time to create the task products than were students in the Adobe Reader group and the two other groups using the LiquidText application (i.e., the LiquidText Access group and the LiquidText Non-Guidance group).

Method

Participants

Participants were 88 students (80% female) enrolled in an undergraduate psychology program at a public university in southern France. The students ranged in age from 18 to 51 years and had an overall mean age of 22.6 years (SD = 5.16). Twenty-two participants were enrolled in the first year of the program, and the rest were enrolled in the third year. Thirty-four participants reported that they owned a tablet, with most of these tablets having a regular screen size (i.e., 9–10 in.). Moreover, 29% of participants’ tablets used an Apple iOS operating system, while 29% and 41%, respectively, used Windows and Android as operating systems. Participation in the study was anonymous and voluntary and remuneration took form of a 0.5 point course credit. A Welch’s analysis of variance (ANOVA) with age as the dependent variable showed that the four conditions (see below) were statistically equivalent with respect to age, Welch’s F(3, 44.8) = 0. 84, p = .477.

Design and Conditions

The study used a between-subjects experimental design in which participants were randomly assigned to one of four conditions, with participants in three conditions using the multiple document reading application LiquidText and participants in one (control) condition using the traditional reading application Adobe Reader. All participants in all conditions worked with the same set of five documents on the controversial socio-scientific issue of sun exposure and health on an iPad Pro and afterwards wrote an explanation of the issue in response to a prompt. The time used for processing the documents and the time used for writing were recorded for all participants, and all participants responded to questions regarding their acceptance of tablets as a multiple document learning tool after they completed the writing task (see Materials section).

In the condition termed LiquidText Guidance, participants (n = 21) were guided to use the innovative functions of the multiple document reading application when performing the task in accordance with the SOI-model of meaningful learning by Mayer (e.g., Mayer 1988, 1996; Mayer and Wittrock 2006). This means that they were explicitly guided to (1) highlight information from single documents that were simultaneously accessed, (2) select and extract task-relevant information based on their highlighting, and (3) conceptually and spatially organize and integrate information across multiple documents, with all these processes afforded and supported by the LiquidText application.

In the condition termed LiquidText Access, participants (n = 22) could use the multiple document reading application to access more than one document simultaneously and highlight information within each of those documents, without being able to use the other innovative functions provided by the application. This means that they could not use the functions designed to support the selection and extraction of task-relevant information from single documents, or the functions designed to support the organization and integration across documents (for further description of the innovative functions of the LiquidText application, see “Device and applications” section).

In the condition termed LiquidText Non-Guidance, participants (n = 24) could use all the innovative functions provided by the multiple document reading application but were not explicitly guided to use any of these functions. This means that they were free to not only access more than one document simultaneously and highlight information within each of those documents, but also to select and extract task-relevant information from single documents and to organize and integrate information across documents. Please note that all participants using the LiquidText application, including those in the LiquidText Non-Guidance condition, were informed about all the innovative functions of the application before starting to work on the documents (see “Procedure” below).

Finally, in the control condition, participants (n = 21) used a traditional Adobe Reader application, which supported sequential access to single documents and provided functions for highlighting and annotating those documents. Participants in this condition were free to use all the functions provided by the Adobe Reader but were not given any explicit guidance in using them.

Our reasons for including use of a traditional pdf viewer that only allowed for sequential access to and reading of single documents as a control condition are twofold. First, because the LiquidText Access condition included some functions (i.e., simultaneous access to multiple documents) that came in addition to functions available for this pdf viewer (and that we could not remove technically), we needed a more basic control condition in which those functions were removed and that therefore allowed for only sequential reading of single documents. Second, including such a traditional pdf viewer as a control was also considered to add ecological validity to our study because this is presumably what many people use to read documents on a daily basis.

Materials and Measures

Documents

The document set included five separate documents on the controversial issue of sun exposure and health (Moan et al. 2012). The documents were based on authentic source materials derived from the Internet and represented partly conflicting views on the issue, including the view that sun exposure is healthy due to the production of vitamin D and the view that sun exposure is harmful due to the risk of developing skin cancer. The document set was originally developed in Norwegian and has been used in prior research on multiple document comprehension (e.g., Bråten et al. 2014; Ferguson and Bråten 2013). The documents used in the current study were French adaptations of the documents developed and used by Bråten and colleagues. Information about the source, such as author, author credentials, document type, and publication, was presented at the beginning of each document, with the current documents adapted such that they came from sources that corresponded to the original ones (e.g., the French Cancer Association was used as a parallel to the Norwegian Cancer Association, and the French newspaper Le Monde was used as a parallel to the Norwegian newspaper Aftenposten).

Specifically, the first document was a 469-word excerpt from a textbook that explained various forms of ultraviolet radiation in neutral academic terms. The second document was a 503-word popular science article from a university research magazine arguing that sun exposure can protect against cancer because it increases the production of vitamin D. The third document was a 473-word popular science article from an online research magazine arguing that sun exposure is harmful because it can cause skin cancer. The fourth document was a 414-word newspaper article presenting evidence that sun exposure can protect against all forms of cancer because of increased vitamin D production. Finally, the fifth document was a 469-word document from the French Cancer Association describing how sun exposure can lead to different forms of skin cancer, as well as explaining different ways people can protect themselves against the risk of developing skin cancer. Thus, apart from the first document, the four other documents presented partly conflicting views on the issue of sun exposure and health that are also represented in the scientific literature on the issue (Moan et al. 2012). Of note is that all documents could be considered to provide relevant and reliable information about the issue. An overview of the five documents is provided in Table 1.

Table 1 Overview of the five documents
Full size table

We used the Gunning fog index (Gunning 1952) as an indication of text difficulty. The index is based on the average sentence length and the percentage of complex words (i.e., words consisting of three or more syllables), resulting in values ranging from 6 (easy text) to 17 (difficult text). The documents used in this study ranged from 8.30 to 14.00 (M = 11.46, SD = 1.93), suggesting that the difficulty level was suitable for undergraduate readers.

Device and applications

All participants, regardless of condition, worked with the documents on a first generation iPad Pro tablet with a 12.9-inch (33 cm) screen size developed by Apple Inc. However, whereas participants in the control group used the Adobe Reader application developed by Apple Inc., which is a traditional pdf viewer mainly adapted to accessing one document at a time and to highlighting and annotating information within each document, participants in the three other conditions (i.e., LiquidText Guidance, LiquidText Access, and LiquidText Non-Guidance) used the reading application LiquidText developed by LiquidText Inc., which includes several additional functions adapted to working with multiple documents. Specifically, compared to the LiquidText application, the Adobe Reader presumably made it harder to compare and contrast information across documents because learners were required to retain the information they might want to compare and contrast in memory while switching between documents. Also, the Adobe Reader did not offer any functions for selecting and extracting highlighted information from single documents for the purpose of cross-document organization and integration. During the writing phase of the multiple document task, the screen of the iPad Pro was divided into two halves for participants in the control group, with document information available on the left side and space for creating the written task product available on the right side (see Fig. 1). Participants could zoom in and out on document information on the left side as well as on the written task product on the right side.

Fig. 1
figure 1

Screenshot of the Adobe Reader application

Full size image

The LiquidText application can be assumed to support processes highlighted within multiple document comprehension (Rouet and Britt 2011) through additional functions providing overview over and access to multiple documents simultaneously, selection and extraction of information, and conceptual and spatial organization and integration of information across documents. As can be seen in Fig. 2, the various functions of the Liquid Text correspond to different areas of the iPad Pro screen when using the application for a multiple document task. Thus, in the left area, an overview of all the documents is presented to facilitate navigation among documents; in the middle area, up to three documents can be viewed simultaneously and information deemed important in each document can be highlighted; and in the right area, a workspace allows learners to organize and integrate information that has been extracted from the documents viewed in the middle area and moved to the workplace. In the workplace, learners can not only group and link extracted information to construct an integrated understanding but also create a textbox in which they write their task product on the basis of the extracted and (re)organized information (see Fig. 2). A zoom function made it possible to zoom in and out on documents as well as on information moved to the workspace and the textbox used for writing.

Fig. 2
figure 2

Screenshot of the LiquidText application

Full size image

Multiple document comprehension measure

Multiple document comprehension was assessed by prompting participants to explain the relationship between sun exposure, health, and illness, as well as the different perspectives on that relationship. Specifically, participants responded in writing to three open-ended questions that were modeled on the integrative short-essay questions designed by Rukavina and Daneman (1996) to measure comprehension of a controversial scientific issue. These questions also have been used to measure multiple document comprehension in prior research (e.g., Andresen et al. 2019; Barzilai and Ka’adan 2017; Ferguson and Bråten 2013). The first question is: “Can you explain the relationship between sun exposure, health, and illness?” The second question is: “Can you describe the different views that exist on the relationship between sun exposure, health, and illness?” The third and final question is: “Can there be more than one correct view on the relationship between sun exposure, health, and illness?” Following Rukavina and Daneman (1996), we considered the first question to indirectly require participants to integrate different views across documents, and the second and third questions to directly require them to compare and contrast different views on the issue. Thus, taken together, the three questions could be assumed to capture how well participants were able to reason about the issue in terms of the claims, reasons, and evidence provided across documents (see also, Kuhn 2010). This interpretation is also supported by other research using these questions to measure multiple document comprehension in which participants’ scores have been predicted by efforts to justify knowledge claims across multiple sources and deeper-level intertextual strategies used to compare, contrast, and integrate information across documents (Bråten et al. 2014; Ferguson and Bråten 2013).

Participants’ scores on the first two questions could range from 0 to 6, and on the third question, they could range from 0 to 5. In accordance with the scoring system established in prior research (for details, see Andresen et al. 2019; Barzilai and Ka’adan 2017; Bråten et al. 2014; Ferguson and Bråten 2013), top scores (i.e., 5 or 6) were awarded when participants provided explanations or reasons for each view on the issue as well as related the different views to each other by comparing, contrasting, or trying to reconcile them. For example, on the third question, participants were given a score of 5 if they acknowledged that sun exposure could be both healthy and harmful, provided explanations or reasons for the different views (due to vitamin D production and risk of skin cancer, respectively), and related the different views to each other by describing how they might be reconciled (e.g., sun exposure may be more healthy than harmful when using sun screen). Only participants’ total scores on the multiple document comprehension measure, potentially ranging from 0 to 17, were used in subsequent statistical analyses. All participants’ written responses were independently scored by two raters, yielding a Pearson’s inter-class correlation (ICC) of .924.

Acceptance measures

To assess participants’ acceptance of tablets as a multiple document learning tool, we administered three measures based on acceptance measures developed and used within the Technology Acceptance Model (TAM; Davis et al. 1989) and the Unified Theory of Acceptance and Use of Technology (UTAUT; Dwivedi et al. 2019; Venkatesh et al. 2003). Specifically, the measures that we used were adapted from an instrument originally designed by Davis et al. (1989) and previous research on the acceptance of tablets in education (Moran et al. 2010; van der Linden et al. 2019). Because the learning task in the present study concerned the comprehension of multiple documents, all items included in the three measures targeted the use of tablets for performing multiple document tasks. On all items, participants rated their acceptance on a 10-point scale ranging from strongly disagree (1) to strongly agree (10).

Six items focused on the perceived ease of using tablets when performing multiple document tasks (sample item: “I find tablets easy to use for comparing multiple documents”).

The reliability estimate (Cronbach’s α) for participants’ scores on this six-item measure was .86. Moreover, five items targeted the perceived usefulness of tablets when performing multiple document tasks, that is, the extent to which participants perceived that using tablets in such a task context would enhance their performance (sample item:”Using a tablet is beneficial when comparing multiple documents”). The reliability estimate (Cronbach’s α) for participants’ scores on this five-item measure was .91. Finally, two items were used to assess participants’ intention to use a tablet when performing future multiple document tasks (sample item: “I intend to continue using a tablet when studying documents in order to compare them”). The reliability estimate (Cronbach’s α) for participants’ scores on this two-item measure was .89.

Covariates

We included measures of prior knowledge, reading comprehension, and topic beliefs as potential covariates.

Prior knowledge

We assessed participants’ prior knowledge about the topic of sun exposure and health by means of a 12-item multiple-choice measure. This measure was a French adaptation of a measure that has been used and validated in several studies (Andresen et al. 2019; Bråten et al. 2014; Ferguson and Bråten 2013). Taken together, the items covered both factual information and conceptual understanding concerning sun exposure and health that were discussed in the document set (e.g., vitamin D production, skin cancer, and sun protection). Participants’ scores were the number of correct responses (maximum score = 12). The internal consistency reliability (Cronbach’s α) was .58. Although lower than desirable, this estimate may be considered acceptable when a measure is used for research purposes (Hair et al. 2006; Kerlinger and Lee 2000). Thus, deciding whether a reliability estimate is acceptable or not can be considered to depend on how the measure is used and what type of decision will be based on the measurements (Kerlinger and Lee 2000). In our case, the purpose was to develop a measure that could be useful in research on students’ reading of multiple documents about sun exposure and health rather than in making any important and irreversible decisions concerning individuals. The low reliability of the prior knowledge measure likely was related to the fact that different aspects of the topic (e.g., vitamin D production and sun protection) were covered by the items.

Reading comprehension

We assessed participants’ reading comprehension by means of a French version of a standardized reading comprehension test developed and validated by Martínez and colleagues (Martínez et al. 2008, 2009). This test requires the identification of main ideas, bridging inferences, elaborative inferences, and the construction of macro-ideas (Martínez et al. 2008). In this study, participants read a 577-word French translation of an expository text about penguins and answered 10 multiple-choice questions with four response alternatives designed to capture the component skills described above. Participants’ scores were the number of correct responses (maximum score = 10). Because the internal consistency reliability was very low in the present sample (Cronbach’s α = .41), we did not include this variable in further statistical analyses.

Topic beliefs

Because research has shown that learners’ prior beliefs about a topic may influence their processing of information and, in particular, the likelihood that they construct integrated understandings from documents presenting conflicting views (for review, see Bråten and Strømsø 2020), we assessed participants’ prior beliefs about the topic of sun exposure and health by means of a 5-item measure. On this measure, participants rated their agreement with statements about the potential risk associated with sun exposure on a 10-point scale (1 = not at all true, 10 = very true). Thus, high scores on this measure indicated that participants judged such exposure to be harmful (sample item: “Sun exposure is dangerous for your health”). Because the internal consistency reliability was very low in the present sample (Cronbach’s α = .40), we also did not include this variable in further statistical analyses.Footnote 1

Procedure

The data for this study were collected individually in two one-hour sessions with one day between the sessions. In the first session, each participant completed the prior knowledge measure, the reading comprehension measure, and the topic belief measure online by means of the Qualtrics survey software. Afterwards, they were informed about and explored all the functions of the LiquidText or the Adobe Reader (dependent on the condition). Specifically, participants received an overview of all the functions of their assigned application in the form of a check list, with each function included in the list ticked as participants used the application when working with two documents on another topic (global warming). During the training task, participants used all the functions of their respective application when working on the two documents, as ensured by the completion of the check list, but were not asked to create a task product.

In the second session, participants worked individually with the multiple document task using the document set on sun exposure and health described previously. In all conditions, the multiple document task was introduced with a scenario in which a fictitious friend asked for participants’ opinion regarding the healthiness or harmfulness of sun exposure before going to Mexico on a vacation. Based on this scenario, participants were told that an Internet search on the issue had resulted in five documents being selected and saved in the application for them to study in order to write an answer to their friend. Participants were also told that the documents would be available during writing, and that their answer should (1) explain the relationship between sun exposure, health, and illness; (2) describe the different views that exist on the relationship between sun, exposure, health, and illness; and (3) explain whether there can be more than one correct view on the relationship between sun exposure, health, and illness.

In addition, before starting on the multiple document task, participants in the LiquidText Guidance condition were instructed to use the functions of the application to access documents and highlight important information in each document during reading, select and extract task-relevant information in each document and move it to the workspace, and organize and integrate extracted information in the workspace by comparing and contrasting information across documents, such as by spatially grouping document contents that were conceptually consistent (i.e., regarding the healthiness vs. harmfulness of sun exposure; see Fig. 2). In addition to being reminded about these functions as utilized during the training task, the participants in this condition were provided with a cue card that reminded them about the steps of this procedure. Thus, participants in the LiquidText Guidance condition were only asked to use the functions of the application in selecting, organizing, and integrating information, and reminded about the use of these functions, before starting on the multiple document task, but the cue card describing the steps of the procedure was available for consultation during the entire task. No parts of the documents were highlighted in advance.

Participants in the LiquidText Access condition were instructed to use the access and highlighting functions of the application but not the workspace. That is, these participants should not select and extract information from the documents and move it to the workspace for further organization and integration.

Participants in the LiquidText Non-Guidance condition were told that they were free to use all functions of the application as they saw fit but were not instructed to use any of them.

Finally, participants in the control group were told that they were free to use all the functions of the Adobe Reader but were not instructed to use any of them.

Of note is that both the LiquidText and the Adobe Reader were used without any changes in the computer codes, which means that participants in the LiquidText Access condition were just told that they should not use certain functions of the application (but could theoretically use them), and that it was not possible for participants in the control group to open different documents in multiple windows simultaneously. Two complementary methods were used to ascertain that participants in each condition adhered to their respective instructions: (a) the experimenter supervised participants’ activities during the experimental session, and (b) screencast recordings were checked retrospectively to ensure that the instructions had been followed.

After processing the documents, participants wrote their answers to the fictitious friend guided by the three integrative questions, with participants using the LiquidText writing in a textbox created within the workspace and participants using the Adobe Reader using the right half of the screen for writing (see Figs. 1 and 2). Finally, all participants completed the acceptance measures using the Qualtrics online survey software.

Data Analysis

We used one-way between-subjects ANOVAs to perform a preliminary analysis of possible differences between conditions on the prior knowledge measure, as well as to test our hypotheses regarding comprehension, acceptance, and processing. The results of the evaluation of the assumptions for performing these ANOVAs were satisfactory with respect to normality and homogeneity of variance. To control the familywise error rate in these analyses, we used the Ryan-Einot-Gabriel-Welsh F (REGWF) procedure, a modification of the Student–Newman–Keuls (SNK) test that is a conservative stepdown pairwise comparison procedure (Toothaker 1993).

Results

The preliminary one-way ANOVA indicated that there were no differences between the four conditions with respect to prior knowledge about the topic, F(3, 84) = 0.76, p = .519. Because there were also no correlations between participants’ scores on the prior topic knowledge measure and their scores on the multiple document comprehension measure (r = .17, p = .109) or the acceptance measures (rs < .09, ps > .421), we did not include prior topic knowledge as a covariate in subsequent statistical analyses (Tabachnick and Fidell 2014),

Next, we performed a one-way ANOVA to test our hypotheses regarding effects of conditions on the multiple document comprehension measure. This analysis showed that the conditions differed statistically significantly with respect to multiple document comprehension, F(3, 84) = 7.46, p < .001, ηp2 = .210. Post-hoc comparisons using the REGWF procedure showed that participants in the LiquidText Guidance condition (M = 12.27, SD = 1.77) scored statistically significantly higher than did participants in the LiquidText Non-Guidance condition (M = 10.42, SD = 2.09, Cohen’s d = 0.96), the LiquidText Access condition (M = 10.34, SD = 1.82, Cohen’s d = 1.08), and the control condition (M = 9.38, SD = 2.34, Cohen’s d = 1.41), whereas there were no statistically significant differences (p > .05) between the three last mentioned conditions (see Fig. 3). This partly confirmed our hypotheses in that the LiquidText Guidance condition performed much better than all other conditions (H1b). However, our expectation that the two other LiquidText conditions would also outperform the control condition (H1a) was not confirmed. Of note is that the effect size for the difference between the LiquidText Guidance group and the control group was larger than the effect sizes obtained for integration of information across multiple documents in 20 of the 21 intervention studies reviewed by Barzilai et al. (2018), and that the effect sizes for the differences between the LiquidText Guidance group and all the other groups were large even according to the effect size benchmarks suggested by Cohen (1988) for the entire domain of social science.

Fig. 3
figure 3

Multiple document comprehension by condition. Error bars represent standard errors

Full size image

We performed three separate one-way ANOVAs to test our hypotheses regarding the effects of conditions on the acceptance of tablets as a learning tool (for descriptive information, see Table 2). The first analysis, using perceived ease of use as the dependent variable, did not indicate any statistically significant differences between the conditions, F(3, 83) = 0.62, p = .605.

However, when perceived usefulness was used as the dependent variable, the conditions were found to differ statistically significantly, F(3, 84) = 4.43, p = .006, ηp2 = .137. Post-hoc comparisons with the REGWF procedure showed that participants in the LiquidText Guidance condition (M = 4.72, SD = 1.78) perceived the tablet to be less useful than did participants in the LiquidText Non-Guidance condition (M = 6.01, SD = 1.84, Cohen’s d = 0.71), the LiquidText Access condition (M = 6.05, SD = 2.22, Cohen’s d = 0.67), and the control condition (M = 6.82, SD = 1.66, Cohen’s d = 1.22), whereas there were no statistically significant differences (p > .05) between the three last mentioned conditions.

Table 2 Means and standard deviations for the acceptance measures by condition
Full size table

Likewise, when using intention to use a tablet when performing future multiple document tasks as the dependent variable, there were statistically significant differences between the conditions, F(3, 83) = 4.21, p = .008, ηp2 = .132. Post-hoc comparisons using REGWF showed that participants in the LiquidText Guidance condition (M = 3.86, SD = 2.92) scored lower than the control group (M = 6.75, SD = 2.25, Cohen’s d = 1.12), as did participants in the LiquidText Non-Guidance condition (M = 4.73, SD = 2.64, Cohen’s d = 0.83) and the LiquidText Access condition (M = 4.70, SD = 2.92, Cohen’s d = 0.79). There were no statistically significant differences (p > .05) between the three conditions using the LiquidText, however. Thus, contrary to our hypothesis (H2a), the control group displayed much higher acceptance in terms of both perceived usefulness and intention to use a tablet than did the LiquidText Guidance group. Also, the two other LiquidText conditions did not fall in-between with respect to the acceptance of tablets, as we had hypothesized (H2b).

Finally, we tested our hypothesis that participants in the LiquidText Guidance condition would use longer time to process the documents and shorter time to create their task products than would participants in the other conditions. A one-way ANOVA with total processing timeFootnote 2 as the dependent variable showed that the conditions differed statistically significantly, with F(3, 78) = 3.82, p = .013, ηp2 = .128. Post-hoc comparisons using REGWF showed that participants in the LiquidText Guidance condition used longer time to study the documents (M = 30.62, SD = 6.86) than did participants in the LiquidText Access condition (M = 25.63, SD = 6.74, Cohen’s d = 0.73) and the control condition (M = 23.70, SD = 6.39, Cohen’s d = 1.04). However, participants in the LiquidText Non-Guidance condition (M = 25.95, SD = 7.23) did not differ statistically significantly from the three other conditions (see Fig. 4).

Fig. 4
figure 4

Processing and writing times by condition. Error bars represent standard errors

Full size image

When total writing time, that is, the time spent creating the task products, was used as the dependent variable, a one-way ANOVA showed that the conditions differed statistically significantly, F(3, 72) = 3.89, p = .012, ηp2 = .139. Post-hoc comparisons with REGWF showed that participants in the LiquidText Guidance condition used shorter time to create their task products (M = 19.71, SD = 3.69) than did participants in the control group (M = 25.33, SD = 5.30, Cohen’s d = 1.25) but did not differ statistically significantly from participants in the LiquidText Non-Guidance (M = 21.95, SD = 4.80) and LiquidText Access (M = 21.74, SD = 5.70) conditions. There were no statistically significant differences between the conditions with respect to the time used for the entire multiple document task, F(3, 78) = 0.45, p = .717.

The processing and writing time data were consistent with our hypothesis (H3) in that participants in the LiquidText Guidance condition used the longest time studying the documents and the shortest time writing their responses, with large and consistent differences in the hypothesized directions observed between this condition and the control condition. Although there were also consistent differences in the hypothesized directions between the LiquidText Guidance condition and the two other LiquidText conditions, not all these differences were statistically significant.

Discussion

In this study, we extended prior research on multiple document literacy as well as on learning with tablets by investigating students’ comprehension and acceptance when using tablets to perform a multiple document literacy task. Specifically, we examined whether students guided to use the innovative functions of a multiple document reading application implemented in an iPad Pro tablet would outperform control students who used a traditional pdf viewer in terms of multiple document comprehension and, accordingly, would display higher acceptance of tablets as a multiple document learning tool. Following work within multiple document comprehension, we expected that guidance in using the innovative functions would improve students’ comprehension performance relative to controls because those functions supported processes highlighted within theoretical frameworks focusing on the successful completion of multiple document literacy tasks (Rouet and Britt 2011; see also, List and Alexander 2019). Moreover, following work on the acceptance of technological learning tools, we expected that students guided to use the functions adapted to working with multiple documents would display higher acceptance of tablets than would control students in this task context because the former experienced that the application they used was better suited to and beneficial for performing the multiple document task (Amadieu et al. 2016; Jeno et al. 2019).

Regarding multiple document comprehension, the results showed that the students guided in their use of the multiple document reading application were much better able to create a written task product that reflected an integrated understanding of the issue in question than were the control students. This supports the assumption that providing students with innovative functions well aligned with the processing requirements of multiple document comprehension may promote performance in a multiple document task context, compared to providing students with more traditional functions better aligned with sequential reading of single documents. As such, this finding both supports a conceptualization of multiple document processing consistent with the MD-TRACE model (Rouet and Britt 2011) and indicates that a tablet with an application adapted to multiple document reading may be used productively by students when performing a multiple document task.

However, our findings also showed that providing students with a device and an application suitable for performing a multiple document task was not enough, with only students guided in using the functions of that application outperforming the control students in terms of multiple document comprehension in this study. That is, neither the students who could use these functions at their own discretion, nor the students who could use only some of these functions, performed better than the control group, although both groups of students were informed about and practiced using all the functions in the context of a training task. In other words, to reap the potential benefits of the innovative functions of the multiple document reading application, students seemed to need explicit guidance in using selection, organization, and integration processes essential to meaningful learning from and with multiple documents (Mayer 1996; Rouet and Britt 2011). Without being able or willing to exploit the possibilities that such functions allow for in terms of multiple document processing, the consequences of using this application were indistinguishable from those of using a traditional application adapted to single document reading. This finding is consistent with the value theorists have placed on extensive strategic processing of multiple documents (Cho and Afflerbach 2017; Cho et al. 2018), as well as with prior intervention research demonstrating the benefits of guided strategic processing in multiple document literacy contexts (Barzilai et al. 2018; Brand-Gruwel and van Strien 2018).

Regarding acceptance of tablets as a multiple document learning tool, the finding that the control group displayed much higher acceptance of the tablet in this task context than did students guided to use the innovative functions of the multiple document reading application gave us a pause. Thus, the students given strategy guidance did not seem to take their good performance in terms of multiple document comprehension or the suitability or functionality of the application into account when completing the measures concerning perceived usefulness and intention to use a tablet for multiple document tasks. There are several possible explanations for this unexpected finding.

For example, students given guidance in using the functions of the multiple document reading application might have found the tablet more difficult to use than did the control students who worked with a more familiar application, which also might have influenced the perceived competence (Harter 1998) or the self-efficacy (Bandura 1997) of the former students negatively. This explanation seems unlikely, however, given that there was no difference between these two groups with respect to perceived ease of use. Still, motivational factors might have influenced their responses on the acceptance measures. In particular, because the guided group was actually directed to use all the functions of the multiple document reading application, the students in this group might have experienced a lack of autonomy in this task context. In turn, the sense of external control and absence of choice associated with reduced autonomy might well have influenced their perceptions and future intentions negatively, in accordance with the self-determination theory of human motivation (e.g., Deci and Ryan 2000).

One might still ask why such potentially negative motivational factors were not offset or outweighed by factors working in the opposite direction, most notably these students’ level of performance and the suitability of the provided tool for fulfilling their learning needs. Regarding level of performance, however, the guided students may have failed to monitor their task completion or, alternatively, failed to attribute their performance to their use of the guided procedure. Thus, external feedback, not only regarding their level of performance but also regarding the mechanism underlying their performance, may be needed to boost students’ acceptance of the tool. Regarding the suitability of the tool for fulfilling students’ learning needs, it is an open question whether these students really experienced any learning needs in the current task context. As noted by Bråten et al. (2018), students typically do not care much about or engage in complex experimental multiple document tasks that they are asked to perform without any authentic purpose or consequences. Possibly, then, the guided students’ acceptance of tablets as a multiple document learning tool might have been quite different if given feedback about their performance in a learning context that really mattered for them. Needless to say, further research is required to explore these suggestions, as well as the suggestion that the guided students might have been frustrated by a lack of autonomy in the task context. Of note is that the performance-acceptance paradox that we observed among the guided students is similar to the “performance-preference” paradox described previously (Amadieu and Tricot 2014; Oviatt and Cohen 2010), highlighting the lack of consistency between learning outcomes and preference for digital learning tools in educational contexts. The reason we chose to use “acceptance” rather than “preference” in the present article is that we used specific acceptance measures developed within theories on the acceptance of technology (see “Method” section).

Students who could use the innovative functions at their own discretion or could use only some of these functions also displayed less acceptance than did the control students. In those cases, students might simply not have seen the point of having a range of new functions of a less familiar application available without being given any guidance in using them or without being allowed to use them, respectively.

Finally, as we had expected, students guided in using the innovative functions when working with the documents displayed a more expert-like distribution of their time across processing the documents and creating the task product than did the control students. Although not always statistically significant with this sample size, similar differences were observed between the group of guided students and the two other groups using the multiple document reading application. Thus, consistent with previous portraits of efficient problem solvers (Glaser and Chi 1988), the guided students may be assumed to have spent more time on planning, analysis, and mental model construction than did the other students, as well as less time on responding to the task assignment. Not only was the guidance effective in promoting students’ comprehension performance, then, it also seemed to increase the efficiency with which they worked with the documents.

It might be argued that it was the guidance in and of itself rather than the combination of the multiple document reading application and guidance according to the SOI model that made participants in the LiquidText Guidance condition perform the multiple document task more effectively and efficiently than did participants in the other conditions. While this is a possibility that we cannot entirely rule out given the design of our study, it should be noted that the guidance we provided was uniquely suited to the functions of the multiple document reading application. That is, the selection, organization, and integration processes that were guided in this study required not only simultaneous access to multiple documents but also a workspace in which selected information from those documents could be organized and integrated. Therefore, this form of guidance could not have been implemented with a traditional reading application to rule out the alternative interpretation, simply because such an application would not allow for or support the guided multiple document reading processes that we targeted in this study. Still, future research should try to further disentangle the effects of guidance and the functionality of reading applications on multiple document comprehension processes and products.

Another possibility is to include a condition in which participants use a version of the Adobe Reader that allows for accessing and reading multiple documents simultaneously, or a version of the LiquidText without any functions beyond those included in a traditional pdf viewer, and then compare this enhanced Adobe Reader or stripped LiquidText application with other LiquidText conditions with more functions available. Arguably, including such conditions might make it easier to disentangle potential effects of particular applications from the effects of available functions. While our findings obviously should be interpreted in the context of the conditions that we did include in the current design, we believe they, due to the large effects, merit attention from educators (at least) as much as from researchers in the field of multiple document literacy.

Although further work is certainly needed to better understand how the functions and use of a multiple document reading application, such as the one we investigated, may be optimized, we are enthusiastic about the educational potential of implementing applications adapted to multiple document literacy tasks. For this potential to be realized, however, the performance-acceptance paradox that we observed among the guided students has to be addressed, for example by trying to increase students’ sense of autonomy and self-regulation when using the guided procedure. Other tasks, topics, and populations should also be included in future research to probe the generalizability of our findings. For example, it would be pertinent to measure multiple document comprehension with tasks not requiring writing skills, and it would be beneficial for statistical power to use larger samples than what we did in the present study. Ultimately, innovative applications also need to be tested in real-world contexts of retrieving, processing, evaluating, and understanding diverse Internet documents for inquiry purposes.