Introduction

As we move rapidly through the twenty-first century, our goals for education become more and more ambitious. Although there may have been a time when rote learning of facts and procedures was sufficient as an outcome for education, that is certainly not the case today. Anyone with a phone can Google to find facts that they have forgotten. But gaps in thinking and understanding are not easily filled in by Internet searches. Increasingly, we value citizens who can think critically, coordinate different ideas together, solve novel problems, and apply their knowledge in all kinds of situations that do not look like ones they have previously encountered. In short, we want to produce students with deep understanding of the complex domains that constitute the modern knowledge landscape (National Academies of Sciences, Engineering, and Medicine 2018).

Although we have a long tradition of research in the learning sciences related to the problem of how to teach for understanding and transfer, there still is a wide gap between the research, on one hand, and the design and implementation of educational programs, on the other (Hiebert et al. 2002; Lagemann and Shulman 1999; Levin and O’Donnell 1999; Robinson 1998; Strauss 1998; Toth et al. 2000). There are likely a number of reasons for this gap, one of which must surely be the conditions under which much research is carried out. Understanding and deep domain knowledge develop slowly over long periods of time, especially for those domains that are hard to learn (Ericsson 2006). Research, on the other hand, is often carried out in laboratories (Richland et al. 2007), where undergraduate participants are generally available for an hour or so.

Because of this constraint, much of the research has focused on students’ learning of independent facts and procedures—“bits” of knowledge—that can be mastered and tested over relatively short periods of time (e.g., Goldwater and Schalk 2016). We have learned a lot from this research; in particular, we know a lot about how to help students remember bits of knowledge. We know much less, however, about how to help students connect the bits together into a coherent and flexible representation of a complex domain (van Merriënboer 1997). And there is mounting evidence that just mastery of isolated pieces of information is not by itself enough to produce the kind of flexible domain understanding typical of experts (National Academies of Sciences, Engineering, and Medicine 2018; van Merriënboer 1997).

The over-emphasis on mastery of bits of information has been exacerbated by the advent of modern learning technologies, especially adaptive learning platforms. These platforms work by modeling domains as large numbers of skills and concepts to be mastered, then using sophisticated algorithms to provide learning resources to students based on their mastery of prior skills and concepts in an assumed learning progression (Tseng et al. 2008). Although such technologies can be an important part of the toolbox available to educators and instructional designers, we see students in our own classes who master all the bits (or micro-learning objectives) but fail to understand the domain in a deep way. Experts do not see their domains in terms of bits; they see the underlying structure of the domain that makes their knowledge flexible and transferable (Bransford and Stein 1984; Ericsson, Hoffman, & Kozbelt, 2018; Ginsburg 1977; Hiebert and Carpenter 1992).

Our goal in this paper is to present a framework—the practicing connections framework—to guide instructional design for understanding in complex domains. We base our framework on research in the learning sciences but find that we must go beyond the current research literature to get all the way to understanding. For example, although much research focuses on learning in short periods of time, we take as our challenge the design of learning experiences over longer periods of time such as those that make up the typical semester-long college course. And where research often focuses on specific variables, such as cognitive load, or spacing of practice, we are interested in starting with a detailed analysis of what it means to be an expert in a domain and then in charting the pathway(s) students might take to get there.

Other instructional design frameworks take this latter idea seriously, designing pathways that take students from novice all the way to expert participant in the authentic practices of a domain (e.g., Koedinger et al. 2012; van Merriënboer et al. 2002). One such framework is the four component-instructional design (4C/ID) system (van Merriënboer et al. 2002). The 4C/ID system starts with an analysis of the skills that comprise expert performance in a complex domain. It then guides the instructional designer through a series of design decisions, specifying the learning tasks, supportive information, just-in-time information, and part-task practice hypothesized to lead to expert performance. The 4C/ID framework is solidly grounded in research on how people learn (Sarfo and Elen 2007; Susilo et al. 2013).

Although frameworks such as 4C/ID are an excellent starting point, they tend to apply especially well to the development of complex skills. Many complex academic domains, however, require students not only to master complex skills but also to learn to reason with the abstract ideas that form the conceptual structure of the domain (van Merriënboer et al. 2002). Domains such as mathematics, science, and statistics are examples. Part of the domain of statistics, for example, is the complex set of skills that make up the practice of data analysis. But just mastery of these skills is not enough to ensure students understand the core concepts of the domain or that they can use statistical concepts in thinking across a variety of contexts (Richland et al. 2012; van Merriënboer et al. 2002).

The focus on skills over concepts has been noted as a trend in education that cuts across the normal ideological boundaries. On one hand, some educators define learning primarily as the mastery of a set of isolated skills, focusing primarily on students’ repetitive practice of basic facts and step-by-step procedures (e.g., Stigler and Hiebert 2009). On the other hand, proponents of constructivist pedagogies focus more on embedding skills development within the authentic practices that define expert performance in a domain, assuming that the requisite conceptual understanding will evolve naturally from such participation (e.g., Savery and Duffy 1995; van Merriënboer et al. 2002). Neither end of the continuum, however, focuses on the teaching of a domain as a “body of knowledge” that is organized by principles, concepts, and theories (Handelsman et al. 2004; Hodson 1988; Kirschner et al. 2006).

In the practicing connections framework, we propose a complement to frameworks such as 4C/ID by setting core concepts and domain structure on an equal footing with routines designed to support the practice of increasingly complex and transferable skills. From our own experience as teachers, we have become convinced that there is a need, at least in complex domains in which knowledge develops slowly, to connect the development of skills to the deliberate teaching of core concepts and representations that underlie the domain. Gradual improvement in understanding of the core concepts of a domain makes knowledge more coherent, which in turn makes it more flexible and transferable to new situations (Hatano and Inagaki 1986; Richland et al. 2012).

Overview of the Paper

In the remainder of this paper, we outline our practicing connections framework. The ideas we present have been developed in the context of a project in which we have been designing, developing, and implementing an online interactive textbook for introductory statistics (Stigler et al. 2020). This paper is an attempt to make explicit the basis on which we are making design decisions and to justify those decisions based on theories and findings from the learning sciences. The examples we use to illustrate our points will be drawn from our online textbook.

We will start by discussing what we mean by understanding, and the role that understanding plays in the development of transferable knowledge. We argue, based on research, that transferable knowledge is primarily characterized by mental schemas with a high level of coherence and connections.

We then move on to the question of how we can help students develop connected and coherent knowledge in a domain. Our answer, in short, is that students must be provided opportunities to practice making connections.

In addressing the question of which connections in a domain are most important to practice, we focus on those that help support deep understanding and transferable knowledge. Specifically, we propose three types of connections that should be the focus of instructional design: the world (including practices and contexts); core concepts, which organize the domain; and key representations used for communication and thinking within a domain. Our hypothesis is that all of these need to be connected.

Then, having decided on which connections are most important, we lay out the kinds of experiences that will help students construct and strengthen these connections. We propose, based on our reading of the research, three principles to be considered when designing pedagogy: making connections explicit for students (as opposed to relying on self-discovery), engaging students in the hard work of incorporating the connections into their developing understanding of the domain, and providing opportunities for students to engage repeatedly in this process over time as they deepen and extend their domain knowledge.

Understanding as Connected Knowledge

We have made the point that our interest is in the development of understanding and transferable knowledge in complex domains. We want students to develop robust and flexible knowledge that they can take with them into the world, coordinate with other knowledge, and apply to new problems and in new contexts. But what, exactly, is understanding? What makes knowledge broadly transferable? A useful place to start is with the large and growing body of research on the nature of expertise and expert performance (Ericsson et al. 2018; Ericsson et al. 2006; Ericsson and Pool 2016; Hatano and Inagaki 1986). What we want for our students is similar to the domain knowledge we see among experts, which, on the whole, can be characterized as coherent, connected, and relational.

Presumably, after some instruction, novices will achieve some level of expertise. But Hatano and Inagaki (1986), in a well-known paper, proposed a distinction between adaptive and routine expertise. Routine experts have a lot of knowledge, both declarative and procedural, which they have mastered well enough to perform with fluency up to some standard in familiar circumstances. In contrast, adaptive experts stand out for their ability to flexibly apply their knowledge in a wide range of contexts, both familiar and novel.

In some domains, routine expertise is sufficient, because conditions are stable and feedback is reliable (e.g., domains that Epstein 2019, refers to as “kind,” such as chess or music). But our focus here is on domains Epstein refers to as “wicked.” Wicked domains, which in our view include most complex academic domains, are marked by changing contexts and inconsistent or ambiguous feedback (Epstein 2019). Arguably, the aim of instruction is to create adaptive expertise which is presumably driven by what education psychologists call “transferable knowledge” (Bransford and Schwartz 1999; Greeno et al. 1993; Renkl et al. 1996). But well-documented failures of transfer in academic contexts (Bransford and Schwartz 1999; Stigler et al. 2010) suggest that many common instructional practices leave students—at best—as routine experts, critically unable to adapt their knowledge to new circumstances.

In general, research on the nature of expert understanding paints a consistent picture: adaptive experts’ knowledge is organized in a different way (Bilalic and Campitelli 2018; Carbonell et al. 2014; Chase and Simon 1973; Chi 2011; Chi and Koeske 1983; de Groot 1965; Ericsson and Charness 1994; Ericsson et al. 2018). Adaptive experts’ knowledge organization is more coherent, interconnected, and reflective of the relational structure of the domain (Bilalic and Campitelli 2018; Carbonell et al. 2014; Kellman et al. 2010; McKeithen et al. 1981).

The building blocks of this transferable knowledge, sometimes referred to as schemas, emphasize the connections between abstract relations (such as hierarchies, embedded categories, and functional systems) rather than lists of discrete facts and procedures (Bedard and Chi 1992; Chi et al. 1981; Chi et al. 1982; Ericsson et al. 2018; Kellman et al. 2010; North et al. 2011). For example, expert physicists’ schemas for a range of problem situations are organized by fundamental relationships (e.g., the work energy principle) rather than by superficial surface-level details specific to problem contexts (Chi et al. 1981). Experts also have fewer and more interconnected schemas that encompass more instances (Lachner et al. 2012).

Because their knowledge is highly organized and interconnected, adaptive experts perceive the world differently than routine experts or novices. They easily and quickly identify and attend to the relevant structural information critical to understanding the situation at hand, filtering out the irrelevant features of a problem to home in on a solution path (Campitelli and Gobet 2005; Endsley 2018). They anticipate how modifications to a system will influence outcomes and can explain why and how concepts from one scenario may apply to another (Carbonell et al. 2014; Hatano and Inagaki 1986; Holyoak 1991). Their knowledge structures prioritize connections among concepts, examples, and contexts (Bransford and Stein 1984; Ericsson et al. 2018; Ginsburg 1977; Hiebert and Carpenter 1992), such that adaptive experts are able to efficiently chunk information, leaving more working memory resources available (Ericsson 2018; Ericsson and Kintsch 1995; Glaser and Chi 1988). If information were simply stored as discrete entries in a mental list, lacking structural connections to lend coherence, it would be difficult to pick out relevant information and coordinate knowledge efficiently (e.g., Reed 1985). But because expert schemas encode a connected representation of the domain, the knowledge is rendered transferable (Bedard and Chi 1992; van Merriënboer 1997).

For our purposes, we equate deep understanding with the transferable knowledge of adaptive expertise. Thus, we offer the following characterization: understanding is characterized by the ability to perceive and make explicit the underlying structure of a domain, its connections and relations in the form of coherent mental schemas, which allows for transferable and flexible application of domain principles. (The transfer of principles is regarded as the highest form of transfer; see Barnett and Ceci (2002) for an in-depth discussion of the range of types of transfer.)

It is clear from this characterization of understanding that connections are important in order for knowledge to be transferable. Consider, as a thought experiment, the two individuals represented in Fig. 1, below. They both have the same pieces of knowledge, represented as A, B, C, D, E, F, G, and H, and they both are working to solve the same novel problem, the solution to which requires them to access and apply one specific piece of knowledge, B. Because the individual on the left has few connections among his bits of knowledge, he will not be able to transfer what he knows to this new situation. As he works to understand the problem, he keeps coming up with E (his strongest connection). He also tries to apply D. But he has no way to get to B, even though he “knows” it, because it is not connected with the other things he knows or with the problem he is currently facing.

Fig. 1
figure 1

The importance of connected knowledge to achieving transfer

Full size image

The individual on the right (we can call him Mr. Right) approaches the same novel problem from a much stronger position. Like Mr. Left, his first thought is to retrieve and apply E to the situation at hand. He might then, like Mr. Left, think again and end up at D. But from D, he has many more options because of the interconnected nature of his knowledge. He could go from D right to B and then solve the problem. He could go from D to F and then B, from D to F to G to B, and so on. The point is that he has many ways to get to the knowledge he needs. Once he has arrived at B, he must, of course, coordinate B with other pieces of knowledge in order to craft a solution to the problem. He must also adjust his solution, once identified, to fit the unique context in which this novel problem presents itself. Because his knowledge is interconnected, we would say that he has understanding. Based on this understanding, he can retrieve, coordinate, and adapt what he knows to most any situation.

Much of education focuses on teaching students the “bits” of knowledge and skills—the A, B, C, and so on. One needs not go further than a standard math textbook to see lack of connection from one chapter to the next, much less connections made across content addressed at different grade levels. Although the bits are important to learn, and some of the bits require considerable time and effort to master, just learning the bits does not lead to understanding and transfer. Our practicing connections framework takes a different approach: instead of focusing our attention solely on the bits, we are trying to find ways to help students create the connections between the bits, with the goal of producing understanding and transferable knowledge.

We are not the first to propose this focus. Goldwater and Schalk (2016) proposed that relational categories, as distinct from the feature-based categories that are often the subject of cognitive psychology research, might be a bridge between learning sciences research and education. Similarly, Schwartz and Goldstone (2015) suggest a “coordination” approach to learning, in which the goal of instruction is to strengthen skills in relation to other skills, not siloed in isolation. Skills practiced in isolation are destined to remain in isolation; even worse, they can become an obstacle to new learning (Woltz et al. 2000). Schwartz and Goldstone have characterized this kind of coordinated learning as “teaching the brain to dance.” Much of what we want students to understand in academic domains is relational.

We have argued for the value of understanding and transfer and that these things are the result of interconnected knowledge. What we have not yet addressed is how we expect students to gain that knowledge. Although we seek to move away from the learning of bits, the mechanism through which those bits are traditionally learned is one we value: practice. We will expand below on the type of practice we envision, but for now, our hypothesis, simply put, is this: students should practice making connections. If the goal of instruction is to support students in their practice making connections, two questions emerge, which we address in the rest of this paper: (1) What are the connections that need to be practiced? Obviously, this will be highly dependent on the domain, but we offer some principles to guide the instructional designer; and (2) how can we design instruction to give students more opportunities to practice the key connections that have been identified for the domain? In the following sections, we lay out our framework to answer these questions, illustrating our points with examples from the domain of introductory statistics.

Practicing Connections: What Connections to Practice?

We begin with connections. It is easy to make the case that students’ knowledge should be interconnected and coherent. But just saying this skirts an important question: Which connections, of all the possible connections students could practice making, should be the focus of instruction? Our work in statistics, and our reading of the research literature, leads us to propose that three types of connections are critical for learning in any complex domain. These are the following: (1) connections with the contexts and practices in the world to which the domain knowledge is intended to apply (Engle et al. 2012; van Merriënboer 1997), (2) connections with core concepts that serve to organize and lend coherence to the domain (e.g., National Council for Teachers of Mathematics 2000; National Governors Association 2010; Richland et al. 2012), and (3) connections with key representations used for thinking and communicating in the domain (e.g., Ainsworth 2008; Kozma 2003; Strauss 1998).

Connections #1: Contexts and Practices of the World

At its core, academic instruction should be motivated by the demands of the world beyond the classroom. The hope of instruction is that students will transfer what they are learning to authentic real-world situations, yet successful transfer has proven to be an elusive goal. Research suggests that continuously connecting learning in the classroom with the authentic practices of the domain may be one of the most effective ways of developing transferable knowledge (Barnett and Ceci 2002; Bransford and Schwartz 1999).

When developing our introductory statistics curriculum, we started by examining the practice of data analysis. There are many ways one might describe this practice. The important thing, in our view, is that some description of the world outside the classroom be made explicit—for the textbook authors, for students, and for instructors tasked with implementing the curriculum. Our description of the practice of data analysis is illustrated in Fig. 2, which we often come back to as we teach the class. We tell students from the beginning that the goal of statistics is to explain variation in the world, which we break down into three core practices: (1) explore variation (in data), (2) model variation, and (3) evaluate and compare models.

Fig. 2
figure 2

The practice of data analysis

Full size image

We organized our textbook around these three practices, continually reminding students of where the specific skills and concepts they are studying connect with the work they might one day do if they pursue work as a data scientist. For instance, data scientists explore variation when they construct a graph to accompany a corporation’s annual report. They model variation when they predict the effect of a particular variable on future sales. And they evaluate models when they compare competing explanations for customer satisfaction. By grounding instruction in the practice of data analysis, students can participate, from the beginning, in activities with clear connections to the goals and routines used by experts (Lave and Wenger 1991). Rather than teaching students isolated bits and hoping they will put them together to solve a larger problem later, students practice recognizing the need for their developing knowledge in the context of authentic tasks (National Academies of Sciences, Engineering, and Medicine 2018).

The world beyond cannot be fully described, though, just in relation to the practice of data analysis. It is also important to identify the range of situations and contexts to which students’ developing skills and understanding should be applicable. Continually varying contexts (which in statistics could be as simple as varying data sets) help hone students’ perception and understanding of which features of a situation are critical, and which superficial, for the application of domain knowledge (Gentner 1983; Kellman et al. 2010; Son et al. 2011). When students learn about an abstract idea in one context, it is as if a rubber band is tightly bound around that first learning context. As students experience more contexts, there is a gradual stretching of that rubber band to include an ever widening range of situations. By stretching to include more situations, the concept becomes more differentiated, coherent, and flexible and more likely to transfer to new situations.

One example from our own teaching of statistics is the concept of observational unit. Most of the data sets we use in psychology use people as the units of analysis; so, each row in the data table, or dot in the scatter plot, typically represents a person. But if students work only with data in which people are the units, they will have developed a limited concept of observational unit and will have trouble applying the concept in new situations. Following our rubber band analogy, we work from the beginning to give students practice with a variety of data sets in which the rows are not people but instead represent families, states, countries, or companies. In this way, students’ development of the concept of observational unit gets connected to a broader and more diverse set of contexts.

We want students to activate the concept of observational unit not only in the context of different data sets, but also within the different activities that comprise the practice of data analysis. For example, in the context of exploring variation in data, we present students with a scatter plot of data in which state (as in each of the USA) is the observational unit. With each dot representing a state, they can see a high correlation between the percentage of the state’s population that is obese and the percentage that are smokers (see Fig. 3). Students at first mistakenly interpret this as evidence that smokers are more likely to be obese. To overcome this misconception, students must learn to connect the variation they see in the scatter plot with the units that are measured; they must learn to say, “States that are high in percentage of residents who smoke are also high in percentage who are obese.”

Fig. 3
figure 3

Scatterplot showing percentage of a state’s residents who are obese as a function of the percentage who smoke (from the US States data set)

Full size image

Connections #2: Core Concepts That Organize the Domain

The second type of connection we want students to make is with core concepts that organize the domain. Studies show that expert knowledge in a domain is generally organized around a small set of core concepts (e.g., Lachner and Nückles 2015) that imbue coherence to even wicked domains. Because they are highly abstract and interconnected with other concepts, core concepts must be learned gradually, over extended periods of time and through extensive practice. As students practice connecting concepts with other concepts, contexts, and representations, these core concepts become more powerful and students’ knowledge becomes more transferable (e.g., Baroody et al. 2007; National Council for Teachers of Mathematics 2000; Rittle-Johnson and Schneider 2015; Rittle-Johnson et al. 2001).

If only we could just ask an expert what the core concepts are, we could plan our statistics curriculum around those concepts. But the concepts most useful for novices in the early stages of learning might not be the same ones developed by experts over many years of experience (Kirschner et al. 2006). The design challenge is to select concepts that are accessible to novices, but recognizable to experts as having validity and utility in the domain. Critically, although the core concepts introduced to novices may be over-simplified in some ways—indeed, in complex domains, the concepts almost always need to be simplified for beginning students—it is important that the concepts not be simplified to the extent that misconceptions are introduced and that the simplification does not yield a collection of concepts that fail to form a coherent whole.

Through a lengthy process of study and discussion (which included testing by teaching), we chose to focus our textbook around three core concepts: statistical model, distribution, and randomness. We chose these concepts based on several criteria. They should be concepts (1) that we could continually connect to throughout the course, (2) that novices could engage with (i.e., within their zone of proximal development), (3) that would not have to be unlearned later as students progressed to higher levels of understanding, and (4) that, taken together, they would help students build a coherent understanding of the domain of statistics. (Although we will briefly describe the concepts we chose here, a more detailed development of our approach to core concepts in statistics is available in Son et al. under review.)

Statistical Model

Anyone who studies statistics at the advanced levels will be familiar with the concept of statistical model. Yet, the concept is almost never mentioned in the introductory course. Our view is that if we want students’ knowledge of introductory statistics to transfer to advanced courses and to the wider practice of data analysis, we should connect what they are learning to the concept of modeling from the very beginning. Thus, we endeavor to do this in our introductory book. We start by conveying the overarching goal of data analysis as explaining variation in data. We then, from the beginning of the book to the end, connect everything students are learning to the statement DATA = MODEL + ERROR.

When we introduce the mean, for example, we conceptualize it as the simplest of all statistical models. If we use this simple model to predict each point in a distribution, most of our predictions will be wrong (though the mean will be unbiased and better than just a random guess). If the mean is a model, we can conceptualize error as the deviation from each predicted score to the actual score. Standard deviation is introduced, then, as a means of quantifying how much total error there is around the model predictions. As we add explanatory variables to the model, the accuracy of the predictions will go up, while the error goes down. Connecting the activities of data analysis to these fundamental abstract concepts yields knowledge that is more highly interconnected and therefore more usable.

Distribution

Following Wild (2006), we adopted distribution as our second core concept. Wild defines distribution as “the pattern of variation in a variable” (cf. Garfield and Ben-Zvi 2005). The concept of distribution is the lens through which we view variation (see Fig. 4). In statistics, it is critical to situate reasoning relative to three distinct types of distributions: the sample data, the data generating process (DGP) that gave rise to that data (related to the concept of population), and sampling distributions, which are the imaginary distributions from which sample statistics are generated. Together, we refer to these three types of distributions as “the distribution triad.”

Fig. 4
figure 4

Variation in the world is captured by measurement and turned into variation in data, which we then analyze through the lens of a distribution (figure from Wild 2006)

Full size image

The concept of sampling distribution is exceedingly difficult for students to understand, yet critically important for the process of statistical inference. We spend time helping students practice the connections between different distributions and the questions they are best suited to answer. For example, asking about the probability that a group of people will have an average weight over some prescribed limit (e.g., 200 pounds) requires a sampling distribution. Asking about the probability of a single individual weighing more than 200 pounds, in contrast, will require us to use a distribution of sample data or a model of the data generating process constructed based on a sample of data.

Randomness

The third core concept we emphasize is the concept of randomness as a data generating process. Students naturally think of causal explanations for variation. Seeing variation as caused by random processes, on the other hand, is not something students come to naturally (Batanero 2016; Batanero et al. 1998; Kaplan et al. 2014). Randomness, of course, plays an important role in statistical thinking, mainly because we know how to model random processes. Students start out associating randomness with “unpredictability” (e.g., a single randomly generated number between 1 and 10 is hard to predict). Through computational techniques such as simulation, bootstrapping, and randomization, we give students the tools and experiences they can use for thinking about randomness as a process that yields predictable patterns of variation over the long-run even though a single sample of data remains unpredictable.

These three core concepts—DATA = MODEL + ERROR, the distribution triad, and randomness as a process—provide an organizing framework to lend connections and coherence to statistical routines and their purposes. For example, although univariate ANOVA and simple regression are typically taught as two separate concepts in most introductory statistics courses, we emphasize that both are examples of linear models. Both kinds of models generate predictions, and both measure error in the same way (i.e., the difference between predicted and actual scores). In both, we compare a more complex model to a simple one in which the resulting distribution is the result of a random data generating process (traditionally called the null hypothesis). We hypothesize that when these analytic techniques are taught as examples of the same thing, students’ knowledge of statistics will be more coherent and more likely to transfer to new situations.

Connections #3: Key Representations for Thinking and Communicating

In addition to building a connected understanding of the core concepts that comprise a domain, we also want students to learn to use key representations that embody those concepts and that can represent explicitly the relational structure of a domain (see Ainsworth 2008; Gentner and Rattermann 1991; Star and Rittle-Johnson 2009; Uttal et al. 1997). Evidence suggests that teaching with multiple representations produces deeper, more flexible learning (Ainsworth 2008; Ainsworth et al. 2002; Brenner et al. 1997; Pape and Tchoshanov 2001). Also, using and understanding symbolic representations are necessary for communicating and developing higher order skills (e.g., Gilbert and Treagust 2009). Because learning to use a representational system requires an investment of time and effort (Ainsworth 2008; Star and Rittle-Johnson 2009), however, we must decide which representations to focus on for novices. In making this selection, we want to find representations that are accessible to novices (Vygotsky 1980), important for the field (Tabachneck-Schijf et al. 1997; Tsui and Treagust 2013), and most productive for making connections (Kaput et al. 2017).

In our introductory statistics course, we decided to focus on five key representations: verbal descriptions, visualizations, word equations, GLM (general linear model) notation, and R code. Accordingly, throughout our course, we repeatedly ask students to translate and connect using verbal descriptions, visualizations, word equations, GLM notation, and R code. Note that we did not choose algebra, in the form of formulas and equations, as one of our key representations, even though it is emphasized in most textbooks. Our reason for this is that our students do not typically find algebra to be readily accessible or useful, a consequence, we surmise, of K-12 mathematics education in the USA.

Graphs and visualizations have always been important to understanding in statistics and are even more important in the age of data science, in which professionals are often called on to create publication-ready figures and graphs. Coding (e.g., in R) is also a representation that has become increasingly important in statistics as the field itself has become more computational in nature. R also gives students a means of participating in the emerging routines of data analysis that place increasing value on producing and sharing reproducible analyses.

In developing students’ understanding of statistical models, we start by having students describe simple models in words (e.g., “knowing someone’s height will help us make a better but not perfect prediction of their thumb length”). We then teach them to write word equations (e.g., Thumb Length = Height + other stuff), which helps them begin to take their initial insight (that statistical predictions are not perfect) and connect it with the concept of error (represented in the word equation as “other stuff”).

Later, students are asked to map word equations to a more general idea represented in a similar way (i.e., DATA = MODEL + ERROR) and then to use the same structure in writing R code to generate a graph (e.g., scatterplot(Thumb ~ Height, data = data set) (see Pruim et al. 2017, and their resulting visualizations). This body of connections then supports learning the mathematical notation of the GLM (e.g., Thumbi = b0 + b1Heighti + ei and more generally Yi = b0 + b1Xi + ei).

We want to expand here on the choice to use the notation of the GLM. It is a difficult representational system for students to learn but exemplifies the features that support connections: generalizability (it can represent many situations, even those that are not present in the course) and alignability (it aligns with the other representations in the curriculum). Furthermore, because it is used by many professionals and frequently published in papers, it is highly ecologically valid. For instance, the notation of the GLM explicitly represents the structural similarity between group models such as ANOVA (e.g., Outcomei = b0 + b1Groupi + ei) and regression models (e.g., Outcomei = b0 + b1Quantityi + ei), supporting students as they work to recognize the structural similarities across the two types of models.

Designing Learning Experiences to Support Practicing Connections

Having laid out the types of connections students need to work on making, we turn now to the question of how, as educators, we can design instruction that will facilitate this process. Based on our reading of the research, we believe there are three key principles that should guide instructional design. First, the connections identified above (core concepts, key representations, and the world) must be made explicit at some point during the instruction. Second, students must be engaged in productive struggle. Connections, as it were, must be earned through hard work, as much as we would like to be able just to give them to students. And finally, opportunities to engage in the work of forging connections must be offered repeatedly to students over sustained periods of time as they deepen and extend their domain knowledge. We will discuss each of these principles next.

Principle #1: Make Connections Explicit

Recognizing structural relatedness is critical to understanding in complex domains (Bassok 2001; Chen and Klahr 1999; Novick 1988; Novick and Holyoak 1991; Reed 1985; Ross 1987). However, research suggests that connections between structurally similar problems often go unrecognized by learners, unless the connections are pointed out explicitly (Gick and Holyoak 1983). Learners are often unable to summon relevant prior knowledge when needed (Reeves and Weisberg 1994). And novice learners, especially, often get stuck on the surface features of a problem and need help to connect the surface representation to deeper underlying concepts (Ross 1987).

These results are consistent with findings from the voluminous research literature on discovery learning. Although it is likely that many high-level experts have acquired their domain knowledge on their own, based mainly on their own persistence and lengthy experience, the research suggests that most students do not benefit from simply being given the opportunity to discover structure on their own (Alfieri et al. 2011). As we will see in the next section, effort is most certainly required on the part of the learner. But effort alone is usually not enough (Mayer 2004). At every age level, guided discovery is more effective than simple discovery learning (e.g., Alfieri et al. 2011; Klahr and Nigam 2004; Weisberg et al. 2015).

A number of research paradigms from cognitive psychology demonstrate techniques for making connections explicit and the benefits of such techniques. Research on learning from analogies (superficially dissimilar but structurally parallel instances) provides one example. When learners draw parallels between two cases and practice aligning similar elements across two systems (Gentner et al. 2003; Son et al. 2011), they are better able to transfer their knowledge to superficially dissimilar novel problems (Alfieri et al. 2013). Learning tasks that make relations explicit, while stripping away distracting details, results in more portable and generalizable knowledge (Gentner and Markman 1997; Kaminski et al. 2008; Son et al. 2008; Uttal et al. 2009).

In our online textbook, we use two specific techniques for making connections explicit. The first is to structure the book so as to afford students’ seeing of connections and then to explicitly point out these connections in the text itself. The constant connections to the concept of statistical modeling in our book exemplify the first strategy. Having decided on modeling as a core concept, we structured the book around the enterprise of modeling—and then testing and comparing models of—variation in data. As statistical concepts and procedures are introduced, each is related explicitly to the statement DATA = MODEL + ERROR.

When the arithmetic mean is introduced, for example, we point out explicitly that the mean is an example of a function that could fill the MODEL slot in the abstract statement DATA = MODEL + ERROR. Residuals, sum of squares (SS), variance, and standard deviation, similarly, are explicitly connected to the concept of ERROR. When sum of squares is introduced, we point out that this particular measure of error is minimized at the mean and relate this fact to the overall goal of minimizing error as a means of increasing the power of a model. These connections increase coherence across the domain by recasting what were previously thought of as separate topics (e.g., “measures of central tendency” and “measures of variation”) as important concepts that relate to the overall work of statistical modeling.

The second strategy for making connections explicit is to repeatedly ask questions of students, the answers to which are the explicit connections we are trying to strengthen. We constantly ask students to explain, for example, how core concepts and representations relate to each other (e.g., Chi 2000; Lombrozo 2006). Across various situations and representations, we repeatedly ask students to explicitly identify DATA, MODEL, and ERROR. We ask students to connect a data point in a scatter plot to a particular value in a table of raw data. After asking them to fit a regression model, we ask them to save the model predictions back into the data frame (using R) and then plot the model predictions as a function of the explanatory variable. We ask them to explain why the model predictions seem to coincide with a linear regression line, while the actual data points do not. We ask students to continually contextualize what they see in data in terms of the overall modeling enterprise.

The same principle of making connections explicit can be seen in the research on worked examples, in which students learn by examining an expert’s problem solution (Atkinson et al. 2000; Paas et al. 2003). When students are asked to solve a problem, much of their effort goes toward generating and executing a solution method, with few attentional resources left over for reflecting on the concepts that underlie their solution. When given the solution and asked to study it, however, learners can reflect on why the solution works, which has the potential to make clear how the solution relates to the core underlying principles of the domain.

The literature on worked examples is closely related to cognitive load theory (CLT; Paas and van Gog 2006; Paas et al. 2010). CLT distinguishes three types of load: intrinsic, germane, and extraneous. Intrinsic load is determined by the task at hand; if you want to do the task, you will need to contend with the intrinsic load. For learning, the goal is to decrease extraneous load (the optional extra stuff that surrounds the task) and increase germane load (Sweller 1988; van Merriënboer and Sweller 2005). In our conceptualization, germane load is the part of attention and effort focused on critical connections in the domain. A desire to increase germane load means accommodating a need to leave more cognitive resources available for conscious reflection on principles and concepts that underlie the deep structure of the domain.

One of the reasons we interleave R throughout our curriculum is that R takes care of the calculations quickly, thus reducing extraneous load and leaving students with more mental resources available for thinking about what the results of the calculations mean. For example, using a simple R function to calculate residuals leaves students with more resources they can direct toward seeing that residuals, as ERROR, can be expressed as DATA − MODEL, an equivalent form of our core conceptual framework. Instead of asking students to engage in repeated subtraction, students run simple R code (e.g., resid(model)) and then focus on explaining the meaning of what they find, such as why some residuals are negative and some are positive, how the residuals relate to the mean and the original data, and how residuals relate to aggregate error (e.g., SS). Making these connections is difficult but part of the germane load of fostering understanding.

Principle #2: Engage Students in Productive Struggle

As important as it is to make explicit the connections that lead to understanding in a domain, it is clear that simply making connections explicit is not sufficient to produce deep understanding in students. In our culture, there is a widespread belief in the myth that understanding comes suddenly, like a lightning strike, in an “a-ha!” moment. If only it were that simple. In reality, understanding is something that must be earned, gradually over time, through the hard work of each individual learner. If a student experiences learning as easy and effortless, this generally means that what was learned will not be retained (Koriat and Bjork 2005). We have all had the experience of watching a teacher solve a problem, thinking, “That looks easy. I can do that.” But in the end, when we are on our own, we often find that we cannot “do that.”

This “no pain, no gain” perspective on learning is consistent with a broad body of research in the learning sciences. Many studies demonstrate the superiority of active learning over passive learning (Bean 2011; Michael 2006; Prince 2004). Neuroscientist Stanislas Dehaene (2020), in a wide-ranging review of the literature, calls active engagement one of the four pillars of learning. Dehaene writes: “A passive organism learns almost nothing, because learning requires an active generation of hypotheses, with motivation and curiosity (p. xxvii).” Bjork and Bjork (2011) have coined the term “desirable difficulties” to refer to the struggle that necessarily and productively accompanies lasting learning. Research on the testing effect, for example, shows that students learn more from trying to answer test questions than they do from being re-presented with the same information (McDaniel et al. 2007; Rowland 2014).

Productive struggle not only is important in its own right but also plays a role in students’ learning from explicit connections. Across a variety of research paradigms, it has been shown that giving students opportunities to struggle in solving a challenging task before presenting them with solutions is more effective than the typical sequence in which direct instruction precedes practice (Hiebert et al. 1996; Vygotsky 1980). When they grapple with a challenging task, learners inevitably work to connect features of the problem with their prior knowledge. Not only does this help them to identify gaps in their knowledge but also prepares them for subsequent instruction that explicitly connects their prior knowledge to the core concepts and representations of a domain (Capon and Kuhn 2004; Lawson et al. 2019a; Schwartz and Bransford 1998).

In our online book, we design each page with a “struggle first” pedagogy, asking students to answer questions—often in an open-response format—before we have presented them with the information and explicit connections they might need in order to give a well-formulated answer. For example, before we discuss the properties of the mean and median, we present students with a distribution of 5 data points and ask: “In what sense might the median be a better model for this distribution? In what sense might the mean be a better model?” Putting the question before the answer in this way is counter to what students expect, and sometimes they will point out to us, in an attempt to be helpful, that we have mistakenly “gotten the order wrong.”

Another strategy we use to increase productive struggle (and germane cognitive load) is to remove opportunities for calculation from the book. We want students to expend their energy grappling with key concepts and the connections between them, but because US college students tend to equate doing math with doing calculations, they often will, if given the chance, start calculating before they have had a chance to think about a problem, how it relates to core concepts, and even which calculations might be most appropriate given the situation. One way to stave off premature calculations and reserve attention for conceptual connections is by giving students problems that do not have any numbers in them, making calculations impossible (Givvin et al. 2019; Lawson et al. 2019b). In our book, we hold off on presenting any formulas or calculations until chapter 5, asking students instead to work on developing their intuitive ideas about models and model comparison by examining and discussing graphical representations of data (e.g., histograms or box plots).

For example, in chapter 4, students create faceted histograms to compare distributions of restaurant tips between two randomly assigned conditions in an experiment, one in which the server puts a smiley face on the check and another where they do not. Instead of calculating or displaying the means of the two groups, we ask students whether they think variation in tips is explained by the experimental manipulation. This leads to a rich discussion of what it means to “explain” variation. Working with an intuitive definition of “explain,” we reformulate the question as: “Does knowing which condition a table was in help us make a better prediction of the tip?” Thinking about this question helps students to see that both the central tendency and the variation are important in answering the question. Much of this discussion would have been cut off if students had rushed to calculate the mean difference between the two groups.

Principle #3: Provide Opportunities for Deliberate Practice

Our first two principles work together to form the in-the-moment experience of a specific learning opportunity. Struggling productively to make connections between problems in a domain and the core conceptual structure of the domain is the raw material from which understanding is forged. But developing transferable knowledge in a complex domain is a long-term prospect (Ericsson 2018; Ericsson et al. 2007) and happens in fits and starts (Felder and Silverman 1988). For students to develop a deep understanding of a domain, we will need to find ways to provide repeated opportunities for struggling with important connections over long periods of time. In other words, students must practice the connections. It is generally accepted that skill learning requires practice; we argue that understanding requires practice, too.

This leads us to the third principle in our framework: deliberate practice. Deliberate practice is a term that comes from the expertise literature (Ericsson 2017; Ericsson et al. 1993). Although it is normally discussed in relation to activities like music, chess, and motor behavior, much of what is posited about what makes practice effective should apply equally well when the task is to develop proficiency with concepts and making connections between them, as it does when applied to skill acquisition.

It is helpful to begin with the clarification that deliberate practice is distinct from repetitive practice. Repetitive practice leads to fluency, creating in learners the feeling that a concept or skill is getting easier over time (e.g., Bjork et al. 2013). Automaticity, which is the goal of repetitive practice, can also be its downfall. The feeling of fluency is a sign that learners have plateaued, not that they are deepening their understanding (Ericsson 2008).

The key challenge for aspiring expert performers is to avoid the arrested development associated with automaticity and to acquire cognitive skills to support their continued learning and improvement. By actively seeking out demanding tasks – often provided by their teachers and coaches – that force the performers to engage in problem solving and to stretch their performance, the expert performers overcome the detrimental effects of automaticity and actively acquire and refine cognitive mechanisms to support continued learning and improvement” (Ericsson 2006, p. 696)

Although we acknowledge the need for certain skills to be readily available in a student’s repertoire, our ultimate aim is that students be capable of calling upon them in a way that is not routinized, so that they can make use of them in novel situations.

Deliberate practice requires that instructional materials maintain a constant (and high) level of challenge as abilities develop. This is accomplished by raising the difficulty of the connections being practiced to meet an individual’s growing capability, all in accordance with a desired learning trajectory. As content designers (and instructors), we must help define and design the tasks that are practiced, “to correct some specific weakness while preserving other successful aspects of function” (Ericsson 2006, p. 700). We must keep students practicing connections in increasingly challenging situations, spending time where weaknesses are greatest. Returning to our rubber band analogy from earlier in the paper, our book must continually expand the scope of the rubber band in order to keep ahead of learners’ developing understanding in the domain while continuing to encompass what was initially contained within it.

One of the most effective ways of increasing challenge (and expanding the rubber band) is to continually vary the context, forcing learners to continually adapt their conceptual understanding to new situations. One way to do so in a statistics curriculum is by constantly introducing new data sets to which students must apply their developing knowledge. In our online book, we use a very limited number of data sets. But when we teach the class, we introduce new data at almost every class session. We leave this to the teacher because statistics is taught in multiple academic departments and the students in those departments have different interests. Data is the most direct way of connecting students’ practice of data analysis with the things they care about most. We want students to come to see statistics as a tool they can apply generally, across a variety of contexts, to help make sense of the world.

If the principles of deliberate practice, alone, were to be applied to statistics learning, one might find a student gradually learning a series of data analysis skills with ever-increasing complexity. Even within a single statistical test—ANOVA for instance—they might begin with a simple 2 × 2 factorial and gradually learn how to manage experimental designs with covariates, repeated measures, and nested levels. All of that might be accomplished, however, without attention paid to conceptual connections across these special cases and how those concepts might apply to other statistical tests. So while we add deliberate practice to our triad of principles that inform the design of learning experiences, applying it in combination with the first two is critical.

It is not just skill acquisition we are after; our goal is to support students’ ability to see connections across contexts and practices, core concepts of the domain, and key representations. It is these connections that will make it possible for students to coordinate their skills and adapt them to the needs of new contexts and situations. And we believe that the only way to increase the strength and robustness of these connections is through the experience of productive struggle. Simply put, if we want students to have knowledge that is flexible and transferable to new and even unforeseen situations, we want to create opportunities for deliberate practice of transfer itself.

Putting It All Together

In summary, it is useful to situate the practicing connections framework in a broader pedagogical landscape. A simple two-by-two table helps us to do this (see Fig. 5). On one dimension, we have productive struggle, on the other, explicit connections. Where neither feature is present, as in the upper left corner, we have rote memorization of disconnected facts and procedures. Students just listen to the teacher and then repeat back what the teacher has said or the steps that have been demonstrated.

Fig. 5
figure 5

The pedagogical landscape for practicing connections

Full size image

If we have struggle but no attempt to make explicit connections with the core concepts or representations of the domain, we end up in the lower left corner. This is the province of discovery learning. If we have explicit connections but no struggle, we have a well-formed lecture. A lecture gives the feeling of fluency because a good lecturer lays out the structure explicitly and clearly. Yet, unless the student is engaged in the active work of internalizing connections into her own mental model, understanding will not result. The sweet spot, as we call it, is in the lower right quadrant. Here is where we have both productive struggle and explicit connections.

Just finding the sweet spot once or even occasionally, however, is not sufficient. Deliberate practice adds in the dimension of time. Understanding unfolds slowly. In complex academic domains, such as statistics, understanding is achieved over weeks, months, and years. The instructional designer, therefore, needs to think carefully about how to sequence instructional activities so as to support the gradual deepening of conceptual understanding. Much has been written about the development of complex skills (van Merriënboer 1997). Our focus is specifically on the understanding part. Understanding is at the root of far transfer and flexible knowledge (Barnett and Ceci 2002; Bedard and Chi 1992). We are just beginning to think explicitly about designing instruction that specifically takes the growth of conceptual understanding as its target.

Summary

In this paper, we have proposed a practical framework for instructional design in complex academic domains. Our specific focus is on the development of understanding. Following the research on expertise, we are proposing what we think it would take to produce students who “see the structure” of the domain and are able to transfer their knowledge to new and unforeseen situations. We call our framework the practicing connections framework. Many students learn the bits of knowledge—facts and skill—that make up a domain but fail to see how they all fit together. Our hypothesis is that it is the interconnectedness of knowledge that makes it transferable. As instructional designers, we need to give students opportunities, over extended periods of time, to practice making connections.

There are two parts to our proposed framework, each part answering an important question. The first question we address is this: If connections are important for understanding, what specific types of connections, of all the possible ones, do we want students to make? We proposed three types of connections: with core concepts, with key representations, and with the world to which the domain knowledge is expected to apply. We reviewed the rationale for each type of connection and then gave some examples from our own experience trying to apply the framework to the design and implementation of an introductory statistics textbook.

The second question addressed by the framework is: How, given that we have specified what the important connections are for a given domain, do we create opportunities for students to practice these connections? Although the literature on teaching and learning is vast, we tried to present a simple organizing framework. Students need to experience productive struggle with explicit connections, and they need to do this continually over extended periods of time (deliberate practice). There is a sweet spot for the design of learning experiences, and we believe this spot is well-supported by research. Yes, we have glossed over many details. But in a real sense, these details need to be filled in by educators and designers working in the trenches to produce deep learning in all students across a wide variety of complex domains.

Challenges of Implementation

Implementing this framework will not be easy in practice, and we want to acknowledge that. First, identifying the core concepts that organize a domain is hard. It requires deep content knowledge as well as pedagogical content knowledge. Second, identifying which connections to make explicit as the course unfolds is not a trivial task. This critical element of our pedagogical approach is one of the things that sets it apart from more traditional teaching, and there is no ready source for a list of connections in the same way that there is for a list of “bits.” Third, progress through a curriculum that results from applying the three design principles we have outlined and offering students sufficient opportunity to practice connections is a slow process. We are convinced, though, that the “slow and sticky” (Hess and Azuma 1991) pedagogy that results contributes to students’ ability to retain their learning over time.

We know the difficulty of implementation from firsthand experience as instructors who tried to break down a wicked domain (statistics) into the relevant connections and then get our students to practice making those connections many times over a school term, all while dealing with broken projectors, exams, office hours, and everything else that goes into teaching a course. We acknowledge also that, although grounded in the experience of having gone from core concept identification all the way through to the implementation of a full curriculum, our practicing connections framework has been applied in only one domain. We invite other educators to take on the challenge of doing the same in other content areas. In doing so, they will no doubt contribute to refining and improving the framework. Our experiences have deepened our belief that this framework can help to guide us toward more successful instructional design.