The use and effects of incentive systems on learning and performance in educational games

https://doi.org/10.1016/j.compedu.2021.104135Get rights and content

Highlights

  • A motivational architecture for educational games was proposed and tested.

  • Content-related supports use increased and game-related supports use decreased.

  • Viewing more content-supports predicted learning and game performance.

  • Viewing more game-supports had no effect on learning or game performance.

  • Students' behavioral data confirmed that students did not abuse game-related supports.

Abstract

We examined the use and effectiveness of an incentive system—one of the five elements of a theory-based motivational architecture in educational games that we proposed—in a computer-based physics game on students' learning and performance. The incentive system's purpose was to motivate students to access learning supports designed to facilitate content knowledge acquisition (i.e., content-related supports) and minimize the use of solution videos (i.e., game-related supports). Students (n = 199) could earn game money by accessing content-related supports but had to pay to watch solution videos. Results indicated that the incentive system effectively increased how often students viewed content-related supports and decreased their reliance on solution videos. Furthermore, students who viewed more content-related supports showed higher posttest scores and solved more game levels compared to those who watched fewer supports, controlling for pretest scores. Solution videos, however, appeared to have no effect on either posttest scores or number of levels solved. Finally, our analysis of students' behavioral data, extracted from log files, confirmed that students did not abuse solution videos. We conclude by proposing areas of interest for future research.

Introduction

The popularity of video games among children and adults is undeniable. For instance, 75% of Americans have at least one gamer in their household (Entertainment Software Association, 2019). Therefore, the growing video game industry has much potential for innovative researchers who want to use games as vehicles for learning (Gee, 2005; Prensky, 2001). However, not all video games are useful for learning—generally just well-designed ones (Shute & Ke, 2012).

Well-designed video games are intrinsically motivating to children (and adults). Players can make mistakes and learn from them without being afraid of negative consequences (Gee, 2005; Shute & Ke, 2012). Such games can be useful learning tools because they provide ongoing feedback and interactivity, and require active participation (Gee, 2003; Ifenthaler, Eseryel, & Ge, 2012; Prensky, 2001). Moreover, the challenges in well-designed video games are incremental, which helps students build the required skills as they progress through the games (Gee, 2005). Progression in these games often leads to the improvement of players’ skills, which in turn keeps players motivated and facilitates the state of flow (Csikszentmihalyi, 1988). When one is experiencing flow, he or she loses track of time, becomes fully immersed in the task environment, and performs at his or her best while enjoying the experience.

Game-based learning (GBL) research has shown that educational games foster learning by including relevant learning supports and appropriate learning principles (e.g., Wouters & van Oostendorp, 2013). For example, educational games have been used to teach content such as mathematics (e.g., Ke, 2008), physics (Shute et al., 2020), and ecosystems (e.g., Metcalf et al., 2011), and have been shown to assess and improve students’ non-cognitive skills such as creativity (e.g., Blanco-Herrera, Gentile, & Rokkum, 2019; Shute & Rahimi, 2020), and persistence (e.g., Ventura & Shute, 2013). These and other research studies have shown positive results, but more research is needed in this area.

For instance, in recent years, researchers and game designers have faced the challenge of developing educational games that maximize learning while maintaining engagement in educational games (e.g., Hamari et al., 2016; Shute, Ke, et al., 2019)). In a systematic review on the effects of digital games, Boyle et al. (2016) found that educational games mainly influence participants' knowledge acquisition, but further research in this area should investigate which game features are most effective in enhancing engagement and supporting learning (Boyle et al., 2016). This study focuses on one important game feature—the incentive system—which is often included in commercial games, but just occasionally in educational games. In particular, we examine the effects of an incentive system on regulating players’ use of learning supports to enhance performance and learning in an educational game.

One effective method used to maximize learning and performance in educational games involves embedding learning supports in games (Shute, Ke, et al., 2019; Shute & Ke, 2012; Wouters & van Oostendorp, 2013). The effectiveness of learning supports in educational games depends on factors such as level of guidance (e.g., indirect vs. direct), type of support (e.g., specific feedback vs. general advice), purpose (e.g., to support game performance vs. content knowledge), and time of delivery (i.e., before vs. after gameplay) (Schrader & Bastiaens, 2012a; Tsai, Kinzer, Hung, Chen, & Hsu, 2013; Wouters & van Oostendorp, 2013). The effectiveness of learning supports in promoting learning and performance, however, can vary from low to high. This depends on various factors such as where, when, and how the supports are being used in educational games (Leemkuil & De Hoog, 2005).

For example, students who have access to direct guidance (e.g., expert video solutions for a particular game level) during gameplay might simply rely on the examples to solve game levels without engaging in learning (Kao, Chiang, & Sun, 2017; Sun, Chen, & Chu, 2018). Based on these findings, one could argue that it is better to exclude direct guidance from educational games. However, students may need help via direct guidance when they are stuck and growing increasingly frustrated (e.g., ter Vrugte et al., 2017). Moreover, while some frustration can help learning (e.g., Gee, 2007), sustained or intense frustration can lead to quitting behavior (Kapoor, Burleson, & Picard, 2007; Spann, Shute, Rahimi, & D’Mello, 2019). Thus, it is important to regulate students’ access to different types of learning supports. We need more research in this area which is the focus of the current study.

Incentive systems, a key feature in many commercial games and in some educational games, are designed to reinforce specific gameplay behaviors and motivate students to play (King, Delfabbro, Griffiths, & Gradisar, 2011; McKernan et al., 2015). Incentive systems are based on behaviorist reinforcement methods (Skinner, 1953) with the purpose of increasing players' extrinsic motivation (Sansone & Harackiewicz, 2000). Extrinsic motivation is defined as “performing behavior in order to achieve some separable goal, such as receiving rewards or avoiding punishment” (Vallerand, 1997, p. 271). Intrinsic motivation is defined as “behavior performed for itself, in order to experience pleasure and satisfaction inherent in the activity” (Vallerand, 1997, p. 271). It is harder to foster intrinsic motivation in educational games than extrinsic motivation (Konetes, 2010). Incentive systems can be used as extrinsic motivators to regulate player's behaviors in educational games using different reinforcement strategies. If designed well, incentive systems can eventually lead to the enhancement of the player's internal motivation. High motivation (intrinsic and extrinsic) in educational games can lead to high engagement, better performance, and ideally improved learning in educational games, which is the goal we pursue: maximizing learning without sacrificing the fun in educational games.

More research is needed that investigates how incentive systems can regulate students’ behavior and motivation in educational games to maximize learning. The findings from such research—including this study—can help GBL researchers design educational games that can direct students to desired behaviors (e.g., using content-related learning supports) and regulate behaviors that can be detrimental to learning. In the next section, we review relevant theories about motivation in games in general, and propose a motivational architecture with five elements needed in educational games (see Fig. 1) to motivate students to engage in gameplay and learning without disrupting the fun in educational games.

As mentioned, video games are usually intrinsically motivating to play (Gee, 2003; Habgood & Ainsworth, 2011). There are various theories—e.g., Self-determination Theory (SDT) (Deci & Ryan, 1985; Ryan & Deci, 2000)—that explain why this is the case (Egenfeldt-Nielsen, 2006). The intrinsic motivational appeal of video games can be understood using SDT (Denis & Jouvelot, 2005; King & Delfabbro, 2009; Przybylski, Rigby, & Ryan, 2010; Sørebø & Hæhre, 2012; Zainuddin, 2018). According to SDT, people feel intrinsically motivated when they gain a high level of competence, autonomy, and relatedness (Deci & Ryan, 1985; Ryan & Deci, 2000). Using SDT, Przybylski et al. (2010) noted that video games can help people (a) achieve high levels of competence in various skills needed in video games (e.g., problem solving, reaction time, creativity), (b) gain high levels of autonomy through various choices and control options available in video games, and (c) develop social bonds with one to tens of thousands of people who live in remote locations from each other. Most well-designed video games are intrinsically motivating, but what about educational video games?

When designing educational video games, careful thought should similarly go into their design to make them as intrinsically motivating as commercial games—especially when subject-matter content is added to the games (Habgood & Ainsworth, 2011). We refer to this as the basic intrinsic motivational layer in educational games (i.e., the second element shown in Fig. 1). Then, when content is added to such games, subtle manipulation of behaviors should be considered. That is, some behaviors should be strengthened (e.g., accessing educational content) or weakened (e.g., reducing the overuse of solution videos). This defines an extrinsic motivational layer (e.g., rewards and penalties).

Behavioral theories suggest that by using rewards and penalties, we can strengthen desired behaviors or weaken undesired behaviors in a learning environment (Ferster & Skinner, 1957; Skinner, 1953). Keller (1979) has similarly noted that when developing learning environments, to influence learners’ motivation, one should provide reinforcements to help sustain desirable changes in behavior. Moreover, behavioral learning theories assume that the behaviors are controlled by their consequences (Ferster & Skinner, 1957; Keller (1979). For instance, if accessing the content in an educational game (i.e., a desired behavior) is rewarded, then chances are that the students will repeat this behavior.

Keller (1979) suggests that students’ motivation can be further enhanced when they receive, learn from, and apply the content in learning environments. Therefore, the integration of content in educational games needs to be contextualized so that its inclusion does not disrupt gameplay and flow (Habgood & Ainsworth, 2011). Kafai (1996) has referred to this type of contextualization as content being intrinsically integrated. Thus, in the context of educational games, accessing learning content in the game is warranted and can be rewarded. In this case, students will be externally motivated to engage with the content and enhance their knowledge. By engaging in this process, students can enrich their in-game performance (Habgood & Ainsworth, 2011). When students benefit from accessing the content in an educational game, this enhances their confidence and satisfaction through game performance—two important components of the ARCS motivational model introduced by Keller (2009). Next, we discuss the literature on incentive systems in educational games.

Given the popularity of commercial video games, we can learn valuable lessons regarding their incentive systems' influence on game performance and learning. For instance, Rapp (2017) investigated the reward features in World of Warcraft and their experiential effects on players. Through observations and interviews with players, Rapp posited that rewards such as points and privileged equipment made students feel more autonomous as they had a greater variety of resources to choose and use in the game. The results of Rapp's study suggested that those rewards pushed students to continually perfect their in-game abilities and enhance their game performance.

Tüzün, Barab, and Thomas (2019) conducted an ethnographic study to explore which incentive elements were more effective at increasing student participation. Students played Quest Atlantis, an educational game where the goal is to save a virtual world from an impending disaster by solving problems related to science, medicine, and mathematics. The game included two types of rewards: non-materialistic (e.g., awards, points, and social approval), and materialistic (e.g., virtual trading post, trading cards, and t-shirts). Students earn points by exhibiting desired gameplay behaviors (e.g., completing educational tasks) and spend points to “buy” materialistic items. Among the items that could be purchased, trading cards were perceived as the most valuable as students explained that they were willing to work harder to obtain them. Tüzün and colleagues also argued that the incentive system could balance the sources for earning and spending points.

In contrast, Snow, Allen, Jackson, and McNamara (2015) examined how students' spending behaviors impacted learning and game performance in a study with 40 US high school students. They found no relationship between students’ spending behaviors and motivation or enjoyment. In addition, they reported that placing a high emphasis on spending behaviors had a negative impact on learning and transfer. Snow et al. concluded that in-game spending, as a game feature, is a complex element, and further research is needed to investigate more deeply the relationship between in-game spending and learning.

In two separate studies reported in one paper, Biles, Plass, and Homer (2020) investigated the effects of using badges, another type of incentive system, on students' learning. Their first study (comparing students who received badges to those who did not receive badges) showed that badges did not work the same for all students. Students with high situational interest (i.e., interest in an unknown content which is generated as a result of one's interaction with a learning environment) performed better with badges, while learners with low situational interest performed worse with badges. In their second study, Biles and colleagues compared two types of badges: mastery goal orientation badges and performance goal orientation badges. The first study's findings again appeared (i.e., the interaction between condition and situational interest). Moreover, students receiving performance badges outperformed the students in the mastery badges condition on the posttest. In general, these two studies suggest that the type of badges available, and students' interest and motivation are essential factors for increasing educational games' learning outcomes.

In general, incentive systems in educational games are intended to motivate students to achieve better game performance and learning. However, we find limited empirical research that directly explores the relationship between incentive systems and educational game performance. Keeping students motivated as well as engaged, and improving in-game performance and learning is a large challenge in GBL research (Boyle et al., 2016). To shed light on this matter, we investigated the effects of an in-game incentive system on students’ access to learning supports and the effects of such supports on learning content-related knowledge and game performance (i.e., solving game levels). Our educational game included two types of learning supports: content- and game-related supports, discussed next.

Content-related supports are designed to help students learn the underlying content in the game (Leemkuil & De Hoog, 2005; Schrader & Bastiaens, 2012b; Shute et al., 2020). For example, Schrader and Bastiaens (2012b) used a physics educational game with 47 eighth graders and included an ever-present button linking to a textbook. Results showed that students who accessed the learning support scored significantly higher on the posttest compared to the students who did not access the support. However, the number of students accessing the learning supports was low, and 17 out of 47 students simply ignored the button. The authors suggested that the infrequent access of supports was because students were forced to leave the game environment to use the supports.

Tsai et al. (2013) embedded content-related supports in a game environment, consisting of questions related to the target content knowledge. In their study, 79 middle school students were split into three conditions: (1) Required access of supports—where students had to access the supports before each level, (2) Optional access of supports—where students had access to the same material but were not required to use them, and (3) No supports. Results showed that students who were required to access the supports scored significantly higher on the posttest than the other conditions. Similarly, Liao, Chen, and Shih (2019) examined the use of instructional videos in game-based learning coupled with collaboration in a 2 × 2 factorial design (i.e., showing vs. not showing instructional videos as one factor, and collaboration vs. individual gameplay as the second factor). The participants in this study (n = 109) were seventh-grade students randomly assigned to one of four experimental groups. The authors found that game-based learning is effective with instructional videos used as learning supports. Specifically, results revealed that use of an instructional video in collaborative digital game-based learning significantly reduced both intrinsic and extraneous cognitive loads—both of which are detrimental to learning.

Overall, these studies suggest that content-related supports are effective in promoting learning outcomes, but without direction or incentives, students are less likely to use them, especially if they are not embedded in the game environment.

Game-related supports are designed to help students manipulate and interact with game mechanics to solve game tasks. One of the common game-related supports is the complete solution, which presents expert solutions to game levels for students to imitate (Lang & O'Neil, 2008). Some researchers, however, have criticized the inclusion of complete solutions in games, stating that overusing them could lead to simply replicating the solution (the “how”) without deep thinking (the “why”). For example, Kao et al. (2017) found that providing partial solutions (i.e., hints) produced greater learning compared to either the provision of complete solutions or no supports. Additionally, they found no significant difference between complete-solution supports and the no-support conditions. The authors suggested that students who received complete solutions may have just replicated the solution without thinking about the concepts underlying the levels. At the same time, complete solutions are necessary to reduce frustration and help struggling students, particularly when tasked with solving difficult levels (Shen & O’Neil, 2006). Thus, some provisions need to be made to ensure that students have access to complete solutions, but not overuse them.

There are two main methods used to reduce players’ reliance on complete solutions: (a) restricting the availability of complete solutions (e.g., make them available only for very difficult levels), and (b) restricting the use of complete solutions by adding costs per usage (Schrader & Bastiaens, 2012a, b; Sun et al., 2018). For example, Sun et al. (2018) implemented an incentive system to regulate the reliance on complete solutions. Two types of incentive mechanisms were implemented. The point-earned mechanism awarded 1000 points for solving levels without using any partial (i.e., hints) or complete supports, and 10 points for solving levels using partial or complete supports. The point-loss mechanism would deduct 2 points for using partial solutions and 10 points for using a complete-solution support. Results showed that students in the “point-loss” condition used significantly fewer supports than those in the “point-earn” condition.

Based on the preceding findings, students should access content-related supports to maximize their learning, and use complete solutions judiciously to reduce their frustration when faced with difficult levels. In this study, we investigated the effects of an in-game incentive system (see Fig. 1) to encourage (and discourage) relevant behaviors. Specifically, we investigated the use of an incentive system to regulate the use of complete solutions (spending money) and encourage the use of content-related supports (earning money).

Toward that end, based on the theories about motivation in educational games such as Self-determination Theory (Deci & Ryan, 1985; Ryan & Deci, 2000) and other motivational and behavioral theories (e.g., Ferster & Skinner, 1957; Keller (1979), and the elements of well-designed video games discussed earlier (e.g., rules, clear goal, feedback and interactivity, and require active participation), we created a motivational architecture intended to help enhance learning by regulating students’ behaviors regarding the use of learning supports in educational games (Fig. 1).

Our motivational architecture includes five main elements with brief justifications for their inclusion:

  • (1)

    The foundation of the educational game: This includes the game genre (e.g., puzzle, sandbox), the game mechanics (i.e., rules and tools), the underlying subject matter (e.g., physics, mathematics, biology), and the learning supports for enhancing learning and game performance (i.e., content- or game-related—more about the supports in the next section).

  • (2)

    The intrinsic motivators: This relates to the general game elements that make gameplay intrinsically appealing and motivating (e.g., clear goal, narrative, interactivity, feedback, incremental challenges), and provide opportunities for gaining high levels of competence, autonomy, and relatedness.

  • (3)

    The extrinsic motivators: This layer represents an incentive system aiming to strengthen desired behaviors with rewards, and opportunities for using the rewards (e.g., allowing the student to buy game-related items in a game store) and weaken undesired behaviors with penalties.

  • (4)

    Use of the incentive system: This relates to students' behaviors relative to the incentive system (e.g., attempting to get rewarded by accessing content-related supports).

  • (5)

    Effects of the incentive system. This layer shows student improvement in game performance, learning, and enjoyment. If accessing content-related supports was rewarded, and those supports were well-designed and useful, students may continue using the supports even when they are not incentivized after the first use. The extrinsic motivators, if designed well, may end up enhancing the intrinsic motivation of the students during gameplay (Habgood & Ainsworth, 2011).

The first and second elements of this proposed architecture have been listed as necessities for well-designed educational games (e.g., Gee, 2005; Prensky, 2001; Shute & Ke, 2012). The third, fourth, and fifth elements relate to an incentive system designed to regulate students’ behaviors. We included those elements based on the past research on incentive systems in video games in general and educational games, in particular. In the current study, we employed all five elements in our educational game and specifically focused on the connection between the intrinsic and extrinsic motivators—i.e., elements 4 and 5. Next, we briefly introduce the game Physics Playground.

Physics Playground (PP; Shute, Almond, & Rahimi, 2019) is a 2D computer-based game created to help middle and high school students learn Newtonian physics such as the laws of force and motion, linear momentum, and torque (see Fig. 4 for the concepts that PP levels cover). The goal in this game, for all levels, is to direct a green ball to hit a red balloon. There are two level types: sketching and manipulation. To solve sketching levels, students draw simple machines (i.e., ramps, levers, pendulums, and springboards) to maneuver the ball to the target balloon (Fig. 2). To solve manipulation levels, students interact with various sliders to change physics parameters (i.e., gravity, air resistance, mass and bounciness of the ball), and also manipulate external forces such as puffers and blowers (Fig. 3).

Multiple studies have been conducted using PP to investigate various research questions—from the validity of various stealth assessment measures (Shute, 2011), to the effectiveness of the game in enhancing students' physics understanding. For example, PP was used to measure creativity (Shute & Rahimi, 2020) and persistence (Ventura & Shute, 2013) using stealth assessment. Stealth assessment, in short, is an assessment method that uses the evidence-centered design framework of assessment (ECD; Mislevy, Steinberg, & Almond, 2003) to design and develop various models that are included directly into the fabric of the game. Overall, PP consistently has shown a positive impact on students’ conceptual physics learning, and both boys and girls equally enjoy playing PP (e.g., Shute et al., 2015; Shute, Ke, et al., 2019; Bainbridge et al., 2020).

Over the past decade, PP has undergone multiple design iterations, and various features have been included in it using design-based research studies. In its most recent design, we have included an incentive system to address an issue we noticed in several of our earlier studies. For instance, results from a recent study (Bainbridge et al., 2020) showed that students, overall, tended to overuse the game-related supports (i.e., solution videos) and neglect the content-related supports (i.e., physics videos). In the current study, we created an incentive system with both point-earn and point-loss mechanisms in PP to increase the use of content-related learning supports and regulate the use of game-related supports. Through the red Help button (shown in Fig. 3), students could access the learning supports (both content-related and game-related supports).

Through three usability studies and a final learning study, we designed, developed, and tested the final set of learning supports in PP (Shute, Ke, et al., 2019). More details about the incentive system and the learning supports are discussed in the Method section. The following are our research questions:

  • RQ 1. What is the effect of the incentive system on using content-related and game-related supports?

  • RQ 2. What is the effect of the usage of content-related and game-related supports on learning and game performance?

  • RQ 3. What is the effect of earning and spending money on learning and game performance?

Section snippets

Participants and procedure

We used a one-group pretest-posttest research design in this study. The sample consisted of 199 students (9th to 11th grade) from a large K-12 school in the southeastern US. Students self-identified as male (n = 105) and female (n = 95), with a wide range of ethnicities. Self-reported ethnicities representing more than 1% of the respondents were: White (n = 83), Black or African American (n = 62), Asian (n = 8), Hispanic (n = 15), Other (n = 6), Black or African American and White (n = 6), and

The effect of incentive system on using the learning Supports—RQ 1

To examine the effects of the incentive system in accessing physics videos and solution videos, we parsed the log data and computed the variables related to the frequency of accessing physics videos and solution videos. The descriptive statistics show that, on average, students accessed physics videos (M = 3.90, SD = 4.55) more than solution videos (M = 2.55, SD = 2.84). This finding contrasts with findings from our previous studies (without an incentive system), suggesting that our incentive

Discussion and conclusion

In this study, we proposed and evaluated a 5-element motivational architecture, based on relevant motivational and behavioral theories, for use in the design of educational games: (1) Foundation of the educational game, (2) Intrinsic motivators, (3) Extrinsic motivators, (4) Use of an incentive system, and (5) Effects of the incentive system. According to the literature (e.g., Kao et al., 2017) and our previous research (Bainbridge et al., 2020), students tend to overuse solution videos when

Implications

Based on these findings, we compiled a list of implications that can be useful for educational game designers and game-based learning researchers to consider:

  • 1)

    Include an incentive system with point loss or appropriate cost mechanism to discourage overuse of viewing solutions. We do not recommend excluding such videos or supports as they might be useful for students who are struggling or frustrated.

  • 2)

    Include a game store with customizable items to allow students to enhance their sense of autonomy,

Limitations and future steps

The first limitation of this study is the research design. We used a one-group research design because we conducted this study as part of a larger experiment with 263 students (4 groups, with one control group) to investigate the effects of adaptive sequencing of game levels (Shute et al., 2020). Future research can investigate similar research questions using an experimental research design with at least two groups (e.g., one with and the other without an incentive system). That kind of study

Declaration of competing interest

We wish to confirm that there are no known conflicts of interest associated with this manuscript.

Credit author statement

Seyedahmad Rahimi: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing - original draft, Writing - review & editing. Valerie Shute: Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, review & editing. Renata Kuba: Project administration, Visualization, Writing-review &

Acknowledgments

This work was supported by the National Science Foundation, United States [award number #037988] and the Department of Education, United States [award number #039019]. We also would like to acknowledge Russell Almond, Fengfeng Ke, Curt Fulwider, Zhichun Liu, Chen Sun, and Jiawei Li for helping in different phases of this project.

References (62)

  • M.L. Biles et al.

    Designing digital badges for educational games: The impact of badge type on student motivation and learning

  • E.A. Boyle et al.

    An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games

    Computers & Education

    (2016)
  • J.A. Blanco-Herrera et al.

    Video games can increase creativity, but with caveats

    Creativity Research Journal

    (2019)
  • K. Bainbridge et al.

    Does embedding learning supports enhance transfer during game-based learning? A case study with Physics Playground

    Learning and Instruction

    (2020)
  • M. Csikszentmihalyi

    The flow experience and human psychology

  • E.L. Deci et al.

    Intrinsic motivation and self-determination in human behavior

    (1985)
  • G. Denis et al.

    June). Motivation-driven educational game design: Applying best practices to music education

  • S.K. D'Mello et al.

    Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features

    User Modeling and User-Adapted Interaction

    (2010)
  • S. Egenfeldt-Nielsen

    Overview of research on the educational use of video games

    Nordic Journal of Digital Literacy

    (2006)
  • Entertainment Software Association

    Essential facts about the computer and video game industry

    (2019)
  • C.B. Ferster et al.

    Schedules of reinforcement

    (1957)
  • J.P. Gee

    What video games have to teach us about learning and literacy

    Computer Entertainment

    (2003)
  • J.P. Gee

    Learning by design: Good video games as learning machines

    E-learning and Digital Media

    (2005)
  • J.P. Gee

    Good video games+ good learning: Collected essays on video games, learning, and literacy. Peter Lang

    (2007)
  • M.J. Habgood et al.

    Motivating children to learn effectively: Exploring the value of intrinsic integration in educational games

    The Journal of the Learning Sciences

    (2011)
  • J. Hamari et al.

    Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning

    Computers in Human Behavior

    (2016)
  • D. Ifenthaler et al.

    Assessment for game-based learning

  • Y.B. Kafai

    Learning design by making games: Children's development of strategies in the creation of a complex computational artifact

  • G.Y.M. Kao et al.

    Customizing scaffolds for game-based learning in physics: Impacts on knowledge acquisition and game design creativity

    Computers and Education

    (2017)
  • A. Kapoor et al.

    Automatic prediction of frustration

    International Journal of Human-Computer Studies

    (2007)
  • J.M. Keller

    Motivation and instructional design: A theoretical perspective

    Journal of Instructional Development

    (1979)
  • J.M. Keller

    Motivational design for learning and performance: The ARCS model approach

    (2009)
  • D.L. King et al.

    Motivational differences in problem video game play

    Journal of Cybertherapy and Rehabilitation

    (2009)
  • D.L. King et al.

    Assessing clinical trials of internet addiction treatment: A systematic review and CONSORT evaluation

    Clinical Psychology Review

    (2011)
  • G.D. Konetes

    The function of intrinsic and extrinsic motivation in educational virtual games and simulations

    Journal of Emerging Technologies in Web Intelligence

    (2010)
  • R. Kuba et al.

    Using the first principles of instruction and multimedia learning principles to design and develop in-game learning support videos. To appear in Educational Technology Research and Development (ETRD)

    (2020)
  • J. Lang et al.

    The effect of presenting just-in-time worked examples for problem solving in a computer game

  • H. Leemkuil et al.

    Is support really necessary within educational games?

    Educational Games as Intelligent Learning Environments

    (2005)
  • C.-W. Liao et al.

    The interactivity of video and collaboration for learning achievement, intrinsic motivation, cognitive load, and behavior patterns in a digital game-based learning environment

    Computers & Education

    (2019)
  • R.J. Mislevy et al.

    Focus article: On the structure of educational assessments

    Measurement: Interdisciplinary research and perspectives

    (2003)
  • Cited by (21)

    • Timing of learning supports in educational games can impact students’ outcomes

      2022, Computers and Education
      Citation Excerpt :

      Additionally, future studies could investigate the effects of students' control on accessing learning supports vs. forced learning supports. One of our studies (Rahimi et al., 2021) investigated the effects of an incentive system in bringing students' attention to learning supports by incentivizing the first-time use of the supports. We found that students used the learning supports even when viewing the supports was not incentivized.

    • Improving serious games by crowdsourcing feedback from the STEAM online gaming community

      2022, Internet and Higher Education
      Citation Excerpt :

      Gold was provided as an incentive system that could be spent at the in-game store for novelty and vanity items. There are a range of benefits for incentive systems of this nature, such as enhancing the player's sense of autonomy, and allowing a break from the gameplay experience (Rahimi et al., 2021). In The King's Request, there was no mention of a store until halfway through the game, which may have been the catalyst for feedback received from the Steam community when players highlighted a broad confusion surrounding the point of finding gold.

    • Game induced emotion analysis using electroencephalography

      2022, Computers in Biology and Medicine
      Citation Excerpt :

      Secondly, video games utilize cognitive and behavioral aspects of personality [13,14]. With this in view, we believe that video games can be used to understand the underlying psychology of the subject to design effective UI/UX, not just for games but for various other applications [15,16]. Thus, the contributions of the paper include:

    • Games as stealth assessments

      2023, Games as Stealth Assessments
    View all citing articles on Scopus
    View full text