Game mechanics and why they are employed: What we know about gamification so far

Katherine Lynn Bevins
The University of Tennessee - Knoxville

Craig Dennis Howard
The University of Tennessee - Knoxville

Introduction

Game-like elements have been present in instructional interventions for years; however, our vocabulary and ability to analyze these instructional tactics is relatively new. Online learning environments present uniquely accessible contexts in which to analyze the use of game-like instructional tactics. A synthesis of the literature surrounding gamification is both timely for scholars of instructional interventions and holds much utility for instructional designers who might employ these tactics. We will start with the term itself.

In Kapp's (2012) book, The Gamification of Learning and Instruction, he discusses how gamification is simply a new and popular term that depicts instructional tactics employed by teachers whereby they "embed stories in the form of case studies to wrap experiences for learners, create challenges to engage learners, and set goals and provide feedback on progress while providing a safe environment for learners to practice skills" (p. 13). Gamification identifies an approach, rather than a design process or full set of instructional methods.

Gamified approaches are now accessible to study in online environments. With the affordances of online learning, and with the massive popularity of gaming in general, new vocabularies have emerged allowing educational researchers to better dissect and analyze the instructional approaches that use game-like design features. When scholars of education speak of gamification, they are generally referring to one of two specific meanings. The first is that gamification is the "use of video game elements in non-gaming systems" (Deterding, Sicart, Nacke, O'Hara, & Dixon, 2011, p. 2425). While the Deterding et al. (2011) definition is used frequently, it is not specific to education. For this reason, we focus on the work of Karl Kapp, whose concept of gamification takes into consideration all games, not just video games, as well as the reasons behind using the approach. We focus on Kapp's definition of the term gamification as he is the leading theorist on gamification. Kapp's theory of gamification has been included in the Instructional-designs theories and models: The learner-centered paradigm of education (Reigeluth, Beatty, & Myers, 2016) as an "emerging instructional-design theory" (p. 317) to help develop the learner-centered paradigm. In this book, Kapp has written the most comprehensive theoretical deconstruction of the concept. He also provided the most detailed instructional strategy in his book The gamification of learning and instruction fieldbook: Ideas into practice (2013). We refer to Kapp's definition of gamification offered in his book The Gamification of Learning and Instruction (2012) because it is the most accessible version and is the most widely cited amongst the articles we reviewed. According to Kapp (2012), gamification "is [more broadly] using game-based mechanics, aesthetics and game thinking to engage people, motivate action, promote learning, and solve problems" (p. 10).

Kapp's (2012) definition incorporates several elements that he considers to be the main elements of gamification. The elements of gamification are game-based mechanics, aesthetics, and game thinking (Kapp, 2012). Game-based mechanics are a group of tactics, rather than simply one tactic. Game-based mechanics can include the use of badges, points, leaderboards, progress bars, goals or challenges, a storyline, or immediate feedback (Kapp, 2012). These game-based mechanics are not the only elements involved in the process of gamification, but they are important to the overall process. The second element is aesthetics. Aesthetics refers to the well-designed experience necessary to help influence the willingness of the learner to accept gamification (Kapp 2012). If learners do not enjoy the experience, then their overall enjoyment and engagement with the game could decrease. The final element of gamification for Kapp (2012) is game thinking. Game thinking is a concept that is not widely discussed within the realm of gamification in online learning. This element consists of "thinking about an everyday experience like jogging or running and converting it into an activity that has elements of competition, cooperation, exploration, and storytelling" (Kapp, 2012, p. 11). This element involves a shift in how learning and curriculum are approached.

For Kapp (2012), however, not only must gamification include the elements of game-based mechanics, aesthetics and game-thinking, but it must also strive to engage, motivate, promote learning, and solve problems. While all educators strive to support motivation and engagement, the idea behind gamification is to "give direction, purpose or meaning to behavior and actions" (Kapp, 2012, p. 12). These outcomes are pedagogical aspirations rather than design features.

 Kapp (2012) acknowledges that these elements of gamification are based on techniques that have been used by instructional designers, teachers, and professors for years. The difference is that gamification gives us a "new way [of] weaving together those elements into an engaging game space that both motivates and educates learners" (Kapp, 2012, p. 12). Kapp (2012) also includes in his definition of gamification the way in which games have a cooperative nature, which helps groups of learners focus on solving a problem rather than just an individual learner.

This review examines the literature surrounding gamification from three perspectives that struck us as having a large amount of utility: the components that are gamified, the aspects of pedagogical value against which their utility is measured, and the means by which this has been accomplished— e.g., the tools or design features. The research questions we asked were then:

1. What are the most studied mechanics of gamification?
2. What discrete pedagogical aspirations are being examined with the use of gamification?
3. What learning activities or educational tools are being gamified?

Methods

Overview

Following guidelines by Paré et al. (2015), we conducted a descriptive literature review of 229 studies of which 17 qualified for final in-depth analysis. A descriptive review seeks "to determine the extent to which a body of empirical [our emphasis] studies in a specific research area supports or reveals any interpretable patterns or trends with respect to pre-existing propositions, theories, methodologies, or findings" (Paré et al., 2015). We only considered empirical studies. The major distinction that makes this review descriptive, rather than narrative, is our use of structured search methods, and our extraction of "certain characteristics of interest from each study" (Paré et al., 2015). We focused our analysis on data within the studies that provided insight into the game mechanics studied, their associated pedagogical aspirations, and the tools to which these applied. Descriptions of designs, advocacy for gamification, and essays in support of gamification would not be included.

 In this review, we conducted three rounds of searches. The first round consisted of a Boolean search on three different databases. The three databases were Academic Search Complete, Educational Research Information Center (ERIC), and Google Scholar. The four search terms used with gamification in the Boolean search were as follows:

1. Online Learning
2. Distance Learning
3. Distance Education
4. Learning

We conducted a reverse search, a search that finds articles citing a certain author, for the second round of literature collection. We searched via Google Scholar for studies that cited Kapp (2012), not only because it is a seminal work, but because it is also the only cited definition of gamification as it pertains to education in the scholarship we found in collection round one.

In our third and final round of collection, we examined the reference lists of all the studies collected in the first two rounds and added any article that was cited in more than one study. We reasoned this would provide a thorough collection of the literature on the topic, from which we could narrow the body of literature to find utility for instructional design. We applied four criteria for articles for inclusion in the final literature review:  

1. Include empirical data in the context of higher education
2. Use gamification in an online setting (i.e. in a learning management system, online learning activities, and online assessments)
3. Use gamification by Kapp's (2012) definition of using game-like elements in non-game contexts
4. Must not use gamification to mean whole educational games or serious games

Details of the collection process development

We searched Academic Search Complete and ERIC with the key terms and retrieved a total of 159 peer-reviewed studies. The first author looked at all 159 peer-reviewed studies. After applying the criteria, we found 11 studies to use in our review. For these two databases, it was possible to scan all the returns. We also searched Google Scholar with the key terms and retrieved 18,800 returns. After applying our criteria, we found four studies that were not retrieved from the other databases in the first fifteen returns. We searched a subsequent 15 more, which returned nothing. We then stopped reading returns from Google Scholar. After the first round of our search, we had a total of 15 studies for our review.

To ensure we had an accurate and well-rounded collection of studies, we then searched Google Scholar for studies that cited Kapp (2012). We used Kapp's definition of gamification to help define our search because he is the leading theorist on gamification and has done the most work with defining and explaining gamification for instructional use (Kapp, 2012; Kapp; 2013; Reigeluth, Beatty, & Myers, 2016). We do not base our review solely on Kapp's (2012) definition but on the whole concept of gamification on which he has worked. The Google Scholar reverse search for studies citing Kapp (2012) resulted in 1,240 results. We examined the first 40 studies. We found six of the 40 studies had already appeared in our first round of collection. Of the other 34 studies found, only one met our criteria for this literature review. After the second round of our search, we had a total of 16 studies. In a final procedure applied to the literature collection, we examined the 16 studies that had met our criteria for common citations. This resulted in one more study for inclusion as that study also met our criteria, making a data set of 17 studies altogether.

Results

Results of initial analysis stemming from discarded studies

While searching for articles examining gamification, we found four different literature reviews that had been published about gamification that were noteworthy but not relevant for inclusion in our data set (Dicheva, Dichev, Agre, & Angelova, 2015; Hamari, Koivisto, & Sarsa, 2014; Nah, Zeng, Telaprolu, Ayyappa, & Eschenbrenner, 2014; Xu, 2012). Two literature reviews examined articles discussing gamification in the context of education/academia (Dicheva et al., 2015; Nah et al., 2014). Xu (2012) reviewed the approaches being used in gamification design, the available gamification platforms, and the related terms often used along with gamification such as serious games. Hamari et al. (2014) reviewed the literature to create a framework of the type and number of game-based mechanics (called motivational affordances in the article) used in the empirical research, and they also reviewed the psychological and behavioral outcomes associated with the affordances. Both Xu (2012) and Hamari et al. (2014) reviewed empirical studies in several fields, while Nah et al. (2014) and Dicheva et al. (2015) were focused on the field of education. Nah et al. (2014) focused on the different types of game elements that were found in the empirical studies examining gamification within educational contexts, while Dicheva et al. (2015) focused on creating a systematic mapping analysis via plotting educational contexts by their studied game elements. While all appeared useful resources, none juxtaposed elements of gamification and pedagogical values such that it would prove useful for us as instructional designers.

A second noteworthy result of the articles discarded from the final dataset is a common misnomer among titles and keywords of some studies. Several studies which were titled using the term gamification or game-like design were actually studying the use of wholly intact educational games within educational contexts. These studies were not included in this literature review because we limited this review in scope by Kapp's (2012) definition.  

These two previous results, as well as our focus on online learning, are what limited our review to a small number of articles. We went to great pains to enlarge our sample size; however, we were not able to enlarge it, as the criteria would have resulted in a diffuse analysis of the discussion of the literature. Our focus was on an analysis of this instructional design specifically in online environments.

Dataset overview

Of the 17 studies examined in this review, 13 of these studies (76.4%) were empirical studies found in journals, and the other 4 studies (23.5%) were conference proceedings. We found studies from ten different journals. Of the 10 journals, 8 of these journals have a title with at least one word relating to computers, technology, or e-learning, as can be seen in Table 1. The other two journal publications were journals with titles related to education and learning. The journal Computers and Education was the most represented publishing venue, with four studies total. 

Results relevant to RQ #1: Game mechanics studied

To present the areas of gamification that have received the most scrutiny and thus might be the most advanced, we address each mechanic separately. We present our results first with this overview. We then have arranged the mechanics studied by order of frequency: leader boards, badges, goals/challenges, points, rewards, immediate feedback, progress bars, and storylines. It should be noted that only four studies (4/17) investigated a single game mechanic. 

Components of gamification are referred to as game mechanics in this body of literature. According to Kapp (2012), game mechanics are "crucial building blocks used during the gamification process" (p. 11). We found seven different types of game mechanics examined in these articles: leaderboards, badges, points, goals, immediate feedback, progress bars, and storyline. No single study addressed all the mechanics we found in our review; on average, the articles we found examined 2.8 different game mechanics. However, 13 of the 17 articles examined more than one game mechanic in their gamification approach, with the most common number being two game mechanics studied in six articles. The game-mechanic "badges" was examined in 11 of the 17 articles, making it the game-mechanic examined the most. The game-mechanics of badges and leaderboards were examined together more often than separately. Of the 11 studies that examined the use of badges, eight of those same studies also examined the use of leaderboards. 

Arranging the studies chronologically, we see that the articles published in 2015 limited the amount of game-mechanics they examined to one to three game-mechanics examined. The articles in 2015 on average examined two game-mechanics, while the articles in 2013, 2014, and 2016 on average examined three or more game-mechanics. This suggests that there does not seem to be a linear trend in how researchers approach mechanics in unison.

Badges. Badges are prizes that incentivize learning performances within the gamified system. Badges are typically "graphical icons that appear[s] to the user after reaching an achievement" (Hakulien et al., 2013, p. 47). Badges are typically used as a reward system, meaning that they do not open new levels or other exciting facets of a game, but they simply reward the student for accomplishing a goal, challenge, or even a desired behavior within the gamified activity (Hakulien et al., 2013). As can be seen in Table 2, eleven studies examined the use of badges or rewards in gamification, making badges the most popular game mechanic examined. Codish and Ravid (2014) examined the perceived playfulness of different game mechanics in a gamified learning activity and found that extraverts enjoyed the use of the badges significantly more than introverts. The results of the study by Codish and Ravid (2014) also suggest that both extraverts and introverts perceive badges to be playful.

Two studies examined only the use of badges within their gamified intervention (Hakulien et al., 2013; Tu et al., 2015). In the study by Hakulien et al. (2013), the different achievement badges that were used in the gamified activity were broken down into several different categories in order to determine the effects, if any, the different types of badges had on student behavior: time management, carefulness, and learning. Overall, Hakulien et al. (2013) found that some achievement badges influenced student behavior; "however, the group of students that changed their behavior because of the badges was not big" (p. 51). Their results show that only 22% of the students in the gamified treatment group earned at least one badge.  Hakulien et al. (2013) found that with computer science major students the time management badges had a stronger effect than these badges had with the minor students. However, the carefulness badges had a stronger effect with the minor students than with the major students. In regard to the learning badges, students in the treatment group earned more points on the assignments in the learning badges category than the control group, although this difference was not statistically different.

Hanus and Fox (2014) found that using badges as a form of giving rewards harms motivation. O'Donovan et al. (2013) found that overall students felt that the gamification of the course had improved their engagement and understanding; however, badges were ranked the least motivating when compared to the game mechanics of leaderboards, points, and progress bars. Olson et al. (2015) also found mixed results in their qualitative case study in regard to badges. While students found the badges interesting, the badges were not the principal motivation for students. Olson et al. (2015) examined the use of badges and progress bars in two different courses, a course on game-based learning and a course for university teachers on e-learning. The students in the game-based learning course were indifferent to the digital badges.  Their principal motivation was the grade in the course. Some of these students even found the badges to be redundant and disturbing (Olson et al., 2015). The university teachers found the badges to be interesting but less meaningful than the progress bar. This course was not mandatory for the university teachers; therefore, Olson et al. (2015) concluded that these learners were intrinsically motivated without the need for extrinsic motivation that the badges can provide. As the most popular game mechanic studied, the majority of the studies suggest that badges, when disassociated from the other game-based mechanics, do not support learner motivation but are perceived to be playful.

Leaderboards. Seven studies examined the use of leaderboards or public rankings (public meaning shown to the whole class) as an element of gamification (see Table 2). Leaderboards are a public ranking system to allow students to see how they rank in comparison to their peers.  Leaderboards can also promote "competition and a sense of belonging to a similar minded group" (O'Donovan et al., 2013, p. 244). The majority of these studies suggest that leaderboards within gamification support learner motivation and academic performance.

O'Donovan et al. (2013) found that students ranked leaderboards as the most motivating aspect of the gamification model, above goals, points, progress bars, and storyline. Overall, O'Donovan et al. (2013) found that students felt there was an improvement in their understanding and in their engagement with the gamified approach. Cheong et al. (2013) found that students reported the leaderboard promoted competitiveness, knowledge acquisition, and learning as part of the gamified implementation. Frost et al. (2015) also found that leaderboards positively affected student's perceptions of gamification. Frost et al. (2015) noted that students found the comparative and competitive influence of the leaderboard as positive.  Students also noted a stronger connection to the class with the use of the leaderboard, despite their own ranking on the leaderboard. Frost et al. (2015) do note, however, that several students believed the leaderboard to be discouraging. Ranked and unranked students on the leaderboard also reported having a negative experience (Frost et al., 2015).

In regard to leaderboards, Codish and Ravid (2014) found that extraverts who found the entire gamified system to be playful did not find the leaderboard to be playful. Codish and Ravid (2014) examined the effect of gamification on perceived playfulness of introverts and extraverts. Both introverts and extraverts enjoyed being the leader on the leaderboard, but according to Codish and Ravid (2014) extraverts determined the leaderboard not to be playful since they "would prefer to be able to brag about it in real-time and in a face-to-face situation" (p. 143).  

The study by Christy and Fox (2014) was the only study in the group who explored just the game mechanic of leaderboards. Christy and Fox (2014) examined the effect of leaderboards on social comparison and stereotype threat. The results show that the women who were presented a male-dominated leaderboard performed better on the quiz than the women who were shown a female-dominated leaderboard. However, the results also show that the women in the female-dominated leaderboard group had a stronger academic identification than those women in the male-dominated leaderboard group.

In their qualitative analysis, Dominguez et al. (2013) received both positive and negative comments regarding the use of a leaderboard in their gamified LMS. While some students liked the competitive atmosphere that the leaderboard gave to the class as well as how it affected their contribution to the participation score, there were other students who disliked the competition and the uneasiness that the leaderboard created in the class (Dominguez et al., 2013).

There was one study examining the use of leaderboards that suggests that gamification does not support student motivation and learner outcomes. In relation to leaderboards, Hanus and Fox's (2014) results suggest "encouraging competition and social comparison via a digital leaderboard harms motivation" (p. 159). Overall, Hanus and Fox (2014) found that the gamification approach can harm both educational outcomes as well as intrinsic motivation. When viewed apart from the other game mechanics, the results of the studies examined suggest that leaderboards create and promote competitiveness, but the results of two other studies suggest leaderboards are not perceived as playful and can harm motivation.  

Points. We found seven studies that examined gamification by using points (Cheong et al., 2013; Codish & Ravid, 2014; Dominguez et al., 2013; Moccozet et al., 2013; O'Donovan et al., 2013; Rose et al., 2016; Ténorio et al., 2016). Of these seven articles, only three report results they found specifically in relation to points. When examining the perceived playfulness of game mechanics in a gamified course, Codish and Ravid (2014) found a strong relationship between the points system and perceived playfulness with introverts. Codish and Ravid (2014) also noted that students related the points system to the progress bar rather than the leaderboard. This lack of relationship between the points and the leaderboard could have affected the competition, and therefore the perceived playfulness, of the leaderboard. O'Donovan et al. (2013) used both experience points (the points students gained when completing assignments, attending lectures, and participating in class) and steam points (points that are earned as a result of high experience points and can be used to purchase assignment do-overs, extensions, and hints for completing some of the activities). The experience points were used to rank the students.  There were five different titles or ranks that students could achieve depending on the amount of experience points they gained. Both the rank and the steam points were found to be less motivating than the leaderboard and more motivating than the progress bars and badges. Moccozet, Tardy, Opprecht, and Leonard (2013) studied only the use of points in an online peer assessment activity. Overall, the results show that this "collaborative learning platform [using points] encourages students to contribute and collaborate" (Moccozet et al., 2013, p. 175). When disassociated from the other game-based mechanics, the results of the studies examined suggest that points do support learner motivation and engagement and are perceived as playful.

Immediate Feedback. Immediate feedback lets students know almost instantaneously how they are progressing in the learning activity or in the course. It is possible to use other game mechanics, i.e. points, badges, and rewards, as a form of immediate feedback. There were six studies, however, that discussed immediate feedback as a separate game mechanic used in their gamified learning interventions (Buckley & Doyle, 2016; Cheong et al., 2013; Fleischman & Ariel, 2016; Geelan et al., 2015; O'Donovan et al., 2013; Rose et al., 2016). Only two of these six studies note results specific to the use of immediate feedback. Fleischman and Ariel (2016) reported on students wanting the immediate feedback that they received in the gamified learning intervention, which shows engagement with the activity. Geelan et al. (2015) used immediate feedback as a game mechanic in their online learning activity in order to "allow students to receive more active and immediate indications of their progress" (p. 6). Students reported that the positive and negative feedback they received within the online gamified learning activity helped them improve their content knowledge and that the immediate feedback helped their motivation and engagement in the activity as well (Geelan et al., 2015). When viewed as a group, these studies suggest immediate feedback supports learner motivation and engagement as well as learning outcomes.

Goals/Challenges. Goals that challenge a learner to master objectives can be as simple as completing an activity or as difficult as mastering multiple objectives through the successful completion of several different sets of learning activities. We found a total of four studies that examined the use of goals or challenges as a game mechanic. All four articles will be discussed later (De-Marcos et al., 2016; Dominguez et al., 2013; Fleischman & Ariel, 2016; Ténorio et al., 2016) in the pedagogical aspirations and gamification targets sections. Because goals/challenges has not been examined when separated from the other game-based mechanics, more research needs to be done in examining how goals/challenges affect pedagogical aspirations. 

Progress Bars. A progress bar is a tool used for participants to track their progression or to orient themselves to their location in a learning activity. We found four studies which examined the use of progress bars in their gamified learning interventions (Codish & Ravid, 2014; Geelan et al., 2015; O'Donovan et al., 2013; Olson et al., 2015). Codish and Ravid (2014) found overall that the relationship between playfulness and progress bars was positive for both introverts and extraverts in quasi-experiment 1, and in quasi-experiment 2, there was a negative relationship between the progress bars and perceived playfulness for introverts. O'Donovan et al. (2013) found that students ranked the progress bar along with an end prize and badges lower than the other game mechanics used, leaderboard, points, and rank. The students rated the progress bar as one of the least motivating game mechanics. In Olson et al.'s (2015) course on game-based learning, the participants found the progress bar "as a positive aid for increased overview" (p. 451). The university teachers in the e-learning course rated the progress bar as either good, very good, and excellent (Olson et al., 2015). The authors interpret this rating to mean that the participants found the progress bar useful and motivating. Olson et al. (2015) also noted the usefulness of the progress bar from the teacher's perspective. The progress bar can be used by the teacher for tracking the progress of the participants, watching the engagement of the participants throughout the learning intervention, and checking to see if participants are preparing before attending the seminars (Olson et al., 2015). When examined apart from the other game-based mechanics, progress bars were viewed as playful, were rated as useful and motivating, and were suggested to be useful for teachers; however, one study did have students find the progress bar to be the least motivating game mechanic.

Rewards. Most gamification implementations that use rewards do so in the use of badges, points, and leaderboards. However, there were three studies we found that used a different type of reward system (Buckley & Doyle, 2016; Codish & Ravid, 2014; Dominguez et al., 2013). Codish and Ravid (2014) found that introverts did not perceive the rewards system to be playful while a small group of the extraverts in their study did perceive the rewards system to be playful. Codish and Ravid (2014) did find, however, that introverts enjoyed the rewards significantly more than extraverts despite the lack of relationship between rewards and perceived playfulness. Dominguez et al.'s (2013) qualitative analysis of student feedback suggests that the rewards system used (achievements) is an "innovative, fun, and encouraging way to represent progress within an online educative experience" (p. 391). When disassociated from the other game-based mechanics, rewards were viewed to have no perceived playfulness but to support learner motivation.

Storylines. A storyline contextualizes learning in the hope of creating a more meaningful experience for the learner. We found two studies that used a storyline in their gamified context (Frost et al., 2015; O'Donovan et al., 2013). Frost et al. (2015) found student interest to be significantly impacted by the gamified learning management system. The authors contribute this rise in interest to the storyline and the language used in the gamification. Students found the "gamification made the course more exciting and fun" (Frost et al., 2015, p. 65). O'Donovan et al. (2013) found that while students believed the learning activity to be improved by the story and theme, they felt the integration of the storyline into the learning activity was insufficient. These studies suggest that when disassociated from other game-based mechanics, the storyline supports an increase in student satisfaction, but the storyline needs to be effectively integrated into the gamified context.

Discrete pedagogical aspirations

The 17 studies in this literature review examined gamification tactics in unison with five different pedagogical aspirations. The most common aspiration was learning outcomes (53%, 9 of 17 studies), and the least commonly studied was personality studies (11%, 2 of 17 studies). The others were learner motivation (41%, 7 of 17 studies), learner perceptions (35%, 6 of 17 studies) and learner engagement (24%, 4 of 17 studies).

Learning Outcomes. Learning outcomes was the most examined pedagogical aspiration in the studies we found. Learning outcomes includes student scores on quizzes, exams, course grades, activity points, and learning activities. We found eight studies that examined the effect of gamification on learning outcomes. Overall, the body of literature surrounding gamification defines learning outcomes in different ways. Three studies measured learning outcomes as scores on a quiz or an exam. Three studies measured learning outcomes on a collection of assignments rather than just one assignment. Two studies measured learning outcomes as whole course grades or overall activity points.

We found three studies that measured learning outcomes as learners' scores on a quiz or exam. Buckley and Doyle (2016) found an increase in student knowledge on a post quiz after participating in the gamified learning activity. Christy and Fox (2014) found that when women were presented with a female-dominated leaderboard, they performed worse on a math quiz than the group of women who were presented with a male-dominated leaderboard. Despite their lower quiz scores, Christy and Fox (2014) also found that the women in the female-dominated leaderboard group had a stronger academic identification than the group with the male-dominated leaderboard. Hanus and Fox (2015) found that students' intrinsic motivation decreased throughout the gamified course, which in turn led to lower final exam scores. Overall, when measuring learning outcomes as quiz or exam scores, the results of these studies suggest that gamification supports learners' performance on quizzes but does not support learners' performance on final exams.

We found three studies that measured learning outcomes in more than one measure (De Marcos et al., 2016; Dominguez et al., 2013; Frost et al., 2015). De-Marcos et al. (2016) found that all experimental conditions examined (educational game, gamification of the LMS, a social networking platform, and a gamified social platform) supported an increase in learners' assignment grades and final exam scores. The two experimental groups using a social networking platform and social gamification had better results on learning performance than the groups using an educational game and the gamified LMS. Dominguez et al. (2013) found the experimental group of students, who used the gamified LMS, scored better than the control group on learning activities (presentations, databases, spreadsheets) completed throughout the course, which were "concerned with practical application of concepts" (p. 386); however, the experimental group scored significantly lower than the control group on the final examination and on participation. From these results, Dominguez et al. (2013) argue that gamification can help "develop practical competences but somehow they also hinder the understanding of underlying theoretical concepts in contrast with traditional courseware" (p. 386). Frost et al. (2015) found no significant gain in learning outcomes in homework assignments, labs, or on the final exam. Overall, when measuring learning outcomes as performance in more than one aspect, the results of the studies are mixed and suggest that more research is needed when measuring learning outcomes via more than one activity.

We found two studies that measured learning outcomes differently than any other study via whole course grades or points accrued from completing activities within the gamified context (O'Donovan et al., 2013; Hakulien et al., 2013). O'Donovan et al. (2013) compared the course grades of students from a non-gamified course to the course grades of students in a gamified course and found student performance improved significantly in the gamified course. Hakulien et al. (2013) found that students in the gamified class earned more points on the online activities than the students in the control group. When measuring learning outcomes as whole course grades or overall points, the results of these two studies suggest that gamification does support learning outcomes. Overall, we found a wide array of methods for measuring learning outcomes. The results of the studies suggest that gamification supports learning outcomes on smaller assignments, such as quizzes and practical activities, as well as whole course grades; however, these results also suggest that gamification does not support the transfer of knowledge when measuring final exam scores.     

Learner Motivation. Learner motivation was a pedagogical aspiration examined by 41% of the studies we found. We found a total of seven studies that discussed their examination of the effect gamification or a gamified learning activity had on the motivation of learners. Of the seven studies, three of these studies found that their gamified learning activity or course supported learner motivation, three studies found that the gamified target did not support learner motivation, and one study found mixed results between their gamified courses. Thus, the studies appeared to directly contradict each other in a perfect split.

Three of the seven studies found results that suggest that gamification supports learner motivation through intrinsic motivation, relatedness, and participation (Buckley & Doyle, 2016; Frost et al., 2015; Ténorio et al., 2016). Buckley and Doyle (2016) examined gamification's effect on students' intrinsic and extrinsic motivation. The authors found that gamification is effective for students who are intrinsically motivated, while the effect on those students who are extrinsically motivated appears to be limited to those students who are motivated through their identity and will complete activities in order to express that identity. Frost et al. (2015) found that the gamified context improved the relatedness, which is a part of the broad framework that Frost et al. use to define motivation. When interpreting participation as a form of learner motivation, Ténorio et al. (2016) examined participation in a peer assessment model gamified via leaderboards, badges, goals and points. As a whole, Ténorio et al. (2016) found the participation of the students in the gamified peer assessment model increased compared to the non-gamified model. Overall, these studies suggest that gamification supports intrinsic motivation, relatedness, and motivation as defined by student participation.

In their first course, an e-learning course for university teachers, Olson et al. (2015) interpreted their results as the students finding the gamification model to be useful and motivating because of the progress bar. Olson et al. (2015) also found that in their second course, the game-based learning course for undergraduates, the students seemed to be indifferent and therefore not motivated regardless of the gamified elements. 

Three of the studies that examined learner motivation either found no change or found that gamification harmed learner motivation (Dominguez et al., 2013; Hakulien et al., 2013; Hanus & Fox, 2015. Hanus and Fox (2015) found negative results when examining the effect of gamification on student motivation. The authors found that the students in the gamified course showed less motivation than the students in the non-gamified control group. Specifically, Hanus and Fox (2015) found that the intrinsic motivation of the students in the gamified course actually decreased over time as compared to the students in the non-gamified course. Dominguez et al. (2013) examined student participation as a measure of motivation and found the experimental group using the gamified LMS had lower participation scores. Participation was assessed by the interactions on the LMS, class attendance, and contributions to different learning activities. Hakulien et al. (2013) interpreted motivation as student behavior and examined the use of achievement badges in an online learning environment. As mentioned earlier in this paper, while the authors found that the majority of students in both the experimental and the control groups behaved similarly, only a small group of students' behavior significantly changed because of the achievement badges. Overall, the results of these three studies suggest that gamification harms motivation when looking at motivation as intrinsic motivation over time, as student participation, and as student behavior. 

Overall, the results of these studies that examined learner motivation were a mix, suggesting that gamification can both support and harm motivation. The variety of mixed results and the variety of the measures of motivation suggest that more research is needed to determine if gamification supports learner motivation.

Learner Perceptions. Learner perceptions include learner satisfaction, enjoyment, and overall views on the use of gamification. We found five studies that examined the effect of gamification on learner satisfaction and enjoyment. Four of the studies we located found gamification to have a positive effect on learner satisfaction and enjoyment of the gamified learning activity, two of the studies found that gamification supported learners' perceptions of their own motivation and learning, and one study found gamification to have a negative effect on learner satisfaction. 

Four of the studies we examined found that students enjoyed the gamification. Cheong et al. (2013) found a majority (67%) of the students stated that they were not unhappy when playing the gamified quiz, and almost half (46%) of the students stated they were happy when completing the quiz. Cheong et al. (2013) also found a majority of students to not feel worried, exhausted, or miserable while completing the quiz. Overall, the participants in Cheong et al.'s (2013) study found the gamified quiz enjoyable. Dominguez et al. (2013) asked participants to take an attitudinal survey in order to determine their satisfaction level with the gamified LMS. The results of this survey show that the students in this experiment had an overall positive experience with the gamified LMS. Fleishman and Ariel (2016) also examined learner perceptions of their gamified web-based tool and found that the majority of the students reported the web-based tool to be useful or very useful (66%), enjoyed learning using the web-based tool (90%), and stated that they would definitely use or most likely use the gamified web-based tool for study purposes (63%). Frost et al. (2015) found an increase in student interest with the gamified web-based tool. Overall, the results of these studies suggest students enjoy gamification of different learning contexts.

Two studies examined learners' perceptions of the effect of gamification on their motivation and learning. In regard to learners' perceptions of their own motivation, Dominguez et al. (2013) found that the majority (62%) of the students found the traditional activities more motivating than the gamified portion of the course. Cheong et al. (2013) used a survey to determine the students' views on their learning outcomes after participating in the gamified learning activity. Overall, the students' responses were positive with a majority of students indicating that they believe the gamified activity "improved their performance, increased their learning productivity and effectiveness, and helped them achieve better grades" (Cheong et al., 2013, p. 11). Together, the results of these two studies suggest that gamification supports learners' perceptions of their motivation and learning in a gamified context.

In Geelan et al.'s (2015) qualitative case study, students reported both positive and negative aspects about the gamified learning tool. The positive features that emerged in the qualitative data were that the gamified learning tool focused students on the content, the content was highly relevant to the students' studies, the system provided timely informative feedback, the content was visually represented and stratified, there were variable activities with progressive difficulty, and the gamified learning tool supported multiple learning styles. The negative features that were reported were the lack of organizational support, the incompatibility of certain web browsers with the gamified learning tool, the sometimes poor interaction issues the gamified learning tool experienced, and the interface design issues (Geelan et al., 2015). The authors relate most of the negative features found to the attention that is needed when designing gamified learning tools. The results of this study suggest that overall students enjoyed the gamification of the learning tool but that more work needs to be done on how the gamification is implemented.

We did find one study that reported that learners were dissatisfied with a gamified course design gamification. Hanus and Fox (2015) found that student satisfaction with the gamified course decreased over time. When looking at all the studies that examined learners' perceptions in a gamified learning context, the results of the majority of the studies suggest that learners enjoy their gamified learning tools and that gamification supports their own perceptions of their motivation and learning.

Learner Engagement. Four of the studies we found examined the effect of gamification on learner engagement.  All four studies found that gamification supports learner engagement. Cheong et al. (2013) had students fill out a survey to evaluate student engagement, learning, and enjoyment. In regard to engagement, the majority (77%) of the students completed the gamified quiz, and, almost half (46%) of the students reported that they wanted to explore everything the gamified quiz had to offer. Rose et al. (2016) also found evidence to suggest that students in the group using the gamified quizzes had a higher level of engagement than the students in the control group. The students in the gamified group had significantly more attempts on each quiz than the control group. Rose et al. (2016) also found a significantly higher number of students who "demonstrated additional effort to achieve a perfect score, despite not being a course requirement" (p. 6). O'Donovan et al. (2013) found their gamified course had a higher lecture attendance at 79% than other computer science classes in the same department, usually between 30-60%. Lecture attendance was tied to points and rewards as part of the gamified course. Moccozet et al. (2013) found the use of points in their gamified collaborative learning platform encouraged users to contribute to their group work and was an effective way of assessing students' individual contributions within group work. Overall, the results of these studies suggest that gamification supports learner engagement.  

Playfulness: a positive attitude toward learning. Two studies looked at personality traits in relation to gamification features. Codish and Ravid (2014) examined the playfulness of different game mechanics as perceived by extraverts and introverts. Overall, the authors found the game mechanics of leaderboards and points work best for introverts while rewards work better for extraverts. The results also suggest that these game mechanics can have the opposite effect on the opposite personality, i.e. leaderboards and points can have a negative effect on extraverts and rewards can have a negative effect on introverts (Codish & Ravid, 2014). The second study was the examination of gaming personalities and how they relate to the use of game dynamics within a gamified online discussion. Overall, Tu et al. (2015) found that only motivational game dynamics can be predicted by gaming personality, as compared to the other four types of game dynamics examined. Tu et al. (2015) conclude that while their study of how gaming personality relates to game dynamics is "insufficient to design effective gamification" (p. 170), the examination of students' gaming personalities, preferences, and characteristics is important and should be considered when designing gamified learning contexts. Overall, the results of these studies suggest that different game-based mechanics can be perceived as playful or not by different personalities, and more research is needed to determine if gamification is affected by different gaming personalities. 

Gamification Targets

The activities and learning interventions that were gamified in the studies we found fell into five main categories: whole course curriculum, LMS, quizzes and tests, online learning objects, and collaborative approaches. The two most common gamified categories were LMSs and collaborative approaches (both at 29%, 5 of 17 studies). The least common gamified target category was whole course curriculum (12%, 2 of 17 studies). The others were online learning objects (26%, 4 of 17 studies) and quizzes and tests (18%, 3 of 17 studies).

Learning Management Systems. There were five studies that used a gamified context of a LMS as their gamified context. All these studies used a gamification plugin within their LMS in order to examine the effect of gamification. There were different LMSs used, including Moodle (Codish & Ravid, 2014; Olson et al., 2015), Blackboard (De-Marcos et al., 2016; Dominguez et al., 2013), and Integrated Site Management System (Frost et al., 2015). Gamified LMSs, or a gamified plugin for the LMS, was one of the most popular gamified contexts found in our review.

Collaborative Approaches. We found five studies that examined the gamification of online collaborative approaches. Along with gamifying a LMS, Codish and Ravid (2014) gamified a semester long project, where students collaborated to identify, analyze, and solve a business problem and then critique other groups' projects. De-Marcos et al. (2016) examined the gamification of a social networking platform that was used as a way for students to submit and review activities and receive feedback from their peers. Moccozet et al. (2013) used gamification as an assessment of online group work by using points to assess students' individual work on a group work project. Ténorio et al. (2016) gamified a peer assessment model where students are able to provide quantitative and qualitative feedback to their peers. Tu et al. (2015) examined the gamification of an online discussion forum. Overall, we found collaborative approaches to be one of the most popular gamified context, specifically when using gamification as a way to assess group work. 

Online Learning Objects. We found four studies that examined the gamification of online learning objects. Fleischmann and Ariel (2016) examined the gamification of an online learning tool to help students practice and understand the ELISA processes in biomedical science education. Geelan et al. (2015) gamified an interactive learning tool to help students in a fundamentals of bioscience class. Hakulien et al. (2013) gamified an online learning environment (TRAKLA2) in which students practice algorithm simulation exercises. Buckley and Doyle (2016) examined the gamification of a predictive market to help students understand the calculation of tax liabilities. Overall, the category of online learning objects was the second most examined category of the gamified targets.

Quizzes and Tests. We found three studies that gamified a quiz or an exam in their studies. Cheong et al. (2013) used a gamified online quiz tool in their study. Rose et al. (2016) gamified an existing list style quiz in order to provide students with immediate feedback. Christy and Fox (2014) gamified a math test by including leaderboards. Overall, single gamified quizzes or tests were not the most popular gamified target in the higher education context.

Whole Course. Of the 17 studies we reviewed, only two studied the effect of gamification of a whole course curriculum. Hanus and Fox (2015) and O'Donovan et al. (2013) both implemented a whole semester gamified curriculum into a course. The gamified course implemented by Hanus and Fox (2015) "required participation, badge completion, and engagement with an online leaderboard" (p. 155). In O'Donovan et al.'s (2017) gamified course, storyline, experience points, steam points, leaderboards, badges, and progress bars were incorporated into most learning activities and lecture attendance throughout the semester-long course. Overall, we only found two studies that examined gamification on the whole course curriculum.

Discussion

What are the most studied mechanics of gamification?

We found that the most studied game mechanic was badges (11/17 studies), and the second most studied was leaderboards (7/17 studies). In total, seven different game-based mechanics were studied in our higher education literature set. Overall, leaderboards were found to promote competitiveness while points, rewards, and immediate feedback were all found to support learner motivation. Points and immediate feedback were also found to support learner engagement. Progress bars, points, and badges were perceived by learners to be playful, and the storyline was found to support student satisfaction.

Most importantly, 82% of these articles examined more than one game-based mechanic, so clearly the synergy between mechanics is not overlooked by researchers investigating the tactics. So, while we can walk away from this study knowing leaderboards and badges are the most prominently studied, in the majority of cases, those who study these tactics would likely caution that an interpretation of their findings regarding badges and leaderboards must take into account the other tactics employed with them.   

This analysis suggests that a strategic selection of affordances is possible. For educators and instructional designers, when deciding which game mechanics to implement or design within a learning tool or context, pedagogical aspirations can be aligned with design choices. This review supports instructional designers who aim to improve learner motivation, suggesting that points, rewards, and immediate feedback have been fruitful design features that accomplish this in previous gamified contexts. If an intended characteristic of the instructional design is learner engagement, the research supports using points or immediate feedback. If an instructional design can accommodate competitive learner participation, our analysis would support including leaderboards, keeping in mind the caveat that the studies we examined found that leaderboards were not perceived as playful. Lastly, we would argue that there is empirical support for progress bars, points, and badges supporting playfulness and the storyline supporting satisfaction, so design decisions in that direction would also align with this analysis.

What discrete pedagogical aspirations are being examined with the use of gamification?

In the studies that met our criteria, we found five pedagogical aspirations being examined, which, in order of frequency from highest to lowest, were: learning outcomes, learner motivation, learner perceptions, learner engagement, and playfulness as a positive attitude towards learning. The most popular pedagogical aspiration examined with gamification was learning outcomes, and the least popular was playfulness as it related to learner personalities. The literature suggests that gamification supports learner engagement, learner enjoyment and satisfaction, and to a lesser extent, learning outcomes.

With the availability of data in online learning environments, we did not find the high frequency of learning outcomes as a pedagogical aspiration to be surprising. The nascent nature of the topic and the convenience of the data to address it may have played a role in these results. That our review also showed a wide array of meanings when discussing the pedagogical aspirations also suggested this area of study may also be in transition to a more consistent form. This variety in meanings, and in how researchers interpreted the relationship between gamified interventions and pedagogical aspirations, could have something to do with the mixed results that were found for several of the pedagogical aspirations, where studies seemed to directly contradict each other. De Marcos et al. (2016) and Frost et al. (2015) found opposite results when examining the effect of gamification on learning outcomes via multiple measures.

For educators and instructional designers, the results of our review suggest that learner engagement and some types of learning outcomes are two pedagogical aspirations that are best supported through gamification. Educators and instructional designers can use gamification to affect smaller learning outcomes, such as quizzes, tests, and projects; however, these results also suggest that gamification does not support learning performance on final exam scores. This suggests an important limitation to game mechanics in educational interventions, that is, transfer. If smaller (discrete point) outcomes correlate positively with gamified design choices, but larger, end of course measures do not, it may suggest that gamified tactics do not support transfer learning outside of the context of the instructional design.

Our analysis also suggests that gamification can support enjoyment and motivation if used sparingly, because the value of gamification may lie in its novelty as dissatisfaction with game mechanics may build over time (Hanus & Fox 2015).   

What learning activities or educational tools are being gamified?

In our literature review, we found five different categories of gamified educational tools, in this descending order of frequency: LMSs, collaborative approaches, online learning objects, quizzes and tests, and whole courses. The LMSs, online learning objects, and collaborative approaches were the most popular being gamified, which suggests a level of complexity in which gamification is being used.

When employed within collaborative designs, game mechanics show a strong potential to allow some of the more ambiguous tasks involved in peer assessment and group work to be assessed. For educators, the amount an individual student contributes to a group project and the amount of participation a student gives in a peer assessment activity can be very difficult to negotiate with students because the participation lacks direct evidence. However, the studies examined in our review used plugins within LMSs to create measures for this type of participation (Moccozet et al., 2013; Ténorio et al., 2016; Tu et al., 2015). We saw game mechanics within collaborative designs as having more potential than these frequencies might suggest as this is a persistent pedagogical issue.

Conclusion

The malalignment of the term gamification in keywords and article titles with the most popular definitions and meaning of the term (found in Dicheva et al. 2015; Kapp 2012), depicted to us a research area still in its infancy. Furthermore, the often sporadic use and study of gamified elements dispersed across varied pedagogical aspirations suggested that a holistic vision of what the implementation of these design choices was meant to afford was yet absent in the minds of these researchers. Rather, researchers and designers are still exploring what these game mechanics offer. This literature is meant to be descriptive, rather than prescriptive, showing what we have learned so far, even if these pedagogical tactics were clumsily employed by designers across contexts of varied applicability.

A consequence of exploration in design is the shallow use of gamification, i.e. the use of a few game mechanics without a larger game-based vision. Kapp (2012) calls the adding of game mechanics to a non-game context a "narrow approach … that does not lead to learning, engagement, or productivity improvements" (p. 15). Kapp (2012) advocated a holistic approach for designing gamified environments for learning. Along with using game mechanics, we must also use "aesthetics and game thinking to engage people, motivate action, promote learning, and solve problems" (p. 10).  

However, game thinking is not something we came across in any of the studies in this literature review. The ideas of learner engagement, motivation, and learning were mentioned, but the idea of game thinking, which is "the idea of thinking about an everyday experience like jogging or running and converting it into an activity that has elements of competition, cooperation, exploration, and storytelling" (Kapp, 2012, p. 11) was wholly absent. Thus, another area of research would be to examine the learners' choices and strategy in "winning the game" in a gamified learning environment to develop a more nuanced understanding of how these game mechanics actually do impact a learner's reasoning.

The term meaningful was often referred to but remained undefined and vague.  O'Donovan et al. (2013) noted that "practice extensions were so highly valued because they gave the student a real and meaningful reward" (p. 249). One study (DeMarcos et al., 2016) mentions in their discussion and conclusion section that meaningful gamification could be the determining factor of whether gamification supports student learning.

This literature review was conducted in the service of design. As a body of research, the literature says that points, rewards, and immediate feedback all support learner motivation. Points and immediate feedback were also found to support learner engagement. The literature also says that leaderboards support competitiveness, and the storyline was found to support student satisfaction. Areas of focus in the study of gamification gravitate to the pedagogical aspirations of learner motivation, learner engagement, learning outcomes, learner perceptions, and playfulness, while learner engagement and playfulness are less studied. The tools used for gamifying learning in the higher education context are LMSs, online learning objects, and collaborative approaches, and little else.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.