Impacts of Badges and Leaderboards on Academic Performance: A Meta-Analysis

다국어 초록 (Multilingual Abstract)

As technological changes continue to accelerate every day, meeting the needs of a shifting educational landscape requires leaving an exclusively “in-person” education behind. Gamified learning environments should be carefully designed in light of conflicting studies to suit students’ needs. The purpose of this meta-analysis is to draw conclusive results regarding the application of the most commonly used game elements in education, i.e., badges and leaderboards, through a comprehensive analysis of their impact on academic performance in online learning. Review Manager (RevMan 5.4) was used to analyze eligible studies selected from Emerald, SAGE, ERIC, EBSCO, and ProQuest between January 2011 and January 2022. Analyzing 37 studies found that using leaderboards and badges in online education enhanced academic performance when compared to traditional learning without gamification (SMD = 0.39). The badge-only intervention showed a larger effect size (SMD = 0.33) than the leaderboard-only intervention (SMD = 0.27). Badges and leaderboards together exhibited a larger effect size (SMD = 0.48) than individual game elements (SMD = 0.40). The impact of the game elements on academic performance was greater in the humanities (SMD = 0.51) than in STEM fields (SMD = 0.32) and was greater for K-12 students (SMD = 0.63) than for college students (SMD = 0.31). This study contributes to a timely discussion of the use of badges and leaderboards in COVID-19 online learning trends and provides relevant data for designing integrations of online education and gamification models.

참고문헌 (Reference)

1 Goudeau, S., "Why lockdown and distance learning during the COVID-19 pandemic are likely to increase the social class achievement gap" 5 (5): 1273-1281, 2021

2 Jensen, J. C., "Virtualizing cyber-physical systems:Bringing CPS to online education" 1-5, 2013

3 Schardt, C., "Utilization of the PICO framework to improve searching PubMed for clinical questions" 7 (7): 1-6, 2007

4 Goosby, C. L, "Using Digital Badges to Impact First Grade Math Achievement: A Quasi-experimental Study of Digital Badges and the North Carolina Early Numeracy Skills Indicators (Publication No. 28318393)" Northcentral University 2020

5 Mekler, E. D., "Towards understanding the effects of individual gamification elements on intrinsic motivation and performance" 71 : 525-534, 2017

6 Kyewski, E., "To gamify or not to gamify? An experimental field study of the influence of badges on motivation, activity, and performance in an online learning course" 118 (118): 25-37, 2018

7 Kivunja, C., "Theoretical perspectives of how digital natives learn" 3 (3): 94-109, 2014

8 Ahmad, A., "The impact of gamification on learning outcomes of computer science majors" 20 (20): 1-25, 2020

9 Huang, R., "The impact of gamification in educational settings on student learning outcomes : A meta-analysis" 68 (68): 1875-1901, 2020

10 Stetson-Tiligadas, S. M, "The impact of digital achievement badges on undergraduate learner motivation (Publication No. 10037460)" Capella University 2016

11 Sailer, M., "The gamification of learning : A meta-analysis" 32 (32): 77-112, 2020

12 Baydas, O., "The examination of the gamification process in undergraduate education: A scale development study" 28 (28): 269-285, 2019

13 Hakulinen, L., "The effect of gamification on students with different achievement goal orientations" 9-16, 2014

14 Legaki, N. Z., "The effect of challenge-based gamification on learning : An experiment in the context of statistics education" 144 : 102496-, 2020

15 Hakulinen, L., "The effect of achievement badges on students’ behavior : An empirical study in a university-level computer science course" 10 (10): 18-30, 2015

16 Andrade, F. R., "The bright and dark sides of gamification" 9864 : 176-186, 2016

17 Zainuddin, Z., "Students’ learning performance and perceived motivation in gamified flipped-class instruction" 126 : 75-88, 2018

18 Teasley, S. D., "Student facing dashboards: One size fits all?" 22 (22): 377-384, 2017

19 Cohen, J., "Statistical power analysis for the behavioral sciences" Taylor and Francis 2013

20 Von Stumm, S., "Socioeconomic status amplifies the achievement gap throughout compulsory education independent of intelligence" 60 : 57-62, 2017

21 Foon, H. K., "Shaping the future of education, communication and technology. Educational communications and technology yearbook" Springer 71-90, 2019

22 Crabtree, J., "Self-evaluation and social comparison amongst adolescents with learning difficulties" 11 (11): 347-359, 2001

23 McGonigal, J., "Reality is broken: Why games make us better and how they can change the world" Penguin Press 2011

24 Xu, J., "Psychological interventions of virtual gamification within academic intrinsic motivation : A systematic review" 293 : 444-465, 2021

25 Furdu, I., "Pros and cons gamification and gaming in classroom" 8 (8): 56-62, 2017

26 Maslow, A. H., "Motivation and Personality" Harper & Row Publishers 1954

27 Ghasia, M., "Micro-credentials in higher education institutions : An exploratory study of its place in Tanzania" 15 (15): 219-230, 2019

28 Higgins, J. P. T., "Measuring inconsistency in meta-analyses" 327 (327): 557-560, 2003

29 Mah, D. K., "Learning analytics and digital badges: Potential impact on student retention in higher education" 21 (21): 285-305, 2016

30 Christy, K. R., "Leaderboards in a virtual classroom : A test of stereotype threat and social comparison explanations for women’s math performance" 78 : 66-77, 2014

31 Kapsalis, G. D., "Kahoot! as a formative assessment tool in foreign language learning : A case study in greek as an L2" 10 (10): 1343-1350, 2020

32 Uanhoro, J., "Investigation of the effect of badges in the online homework system for undergraduate general physics course" 12 (12): 217-, 2022

33 Uz Bilgin, C., "Investigating the effectiveness of gamification on group cohesion, attitude, and academic achievement in collaborative learning environments" 64 (64): 124-136, 2019

34 Tahir, F., "Investigating the causal relationships between badges and learning outcomes in SQL-Tutor" 17 (17): 7-, 2022

35 Newby, T. J., "Instructional digital badges : Effective learning tools" 68 (68): 1053-1067, 2020

36 Auvinen, T., "Increasing students’ awareness of their behavior in online learning environments with visualizations and achievement badges" 8 (8): 261-273, 2015

37 Margolis, H., "Improving self-efficacy and motivation : What to do, what to say" 41 (41): 218-227, 2006

38 Lam, Y. W., "Improving argumentative writing : Effects of a blended learning approach and gamification" 22 (22): 97-118, 2018

39 Barata, G., "Identifying student types in a gamified learning experience" 4 (4): 19-36, 2014

40 Sailer, M., "How gamification motivates : An experimental study of the effects of specific game design elements on psychological need satisfaction" 69 : 371-380, 2017

41 Zhou, T., "Genome-wide analysis of human hotspot intersected genes highlights the roles of meiotic recombination in evolution and disease" 14 (14): 2013

42 Domínguez, A., "Gamifying learning experiences : Practical implications and outcomes" 63 : 380-392, 2013

43 Dichev, C., "Gamifying education : What is known, what is believed and what remains uncertain : A critical review" 14 (14): 9-, 2017

44 Chen, C. -C., "Gamify online courses with tools built into your learning management system(LMS)to enhance selfdetermined and active learning" 22 (22): 41-54, 2018

45 Landers, R. N., "Gamification of task performance with leaderboards : A goal setting experiment" 71 : 508-515, 2017

46 Sailer, M., "Gamification of inclass activities in flipped classroom lectures" 52 (52): 75-90, 2021

47 Mazarakis, A., "Gamification is working, but which one exactly? Results from an experiment with four game design elements" 1-16, 2022

48 Seaborn, K., "Gamification in theory and action : A survey" 74 : 14-31, 2015

49 Poondej, C., "Gamification in e-learning : A Moodle implementation and its effect on student engagement and performance" 17 (17): 56-66, 2019

50 Strmečki, D., "Gamification in E-learning : Introducing gamified design elements into E-learning Systems" 11 (11): 1108-1117, 2015

51 Özdener, N, "Gamification for enhancing Web 2.0 based educational activities: The case of pre-service grade school teachers using educational Wiki pages" 35 (35): 564-578, 2018

52 Zichermann, G., "Gamification by design: Implementing game mechanics in web and mobile apps" O'Reilly Media, Inc 2011

53 Nieto-Escamez, F. A., "Gamification as online teaching strategy during COVID-19 : A mini-review" 12 : 648552-, 2021

54 Chans, G. M., "Gamification as a strategy to increase motivation and engagement in higher education chemistry students" 10 (10): 132-, 2021

55 Saleem, A. N., "Gamification applications in Elearning:A literature review" 27 : 1-21, 2021

56 Turan, Z., "Gamification and education : Achievements, cognitive loads, and views of students" 11 (11): 2016

57 Werbach, K., "For the win: How game thinking can revolutionize your business" Wharton School Press 2012

58 Zimmerling, E., "Exploring the influence of common game elements on ideation output and motivation" 94 : 302-312, 2019

59 Jagušt, T., "Examining competitive, collaborative and adaptive gamification in young learners’ math learning" 125 : 444-457, 2018

60 Tsay, C. H. -H., "Enhancing student learning experience with technology-mediated gamification : An empirical study" 121 : 1-17, 2018

61 Çakıroğlu, Ü, "Enhancing statistical literacy skills through real life activities enriched with gamification elements : An experimental study" 18 (18): 441-459, 2021

62 Heinert, S. W., "Engaging youth through digital badges to promote health in underserved communities" 22 (22): 631-637, 2021

63 Sarkar, N., "Engaging digital natives through social learning" 15 (15): 1-4, 2017

64 Denny, P., "Empirical support for a causal relationship between gamification and learning outcomes" 1-13, 2018

65 Kim, J., "Effects of gamification on behavioral change in education : A meta-analysis" 18 (18): 3550-, 2021

66 Ongoro, C. A., "Effects of digital games on enhancing language learning in Tanzanian preschools" 11 (11): 325-344, 2019

67 Gündüz, A. Y., "Effectiveness of gamification in flipped learning" 10 (10): 215824402097983-, 2020

68 Ugur-Erdogmus, F., "Effect of gamified mobile applications and the role of player types on the achievement of students" 60 (60): 1063-1080, 2022

69 Kim, J. T., "Dynamical model for gamification of learning(DMGL)" 74 (74): 8483-8493, 2015

70 Looyestyn, J., "Does gamification increase engagement with online programs? A systematic review" 12 (12): e0173403-, 2017

71 Huang, B., "Do points, badges and leaderboard increase learning and activity: A quasi-experiment on the effects of gamification" 275-280, 2015

72 Hamari, J., "Do badges increase user activity? A field experiment on the effects of gamification" 71 : 469-478, 2017

73 Silpasuwanchai, C., "Developing a comprehensive engagement framework of gamification for reflective learning" 459-472, 2016

74 Fanfarelli, J. R., "Designing effective digital badges: Applications for learning" Routledge 2019

75 Balci, S., "Comparing the effectiveness of badges and leaderboards on academic performance and motivation of students in fully versus partially gamified online physics classes" 27 (27): 8669-8704, 2022

76 Homer, R., "Comparing digital badges-and-points with classroom token systems : Effects on elementary school ESL students’classroom behavior and English learning" 21 (21): 137-151, 2018

77 Morris, B. J., "Comparing badges and learning goals in low-and high-stakes learning contexts" 31 (31): 573-603, 2019

78 Greitzer, F. L., "Cognitive science implications for enhancing training effectiveness in a serious gaming context" 7 (7): 2-, 2007

79 Manzano-León, A., "Between level up and game over : A systematic literature review of gamification in education" 13 (13): 2247-, 2021

80 Hanus, M. D., "Assessing the effects of gamification in the classroom : A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance" 80 : 152-161, 2015

81 Ding, L., "Applying gamifications to asynchronous online discussions: A mixed methods study" 91 : 1-11, 2019

82 Kusuma, G. P., "Analysis of gamification models in education using MDA framework" 135 : 385-392, 2018

83 Watson-Huggins, J, "An experimental study on the effects of a gamified software intervention in mathematics achievement among sixth grade students (Publication No. 13820081)" Nova Southeastern University 2018

84 Pursell, D. P., "Adapting to student learning styles : Engaging students with cell phone technology in organic chemistry instruction" 86 (86): 1219-, 2009

85 Festinger, L., "A theory of social comparison processes" 7 (7): 117-140, 1954

86 Bernik, A., "A pilot study of the influence of gamification on the effectiveness of an e-Learning course" 73-79, 2015

87 Ritzhaupt, A. D., "A meta-analysis on the influence of gamification in formal educational settings on affective and behavioral outcomes" 69 (69): 2493-2522, 2021

88 김종우 ; 김상욱, "A Comparative Study on the Differences in Cultural Attributes of Gamification between Korea and Austria" 한국산업정보학회 23 (23): 109-122, 2018