Testing plays a crucial role in software development, and it is essential for software engineering students to receive proper testing education. However, motivating students to write tests and use automated testing during software development can be challenging. To address this issue and enhance student engagement in testing when they write code, we propose to incentivize students to test more by gamifying continuous integration. For this we use Gamekins, a tool that is seamlessly integrated into the Jenkins continuous integration platform and uses game elements based on commits to the source code repository: Developers can earn points by completing test challenges and quests generated by Gamekins, compete with other developers or teams on a leaderboard, and receive achievements for their test-related accomplishments. In this paper, we present our integration of Gamekins into an undergraduate-level course on software testing. We observe a correlation between how students test their code and their use of Gamekins, as well as a significant improvement in the accuracy of their results compared to a previous iteration of the course without gamification. As a further indicator of how this approach improves testing behavior, the students reported enjoyment in writing tests with Gamekins.

\ CCS CONCEPTS

\ • Software and its engineering → Software testing and debugging; • Social and professional topics → Software engineering education

\ KEYWORDS

\ Software Testing, Gamification, Continuous Integration, Education

\ ACM Reference Format:

\ Philipp Straubinger and Gordon Fraser. 2024. Gamifying a Software Testing Course with Continuous Integration. In 46th International Conference on Software Engineering: : Software Engineering Education and Training (ICSESEET ’24), April 14–20, 2024, Lisbon, Portugal. ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3639474.3640054

1 INTRODUCTION

Software testing is a well-established concept and is extensively used in the industry [13]. However, despite the availability of various tools that facilitate the testing process, such as testing support in integrated development environments (IDE) [26] and automated test execution in continuous integration (CI) platforms [8], testing is often overlooked in practice. Speculated reasons for this include developers’ lack of motivation to engage in testing activities and a lack of education in software testing [14, 30, 41, 49, 50]. Although the often inadequate treatment of testing in higher education [41] has recently been countered with a growing recognition of the importance of testing [9, 29, 35], this is nevertheless hampered by learners just like developers tending to perceive testing as tedious and boring [32]. As a result, the industry suffers significant losses due to inadequate software quality and insufficient testing [31].

\ Gamification techniques provide an opportunity to motivate students to test despite their perception of it being boring or tedious. Gamification involves incorporating elements commonly found in games, such as leaderboards and achievements, into nongame contexts [15]. It has been demonstrated that this approach helps engage students in software engineering education [24], for example by gamifying lectures or tools to teach software testing [7, 12, 16, 22, 36]. However, in order for students to internalize testing and establish it as an integral part of their development approach, we argue that seamless integration of gamification of testing into the software development process is necessary.

\ In order to achieve this, we integrate gamification into continuous integration (CI) systems used by students. Specifically, we use Gamekins [47], a plugin designed for the widely used CI platform Jenkins[1], which seamlessly integrates gamification into the software development workflow. By analyzing source code and test results, Gamekins identifies areas where testing can be improved based on code coverage [53] or mutation analysis [28] and then incentivizes improvement using gamification concepts such as challenges and quests, rewarding good testing with points, achievements, and leaderboard rankings. We evaluated the benefits of integrating Gamekins in an undergraduate software testing course, investigating its impact on the course’ learning objectives and student behavior. In detail, the contributions of this paper are as follows:

\ • We propose the use of gamification as a means to incentivize and reward testing activities in CI.

\ • We introduceGamekins as a tool to be used in an undergraduate-level software testing course.

\ • We empirically evaluate the effects of integrating Gamekins into the software testing course, comparing to a previous cohort without gamification and surveying the students.

\ The results of the study demonstrate a correlation between students’ testing behavior and the use of Gamekins, as well as a significant improvement in correct results compared to the previous edition of the course, where no gamification was used on the same assignments. While not all aspects of Gamekins were universally liked by students, overall, they enjoyed its use throughout the course, confirming that Gamekins is a viable teaching tool.

:::info This paper is available on arxiv under CC BY-SA 4.0 DEED license.

:::

[1] https://www.jenkins.io/

Feed: Hacker Noon - Medium

View: Original article

Tags: social testing

Media