Wu’s Castle: Teaching Arrays and Loops in a Game |
|
Michael Eagle |
Advisor: Tiffany Barnes |
AbstractWe are developing games to teach introductory computer science concepts to increase student motivation and engagement in learning to program. Wu’s Castle is a two-dimensional role-playing game that teaches loops and arrays in an interactive, visual way. In this game, the player interactively programs magical creatures to create armies of snowmen. The game provides immediate feedback and helps students visualize the execution of their code in a safe environment. We tested the game in a CS1 course, where students could earn extra credit to play Wu’s Castle. Our results show learning gains for game players, compared both through pre- and post-tests differences and improved performance on relevant final exam questions when compared to students who did not play the game. The results of this study suggest that Wu’s Castle implements good practices for teaching programming within a game. Categories and Subject Descriptors K.3.2 [Computers and
Education]: Computer and
Information Science Education - computer
science education. General
Terms Keywords 1. Problem and MotivationEnrollments in computing have been declining at alarming rates since 2000, and the proportion of underrepresented groups is also decreasing [1]. Attrition rates in computing are as high as 30-40%, with most students leaving after taking either CS1 or CS2 [2]. Increasingly, researchers and educators are calling for improvements in computing education and have turned to games in order to motivate students through assignments, curricula, and undergraduate research [3-11]. On the other hand, the study of games has lacked a coherent research paradigm [12]. Although studies have defined the qualities of games that might also be leveraged to improve education (such as challenge, fantasy, and curiosity [12]), and researchers are beginning to investigate the importance of the |
social context and communities built around games in informal learning [13, 14], there is still a need for controlled studies that might reveal the relative importance of game characteristics in motivation and learning. In other words, we are interested in investigating whether simple games that teach computing might demonstrably motivate students to engage in learning outside of the classroom. 1.1 Background and Related work Dr. Tiffany Barnes has
established the Game2Learn research
project, which engages advanced undergraduate computing students in
building
small games that teach individual CS1 concepts, and testing these games
in
introductory computing classrooms. This project has been successful in
supporting undergraduate retention and recruiting into computing
graduate
programs [15], while also revealing some of the most important aspects
of
educational games through our user studies [16].
We target our games to novice
computing students who are
also novice game players. According to a survey of computing educators,
loops
and arrays are two of the top three programming topics reported to be
difficult
for novice students [17]. Testing
and
debugging were two items often mentioned on this survey as well [17].
Compiler
errors can be confusing for beginning students, and code that compiles
but
produces incorrect results can be even more bewildering. Beauboeuf
cites both
poorly planned labs where teachers debug student code, and lack of
practice
with meaningful feedback as causes for attrition in CS [2]. Both of
these examples
beg for educational systems that assist students in anticipating and
finding
errors, and providing students individualized feedback. We present Wu’s
Castle,
a game developed as a Game2Learn capstone project [16] at the
University of
North Carolina at Charlotte. Wu’s
Castle
was developed to teach loops and arrays in an interactive, visual way.
In Wu’s Castle the
player interactively
constructs C++ code to solve in-game problems.
The player is able to watch how a program steps
through their code and identify
logic errors. We tested the game in a CS1 course where students could
earn
extra credit for playing Wu’s Castle
and taking a pre- and post-test. Questions similar to the pre- and
post-test
were placed on course’s final exam. Students who played the
game achieved pre-
to post-test learning gains and also outperformed other students in the
class
on the array section of the final exam. Our results suggest that Wu’s Castle implements good
practices
for teaching programming within a game. |
2. Uniqueness of the ApproachWu’s Castle was developed in RPG Maker XP, a 2-dimensional role-playing game development program that allows developers to quickly build games through drag-and-drop map and event editing and Ruby script programming. The game was developed using a rapid-prototyping and testing iteration cycle as described in [15].Wu’s Castle contains two main types of interaction: one for manipulating arrays by changing For Loop parameters, the other for physically walking the game character through the execution of nested loops. In the game, the player is first introduced to the story and the game interface. In Level 1, the player explores one-dimensional array manipulation using for loops. In Level 2, the player walks his or her character through a nested loop, and the player is asked multiple-choice questions about the loop’s code; such as, “Which variable controls the outer loop?” In Level 3, the player uses nested for loops to modify two-dimensional arrays. 2.1 Array manipulation (Levels 1 & 3)In Levels 1 and 3 the player uses a magical creature called a machina to create armies of snowmen and escape a mirror universe. To build the snowmen, the player must manipulate an array of values that describe the type of snowman at each position. The array is represented in the game as a line or grid of snowman characters. In each position, there might be no snowman, a regular snowman, a snowman with arms, a snowman with a hat, or a knocked-over snowman. Wu’s Castle provides the structure of a For Loop in C++ and allows the player to change the initial condition, stopping condition, and step size. The player then chooses the body of the loop and watches the loop execute. The player’s code is a set of instructions telling the machina which array positions to visit (through loop parameters) and what snowman to place in each position (through the loop body). As she executes her instructions, the player watches how she changes the array. Her travel through the array visualizes how the parameters of the For Loop control her movement, and what positions in the array are visited. Her animation placing a certain type of snowman at each position demonstrates the body of the For Loop. Figure 1 shows the for loop as the player has set its parameters: for(i=0; i<=19; i=i+1) array[i] = Snowman. The player selected the start, end, and increment values, and the machina is halfway through creating a row of 20 snowmen. Figure 2 shows the player choosing the body of a nested loop to create 40 snowmen.
Figure 1: The machina executing code in Level 1, showing half of the array updated during Mission 1 |
In the underlying code, the student’s constructed For Loop is run to modify an array that controls the snowmen drawn at each position. This dynamic implementation allows the player to visualize the actual effects of changing loop parameters. We carefully adjusted the timing of the animation so the machina would not travel too quickly; making the visualization more clear.
Figure 2: Selecting the body of a nested loop in Level 3 Table 1 shows Level 1 missions. The player must complete each mission in order to move on to the next, more challenging mission. Level 3’s missions follow a similar progression through filling a whole 2-dimensional array, certain rows, certain columns, and finally combinations of subsections, rows and columns. This progression was designed to demonstrate typical for loops and illustrate how changes in the parameters affect the loops. Table 1: Level 1 missions showing loop parameters
2.2 Nested Loop Walk-through (Level 2)This part of the game offers the player a worked example. The player is shown a typical C++ program that contains a nested loop with a cout statement. The player can view the code at anytime by pressing a button and is sometimes asked questions about the sample code; such as “which variable represents the outer loop?” As shown in Figure 3, the map contains two circular paths, representing the two loops in the program. The code statements are represented as chests (1-7) on the path. When the player touches a chest, a part of the code will be executed. The values of the loop control variables are constantly displayed on the screen to show how the variables change during each step of the program. When the player walks around the large (outer) loop, they are directed into the smaller loop where they must circle several times before they are able to escape back to the outer loop. |
|
The student walks through the nested loop execution, starting at the top of the diagram. The player progresses downward and enters the outer loop. The game indicates which line of code is being executed each time the player touches a chest. When the player enters the inner loop the path out is blocked until the player completes the inner loop the correct number of times. The labels in Figure 3 show program statements as described below: 1. Initializes the variables 2. Prints “BEGIN” 3. Enters the outer loop 4. Prints the outer loops variable 5. Enters the inner loop 6. Prints the inner loops variable 7. Prints “END”
Figure 3: Nested Loop execution walk-through This level of the game was designed to help students visualize and interactively walk through the structure and execution of nested for loops. Students begin to realize how inner loops start and how many times they iterate before closing, because their character is not allowed to exit the inner loop until it is complete. 2.3 General Game FeaturesIn our previous studies, we found that prompt, obvious feedback positively affected student attitudes and performance in learning games [16]. Due to the importance of feedback, in Wu’s Castle, the player receives feedback after each attempt at an answer. When the solution is correct a large green circle is displayed accompanied by a positive sound. If the student’s code fails to compile the machina falls unconscious, this represents a syntax error. and the student must try again. An incorrect solution results in a large red X and a negative sound, representing a logic error. If the code goes out of the array’s bounds, the machina explodes, representing a program crash. After any failure, the array and machina are reset. Should the player complete a mission in one or two tries a bonus animation, and sound are played. The obvious feedback helps novice game players understand their progress. |
The player’s score is displayed as an “experience” meter in the bottom left of the screen. The player is awarded with experience points for correct answers and completed missions. Bonus points are awarded when a mission is completed in one or two tries. Incorrect answers can sometimes result in a subtraction of points; the amount varies depending on the severity of the incorrect response. When the bar becomes full the player’s rank is increased, and an animation and sound is played. This rank is used to motivate the players to perform well. Wu’s Castle records a time-stamped log for each game session. At the start of a game session, the player is asked to enter a user ID. The log file is placed into the game directory when the game exits. The log file is lightly encrypted to prevent tampering. 3. ARRAY LEARNING STUDYTwenty-eight students in a CS1 class were offered extra credit for playing Wu’s Castle. The students were asked to sign an informed consent form, take a pre-test, play the game, and then take a post-test and qualitative survey. The course’s final exam was given later, including questions listed in Table 2, which were similar to those on the pre- and post-tests. Table 2: Array questions; the questions on the pre- and post- tests were similar, with some numbers changed.
|
|
The pre- and post-tests and
qualitative survey were
administered through an online surveying tool (www.surveymonkey.com). The game was made
available online, and
students were able to download and play it at home. The students first
took the
pre-test and then played the game. After playing the game, the students
were
asked to email the log files to the first author, and take the online
post-test
and survey. There were no set requirements on how long to play the
game, but we
expected the game to take about 20-30 minutes. 4. Results and ContributationsSince the study was conducted as extra credit at the end of the semester, the level of participation varied across the class. Of 27 students in the class, 11 did not play Wu’s Castle, or take any pre- or post-tests; we call this group Control-NoPretest (cNP). Seven students took the Array pre-test but did not play Wu’s Castle; this group is Control-Pretest (cP). Nine students completed the pre-test, played Wu’s Castle and turned in a log file, and completed the post-survey; this is group Game. Table 3—7 lists the pretest, posttest, and final exam scores for each group. Each question was worth one point. Questions with multiple part-answers allowed students to earn partial credit for each correct part. 4.1 Pretest to Posttest Learning GainsTable 3 shows the average scores on each question and overall for group Game on the pre- and post-tests and corresponding final exam questions. For the Game group, the average scores on the posttest were statistically significantly higher than the pretest (2.70 to 1.65) (p=.003). Comparing the difference between an individual student’s posttest and pretest score reveals an average 17% grade improvement. Overall, 8 of the students in group Game improved their scores, while 1 student showed a slight decrease in scores between the pre- and post-tests. Students (N=4) who made it to the second level of the game performed significantly better on question 5 (p=.03) on the posttest. The variance decreased from 1.73 to 0.49 between the pre and posttest. The average posttest scores compared with the similar questions on the final exam were slightly higher, indicating that the students retained knowledge. Table 3: Results for group Game (N=9)
4.2 Pre-test to Final Exam ComparisonTable 4 lists the pretest performance of the groups Game and cP. Overall, the average pretest scores for these two groups were not statistically significant (p=.35). For each student, we computed the difference between the final exam and pre-test score for each question, and the average of these differences over all students is shown in Table 5. We note that while group Game performed significantly better on the final exam array questions than on the pre-test, the difference in final and pre-test scores was not significant for group cP (p= .34). This shows that exposure to the pre-test questions did not have a significant effect on performance on the corresponding questions on the final exam for group cP. |
Table 4: Pretest scores for groups Game (N=9) and cP (N=7)
Table 5: Average of (final-pretest) score differences
Table 6 shows the performance on the final exam for the questions corresponding to the pre- and post-tests for each group. There was no significant difference between the control groups on the final exam questions (p=.37). Together with the pre-test to final exam comparison for group cP, these results provide evidence that there was no significant effect from multiple exposures to similar questions. Table 6: Final exam comparison between control groups
4.3 Final Exam PerformanceTable 7 shows the percent correct for the array portion, the non-array portion, and the overall final exam for all three groups. The array portion of the exam consisted of the 5 pre- and posttest questions plus 2 additional questions. Although all scores were higher for group Game, none of the score differences are significantly different, except for those on the array portion. Group cP’s scores were also higher than those for group cNP, though this difference is not significant. Performance on the array section of the final exam was almost equal between cP and cNP; however, cP scored about 5 points higher on the remaining portion of the final. The self-selected students (group Game and cP) performed 11% higher on the non-array section of the final exam; while there was no statically significant difference found, the self-selected students may have been slightly better performers. Table 7: Final exam performance by section of the final exam
When comparing Game and Control (cP+cNP), the average score for the array portion of the final exam was 51% higher for students who played Wu’s Castle (p=.01), suggesting that Wu’s Castle improved student learning about arrays. |
4.4 Qualitative SurveyThe qualitative survey consisted of eleven Likert-scale items as listed in Table 8. The scale was a 5-point scale from Strongly Disagree to Strongly Agree. There were also three open response questions that asked the student what part of the game they liked the best, and for suggestions or comments. Table 6 lists the number of students who selected “Agree” or “Strongly Agree” with the listed survey items. The majority of group Game enjoyed playing Wu’s Castle and would like to play more games like it. The students also felt as though the game was serious enough for homework in a CS course. Students generally found the game to start out at a doable level, but did not find it too easy. Most did not guess answers in the game; but were motivated to get the correct answers.
When asked what students liked best in the game, several students said they liked the simple interface. The most common response was that after you made a mistake, you were given the chance to try again with one player saying “It gave me a chance to come up with the answer after I kept getting it wrong and even after I had gotten it right...I could still practice more.” In the recommendations for improvement, almost all of the students desired a better hint system. The game allows players to re-try a mission as many times as desired, but offers little in the way of helping the player, should they become stuck. |
5. CONCLUSION AND FUTURE WORKOur results showed that students who played Wu’s Castle achieved statistically significant learning gains when compared to students who did not play the game. Although the sample size was small, the differences in scores were large. These results suggest that Wu’s Castle implements good practices for teaching programming within a game; these practices include frequent and obvious feedback and interactive visualization of code. On average, students played the game for about 40 minutes, demonstrating that students appreciate many chances to try the same problems. The results of this study should encourage instructors to consider implementing these technologies in classrooms. Future games can be developed with principles similar to Wu’s Castle for other concepts in computer science; and give players a place to practice programming and easily identify mistakes in a safe environment. We plan to continue creating Game2Learn games and running studies to track learning and retention rates in introductory computer science courses. Wu’s Castle was Game2Learn’s first attempt at online delivery. Some of the players reported difficulty installing the game. Emailing the log files is troublesome; and we only know if students are playing when they email us a log file. Future work should include an easier game installer, more concise install instructions, and a way for the logs to be automatically sent. Parsing the games log files for data was time consuming and many times had to be performed manually. A XML based log system has since been developed for future versions of the game. Future work should also include more specialized feedback when the players repeatedly fail at the same task. |
6. References[1] Zweben, S. 2003-2004 Taulbee Survey. Computing Research Association Taulbee Survey, May 2005. [2] Beauboeuf, T & J. Mason. Why the high attrition rate for computer science students: some thoughts and observations. SIGCSE Bull. 37, 2 (Jun. 2005), 103-106. [3] Bayliss, J. The Effects of Games in CS1-3, Microsoft Academic Days Conference on Game Development in Computer Science Education, Feb. 2007, 59-63. [4] Bayliss, J. & S. Strout. Games as a "flavor" of CS1. In SIGCSE2006. ACM Press, New York, NY, 500-504. [5] Becker, K. Teaching with games: The Minesweeper and Asteroids experience. The Journal of Computing in Small Colleges Vol. 17, No. 2, 2001, 22-32. [6] Garris, Ahlers, & Driskell. Games, motivation, and learning: a research and practice model. Simulation & Gaming, Vol. 33, No. 4, 2002, 441-467. [7] Gee, J. P. What video games have to teach us about learning and literacy. Comput. Entertain. 1, 1 (Oct. 2003), 20. [8] Gumhold, M. & Weber, M. Motivating CS students with game programming. Proc. Intl. Conf. on New Educational Environments (ICNEE), Neuchatel, Switzerland, Sep. 2004. |
[9] Parberry, I., Roden, T., & Kazemzadeh, M. Experience with an industry-driven capstone course on game programming: extended abstract. SIGCSE 2005: p91-95. [10] Prensky, M. Digital Game-Based Learning, New York, McGraw-Hill, 2001. [11] Wolz, U., T. Barnes, I. Parberry, and M. Wick. Digital gaming as a vehicle for learning. SIGCSE 2006: p. 394-395. [12] Squire, K. (2003). Video games in education. International Journal of Intelligent Simulations and Gaming, vol. 2, 49-62. [13] Hunicke, R., Robison, A., Squire, K., and Steinkuehler, C. Games, learning and literacy. Sandbox 2006. ACM Press, New York, NY, 19-19. [14] Steinkuehler, C. Learning in massively multiplayer online games. In Proc. Intl. Conf. Learning Sciences, Santa Monica, CA, June 22 - 26, 2004, p. 521-528. [15] Barnes, T. Powell, E. Chaffin, A. Godwin, A. Game2Learn: Building CS1 Learning Games for Retention. ITiCSE'07. ACM Press, New York, NY, 500-504. [16] Barnes, T., E. Powell, A. Chaffin, H. Lipford. Game2Learn: Improving the engagement and motivation of CS1 students. To appear in ACM GDCSE’08. [17] Dale, N. B. 2006. Most difficult topics in CS1: results of an online survey of educators. SIGCSE Bull. 38, 2 (Jun. 2006), 49-53. |