MY ROLE
User Testing
TEAM
Helen Wauck
Po-Tsung Chiu
Ziang Xiao
DURATION
Aug 2016 - Dec 2017
TARGET POPULATION
Who is this game designed for?
Although people of all ages can benefit from spatial skill training, we are currently targeting children ages 7-11 for this project. This is the age range where gender differences in spatial skills in favor of boys begin to emerge. We hope to intervene before this gender gap becomes a career barrier for girls interested in pursuing STEM and other disciplines where spatial skills are essential.
GAME FEATURES
Mouse Version
Homeworld Bound is a fully playable, immersive first person adventure and construction game designed with three guiding principles: Evidence-Based Content, Engagement, and Modularity. Incorporating Evidence-Based Content inspired by the commercial games empirically shown to be effective at training spatial skills allows us to focus on game features that are most likely to train spatial skills rather than groping in the dark. Thoroughly playtesting our game to make sure that it's fun and promotes player Engagement helps us avoid the "chocolate-covered broccoli" style of learning used in many educational games, which may contribute to their ineffectiveness at far-transfer cognitive skill training. Modularity enables us to break our game down into separate, independent units of varying granularity so we can test and isolate specific game features for spatial skill training.
GAME FEATURES
Gesture Version
Our gesture design focused on two specific operations for moving and rotating parts in the game. We refer to these as Movement and Rotation gestures.
Left: the Easy level. Right: the Sledgehammer level. The 2D images on the upper left corners show what the 3D objects would look like when all parts are attached correctly. Rotate and Move modes indicates the spatial operation the player can perform at the moment.
USER TESTING
Method
The goals of our study were to understand how the gestures designed help and engage children to reason and learn in spatial tasks. We also compared the players’ in-game behavior between 1) different versions of the gesture design, 2) the gestural interface and the mouse-based interface. We conducted the study at an afterschool program center and recruited 19 participants through the program director to participate in a 50-minute study session. Their ages ranged from 8 to 12 (median=10, mean=9.84) with 8 of them being female.
USER TESTING
Procedure
The participants spent 20 minutes playing each version of the game (gesture versus mouse control) – 40 minutes in total. After playing, they filled out a post-game survey that asked them to indicate their age and gender and rate how easy, fun, frustrating, and tiring they thought each version of the game was and why. We counterbalanced the gameplay order of the gesture and mouse-controlled version. For each control scheme, participants played the Tutorial level followed by the Easy level, and then were assigned to play one of the more challenging levels: Rocket Boots or Sledgehammer. If the participant was assigned to play the Rocket Boots level for the first control scheme they tried, then they were assigned to play the Sledgehammer level for the second control scheme, and vice versa.
USER TESTING
Findings & Insights
Not surprisingly, most participants reported that the mouse control was easier to use, less frustrating ("Something didn’t work [using gesture]"), and less tiring ("Need to move hands a lot [using gestures]" or "Arms wouldn’t get tired [using the mouse]"). But when asked how fun they thought using the gestural interface was, participants responded with statements like "You have lots of fun and more experiments", "You get to learn and have fun doing it", or simply "Liked to move my hand", "My hand is in the game", etc., showing participants’ engagement with the game and using gestures. In addition, some participants (n=4) reported that the gestural interface was either easier to use or more preferable to play with. Even though the participants’ familiarity with mouse control may give them an advantage in performance or learnability with our mouse control scheme, the additional learnability difficulties introduced by our gesture control scheme could potentially enhance recall and transfer during the learning process, based on the theory of desirable difficulties.
Publications
Supporting Spatial Skills Learning with Gesture-based Embodied Design
Po-Tsung Chiu, Helen Wauck, Ziang Xiao, Yuqi Yao, and Wai-Tat Fu. 2018.
In 23rd International Conference on Intelligent User Interfaces (pp. 67-71). ACM.
[PDF]