It is currently known that certain commercial video games, such as Portal 2, Call of Duty, and Tetris, can train players' spatial reasoning skills, while cognitive training games like Lumosity, Cogmed, and Dual N-Back have either no effect on spatial skills or only near transfer. What determines whether a given video game will train players' spatial skills? This project seeks to answer this question by systematically isolating and testing different game features for their effectiveness at training players' spatial skills in a video game. We developed our own game, Homeworld Bound, to use as a testbed for our studies.

In effort to study how different factors in gesture design could tap into the cognitive operations for spatial reasoning skills, we designed and developed the gesture version by integrating the use of Leap Motion. The gestures were designed with enhancing the user experience and effectiveness of learning in mind, by having a congruent mapping between hand gestures and spatial operations.


User Testing


Helen Wauck

Po-Tsung Chiu

Ziang Xiao


Aug 2016 - Dec 2017


Who is this game designed for?

Although people of all ages can benefit from spatial skill training, we are currently targeting children ages 7-11 for this project. This is the age range where gender differences in spatial skills in favor of boys begin to emerge. We hope to intervene before this gender gap becomes a career barrier for girls interested in pursuing STEM and other disciplines where spatial skills are essential.



Mouse Version

Homeworld Bound is a fully playable, immersive first person adventure and construction game designed with three guiding principles: Evidence-Based Content, Engagement, and Modularity. Incorporating Evidence-Based Content inspired by the commercial games empirically shown to be effective at training spatial skills allows us to focus on game features that are most likely to train spatial skills rather than groping in the dark. Thoroughly playtesting our game to make sure that it's fun and promotes player Engagement helps us avoid the "chocolate-covered broccoli" style of learning used in many educational games, which may contribute to their ineffectiveness at far-transfer cognitive skill training. Modularity enables us to break our game down into separate, independent units of varying granularity so we can test and isolate specific game features for spatial skill training.



Gesture Version

Our gesture design focused on two specific operations for moving and rotating parts in the game. We refer to these as Movement and Rotation gestures.

Left: the Easy level. Right: the Sledgehammer level. The 2D images on the upper left corners show what the 3D objects would look like when all parts are attached correctly. Rotate and Move modes indicates the spatial operation the player can perform at the moment.



The goals of our study were to understand how the gestures designed help and engage children to reason and learn in spatial tasks. We also compared the players’ in-game behavior between 1) different versions of the gesture design, 2) the gestural interface and the mouse-based interface. We conducted the study at an afterschool program center and recruited 19 participants through the program director to participate in a 50-minute study session. Their ages ranged from 8 to 12 (median=10, mean=9.84) with 8 of them being female.



The participants spent 20 minutes playing each version of the game (gesture versus mouse control) – 40 minutes in total. After playing, they filled out a post-game survey that asked them to indicate their age and gender and rate how easy, fun, frustrating, and tiring they thought each version of the game was and why. We counterbalanced the gameplay order of the gesture and mouse-controlled version. For each control scheme, participants played the Tutorial level followed by the Easy level, and then were assigned to play one of the more challenging levels: Rocket Boots or Sledgehammer. If the participant was assigned to play the Rocket Boots level for the first control scheme they tried, then they were assigned to play the Sledgehammer level for the second control scheme, and vice versa.


Findings & Insights

Not surprisingly, most participants reported that the mouse control was easier to use, less frustrating ("Something didn’t work [using gesture]"), and less tiring ("Need to move hands a lot [using gestures]" or "Arms wouldn’t get tired [using the mouse]"). But when asked how fun they thought using the gestural interface was, participants responded with statements like "You have lots of fun and more experiments", "You get to learn and have fun doing it", or simply "Liked to move my hand", "My hand is in the game", etc., showing participants’ engagement with the game and using gestures. In addition, some participants (n=4) reported that the gestural interface was either easier to use or more preferable to play with. Even though the participants’ familiarity with mouse control may give them an advantage in performance or learnability with our mouse control scheme, the additional learnability difficulties introduced by our gesture control scheme could potentially enhance recall and transfer during the learning process, based on the theory of desirable difficulties.


Supporting Spatial Skills Learning with Gesture-based Embodied Design

Po-Tsung Chiu, Helen Wauck, Ziang Xiao, Yuqi Yao, and Wai-Tat Fu. 2018.
In 23rd International Conference on Intelligent User Interfaces (pp. 67-71). ACM. [PDF]