Conducting a Usability Playtest of a Mathematics Educational Game with Deaf and Hearing Students

EasyChair Preprint 4828

Abstract

In this paper, a usability playtest to a pre-Alpha version of a video game namely "OtherWordly Math" is presented focused on Game-based Learning (GBL), Inclusive Design for deaf students and Games User Research. The present study aims to evaluate usability, understanding information and players’ experience to improve gameplay. The sample is composed of nine participants, four being deaf and five hearing children, aged between 10 and 14, five girls and four boys. Three instruments were applied remotely using an online platform: a) participants' self-report on difficulties they may find in-game; c) an observational grid; and d) an emotional scale to assess the intensity of emotions felt by the players in-game. The results show that deaf and hearing children report the same playability constraints: a) the objectives of game challenge 2 were not clear; b) a user interface icon is ambiguous in challenge 3; and c) players expect more exploration in every challenge. Both groups felt "very" satisfied and "a little" confused during gameplay.  A new level design and a new game layout were developed to fix the understanding and usability problems found in challenge 2.  The usability problem related with challenge 3 demanded icon redesign. Regarding the playability issue mentioned in c) the solution points towards a game design review to balance the learning objectives with playability. Solutions are proposed concerning the main design problems reported by observers and by participants and a reflection about the value of User Experience Evaluation in the context of DHH students is provided.

Keyphrases: Games User Research, Research-based Educational Games, Usability Playtest, deaf students

@booklet{EasyChair:4828,
  author    = {Lília Marcelino and Conceição Costa and José Carlos Neves and Andreas Melo and Fernando M. Soares},
  title     = {Conducting a Usability Playtest of a Mathematics Educational Game with Deaf and Hearing Students},
  howpublished = {EasyChair Preprint 4828},
  year      = {EasyChair, 2020}}