One of the challenges of designing game-based assessments is that game-based learning experiences are often non-linear, dynamic, and highly contextualized. Capturing the complexity of learning progression and interpreting the meaning of learning activities based on the contexts can be difficult. For instance, representing students’ competencies using summary statistics or directly observed in-game behaviors may be straightforward, but it overlooks how learning with games is a situated process. Conversely, specific performance rules may be highly context-specific, lacking transferability to a different setting. In this talk, I will discuss two cases of in-depth investigations into learners’ in-game problem-solving behaviors in educational games using quantitative methods. Both cases aim to build an in-context understanding of ordinary game behaviors that have implications for learning and assessments. Finally, I will discuss a generalized approach to designing game-based assessments that is based on fine-grained evidence models with contexts taken into consideration. The findings will help researchers and practitioners think about how to design game-based assessments by understanding learning and how to use game-based assessment as a tool to understand learning.
Zhichun “Lukas” Liu is an Assistant Professor of Learning Design and Implementation Sciences at the Faculty of Education, the University of Hong Kong. His work has been exploring how to design game-based learning experiences by engaging learners in active problem solving. His current work aims at promoting the development and learning transfer of computational thinking among K -12 students and teachers through educational games and robotics. His research interests also include using quantitative analytics methods (e.g., Hidden Markov Model, Bayesian Network, sequential analysis, and social network analysis) to understand learners’ behavior, competence, discourse, and interactions. Prior to joining HKU, he worked at Kaput Center at the University of Massachusetts Dartmouth as a postdoctoral fellow and at Teachers College, Columbia University as a Visiting Assistant Professor in Learning Analytics.