Detection of online student behavior using emotion and eye/head movement

Abstract

During the COVID-19 pandemic of the past few years, online/hybrid teaching has been used around the world, posing challenges for teachers and students alike. One challenge is related to monitoring online student behavior. Facial recognition technologies offer a promising solution, providing useful references for teachers. In this paper, we present our initial work on using emotion, and eye and head movement to detect online student behavior. In particular, we study how these methods can be used to detect five common classroom behaviors: reading slides, writing notes, thinking, checking phones, and engaging in classroom activities, through test cases with the aim of identifying key characteristics. By using the aforementioned methods collectively, more accurate detection results can be achieved. The findings (e.g., key characteristics) should provide valuable insights into understanding online student behavior, and future machine learning work in particular.