Grounding pedagogical intents in embodied AI teachers : a human-centered framework for designing and evaluating instructional gestures

Abstract

Generative AI expands opportunities for embodied agents in HCI, yet a gap persists between human-centered AI principles and practical design methods, particularly for pedagogical agents’ (PAs) co-speech gestures. Automated text-to-gesture systems lack the instructional nuance needed for effective teaching. To address this, we used a Research-through-Design approach to develop a human-centered framework that translates pedagogical intent into gesture specifications for embodied AI teachers. The framework includes four iterative stages: Preparation, which analyzes gesture patterns and instructional functions; Human PA Acting, where educators and designers rehearse gestures through performance; Embodied PA Acting, which transfers human motion to agents using video-based pose estimation; and EPA-assisted Course Delivery, which evaluates student experiences through interviews. Findings indicate that these gestures enhanced students’ perception of the PA’s professionalism, approachability, and instructional rhythm, while boosting overall engagement. This work contributes a design framework and insights for pedagogical gesture design, and exploratory guidance for generative-AI prompting.