Measuring translation trainees’ effort in AI-assisted post-editing : a multi-method approach

Abstract

The incorporation of artificial intelligence (AI) into translator training has garnered growing scholarly attention. Despite this, the effectiveness of AI in mitigating translation trainees’ post-editing (PE) effort remains underexplored. This study, grounded in Krings’ tripartite model of PE effort, investigates the efficacy of AI-assisted post-editing (AIPE) across temporal, technical, and cognitive dimensions compared to traditional post-editing (TPE). Employing a multi-method approach that combines eye-tracking, key-logging, self-rating, and retrospective interviews, this study explores the impact of AIPE across two translation briefs: full post-editing (FPE) and light post-editing (LPE). Twenty-six postgraduate students performed PE tasks on English-to-Chinese machine-translated texts, with half of these tasks supported by GPT-4’s chain-of-thought reasoning and suggestions. Results indicate that while PE mode had no significant effect on temporal effort, AIPE significantly increased technical effort compared to TPE. AIPE demonstrated an inconsistent pattern in mitigating cognitive effort, as evidenced by eye-tracking and pause-related indicators. Regarding translation briefs, FPE consistently demanded higher effort across all dimensions than LPE, though AIPE demonstrated greater potential in alleviating effort in LPE. These findings contribute to the growing body of literature on AI-mediated translation practices, offering insights into integrating AI tools into translator training programmes and refining pedagogical strategies to meet industry demands.