Perceived MOOC satisfaction : a review mining approach using machine learning and fine-tuned BERTs

Abstract

This study investigates the application of machine learning and BERT models to identify topic categories in helpful online course reviews and uncover factors that influence the overall satisfaction of learners in massive open online courses (MOOCs). The research has three main objectives: (1) to assess the effectiveness of machine learning models in classifying review helpfulness, (2) to evaluate the performance of fine-tuned BERT models in identifying review topics, and (3) to explore the factors that influence learner satisfaction across various disciplines. The study uses a MOOC corpus containing 102,184 course reviews from 401 courses across 13 disciplines. The methodology involves three approaches: (1) machine learning for automatic classification of review helpfulness, (2) BERT models for automatic classification of review topics, and (3) multiple linear regression analysis to explore the factors influencing learner satisfaction. The results show that most machine learning models achieve precision, recall, and F1 scores above 80%, 99%, and 89%, respectively, in identifying review helpfulness. The fine-tuned BERT model outperforms baseline models with precision, recall, and F1 scores of 78.4%, 74.4%, and 75.9%, respectively, in classifying review topics. Additionally, the regression analysis identifies key factors affecting learner satisfaction, such as the positive influence of “Instructor” frequency and the negative impact of “Platforms and tools” and “Process”. These insights offer valuable guidance for educators, course designers, and platform developers, contributing to the optimization of MOOC offerings to better meet the evolving needs of learners.