A learning analytics approach to the evaluation of an online learning package in a Hong Kong University

Abstract

In recent years, research using learning analytics to predict learning outcomes has begun to increase. This emerging field of research advocates the use of readily-available data to inform teaching and learning. The current case study adopts a learning analytics approach to evaluate the online learning package of an academic English course in a university in Hong Kong. This study aims to (1) explore the completion pattern of use of the online learning package by students in a generic undergraduate academic skills course; and (2) predict student outcomes based on their online behaviour patterns. Over three academic years, the study examined usage logs for 7000+ students that were available on the university's learning management system. Student assessment component scores, online activity completion rates, and online behavioural patterns were identified and examined using descriptive analysis, bivariate correlation analysis, and multiple regression analysis. The findings reveal insights into different online learning behavioural patterns that would benefit blended course designers. For instance, some students started using the online learning package early in the semester but fulfilled only the minimum required online work, whereas others greatly exceeded the basic requirement and continued doing activities in the online package even after the semester had finished. The relationship between learning activities in the online package and assessment component grades was found to be weak but meaningful. A regression model was developed drawing on the completion rates to predict overall student scores, and this model successfully identified several specific factors, such as total number of attempts and performance in individual online learning activities, as predictors of the final course grade.