Predicting student performance from Moodle logs in higher education ; A course-agnostic approach

Santos, R. M. C., & Henriques, R. (2023). Predicting student performance from Moodle logs in higher education: A course-agnostic approach. In M. Carmo (Ed.), Education and New Developments 2023 (Vol. 2, pp. 77-81). Science Press. https://end-educationconference.org/wp-content/uploads/2023/06/Education-and-New-Developments_2023_Vol_II.pdf ; The institutional adoption of learning management systems (LMS) aims to improve educational outcomes and reduce churn through student engagement with educational content. Modern LMS record all student interactions and store them as activity logs that encode patterns of learning behaviour. Previous research has shown that insights derived from log data can detect students at risk of failing in a single or a few courses, but comprehensive institution-wide surveys are few and far between. The work presented herein uses machine learning to create predictive models to identify students at risk or excellent students using the Moodle logs generated by a sample of 9296 course enrollments at a Portuguese information management school. 31 candidate features were extracted to create and train different predictive models. Model performance was evaluated through 30 repetitions of Stratified K-Fold Cross-Validation, using the area under the receiver operating characteristic (ROC) curve (AUC) and the F1-score. All experiments were repeated with the addition of the average of the intermediate grades obtained by the student in the course as a 32nd candidate feature. The results suggest that features extracted from Moodle logs are good predictors of students at risk, as indicated by the 0.752 AUC score achieved by Random Forest. The addition of intermediate grades significantly improves the predictive performance, leading to an AUC score of 0.922 and F1-Score of 0.693 for the best classifier, Gradient Boosting. However, the performance for identifying excelling students was comparatively lower, with an AUC score of 0.781 and F1-Score of 0.567 for Gradient Boosting. Future work should focus ...