Learning analytics for student modeling in virtual reality training systems: Lineworkers case

Abstract

Live-line maintenance is a high risk activity. Hence, lineworkers require effective and safe training. Virtual Reality Training Systems (VRTS) provide an affordable and safe alternative for training in such high risk environments. However, their effectiveness relies mainly on having meaningful activities for supporting learning and on their ability to detect untrained students. This study builds a student model based on Learning Analytics (LA), using data collected from 1399 students that used a VRTS for the maintenance training of lineworkers in 329 courses carried out from 2008 to 2016. By employing several classifiers, the model allows discriminating between trained and untrained students in different maneuvers using three minimum evaluation proficiency scores. Using the best classifier, a Feature Importance Analysis is carried out to understand the impact of the variables regarding the trainees' final performances. The model also involves the exploration of the trainees' trace data through a visualization tool to pose nonobservable behavioral variables related to displayed errors. The results show that the model can discriminate between trained and untrained students, the Random Forest algorithm standing out. The feature importance analysis revealed that the most relevant features regarding the trainees' final performance were profile and course variables along with specific maneuver steps. Finally, using the visual tool, and with human expert aid, several error patterns in trace data associated with misconceptions and confusion were identified. In the light of these, LA enables disassembling the data jigsaw quandary from VRTS to enhance the human-in-the-loop evaluation.