Measuring User Experience on Massive Open Online Course with User-Centered Metrics: A Case Study

Abstract

Evaluation of user experience (UX) in learning applications is now very important. A
good UX on MOOC will have an impact on increased completion rates and student satisfaction,
student engagement, enhanced learning, and minimizing course implementation rework. In this
study, the authors use MOOC called E-learning: Open of Knowledge Sharing (eLOK) as the
object of research. User-centered metrics or known as HEART framework consisting of five
metrics, including Happiness, Engagement, Adoption, Retention and Task Success. The
framework that the authors used to measure the UX of eLOK is HEART metrics. The instrument
that used in this study is a questionnaire consists of 33 statements which delivered to 94 students
as the sample of the research, with the the number of population 1500. The questionnaire has
been validated using SPSS 23 as the tools. The result of the validation is all the items of the
questionnaire are valid and reliable with the percentage of reliability is 96%. The result of this
study showed that the metrics obtained by Happiness was 76.5%, Engagement was 72.1%,
Adoption was 73.6%, Retention was 72.6%, and Task Success was 78.6%. To achieve the GoalSignal-Metrics set on the HEART framework, it is necessary to re-design the eLOK display by
paying attention to UX and UI aspects, changing the greeting with user-guidance, improving pushnotification features and navigation.