Learning Analytics have played a key role in determining the effectiveness of learning from the courses provided as part of our curricula development. EDSA has analysed data to gain insights around how to improve module curricula outlines, content, and their delivery method. Learning Analytics have been used throughout the project to judge engagement with the learning materials that we produced. This has helped us to ensure that the content is correct, the delivery method is suitable and that they are appropriate for the broad range of audience groups that we have targeted in the project.
The EDSA project has had a duration of 3 years and was coordinated by Prof John Domingue (The Open University). John has been featured in a BBC News article entitled “OU students’ progress to be monitored by software”, reflecting the inherent expertise of the consortium for innovating and leading in this area.
With Learning Analytics it has been possible to obtain valuable information about how learners interact with the EDSA courseware. Our approach has been based on tracking learner activities, which consist of interactions between a subject (learner), an object (learning activity) and is bounded with a verb (action performed). For this purpose, we have used the Experience API (or xAPI or Tin Can API) in order to express learner activities, as well as the open-source Learning Record Store (LRS) Learning Locker in order to store and visualise the captured learner activities.
Case Study: The EDSA Process Mining MOOC
TU/e has applied process mining techniques on recorded student behaviour of the Process Mining MOOC on Coursera, in order to do a preliminary analysis of the learning process within the MOOC. The MOOC itself is set up in a structured way: there are 6 weeks of content and for each content week a quiz has to be made, at least 2 weeks after the week became available. At the end of the course there is also a final quiz. For the students that want to aim for a certificate with distinction, a quiz, peer assessment and peer review is also mandatory.
When we analysed the registration behaviour, we noticed a peak in subscriptions around the opening time, especially for the students that would end up getting a certificate. We then went on to compare the groups of students that did not get a certificate, the ones that got a normal certificate and the ones that got a distinction certificate. Each group was further split into the non-signature track and signature track, e.g. the ones that did not or that did pay for their certificate, which would imply higher motivation and participation rates. The behaviour of the successful students was clearly timelier than the ‘failed’ students, where the signature track students are even timelier.
We then created the expected learning model, where students study sequentially, and verified how well this model fits each of the different groups. Although we expected the successful signature track students to best fit this model, it seemed that the students that failed the course but that did watch all weeks studied the most sequentially. Further investigation is needed to discover the cause of this difference.
For more details, please see the relevant EDSA deliverables: