Driving student success with analytics and personalised support

Driving student success with analytics and personalised support

On Day 2 of the Victorian Global EdTech and Innovation Expo 2021, Sacha Nouwens and Marcus O’Donnell addressed how data analytics are being used to support and retain university students.

The morning session of the second day of the Victorian Global EdTech and Innovation Expo 2021 brought together three education experts to explore the use of data analytics in universities. 

They discussed how analytics are currently being used to support student success, the impact of the pandemic, whether analytics have improved retention rates and the future of analytics in higher education.

Belinda Howell, Strategic Advisor at LINC Education, moderated. She was joined by two panellists:

  • Sacha Nouwens, Executive Director, Student Experience and Insights at Online Education Services (OES)
  • Marcus O’Donnell, Director, Cloud Learning Futures at Deakin University

How data analytics are being used to support student success

Sacha Nouwens explains that OES partners with universities to help them deliver their courses fully online or digitise parts of their learning experience. They use data analytics to:

  • Assist universities in solving problems to increase student success and retention.
  • Show educators the effects their course content changes have on students. Have the changes increased the amount of time students spend studying? Have they helped them achieve better results?
  • Identify at-risk students and help universities understand the steps they need to take to support those students.
“We work with universities to help them make the best use of their data.”
Sacha Nouwens, Online Education Services (OES)
Victorian Global EdTech & Innovation Expo - Sacha Nouwens Day 2 AM Session

Marcus O’Donnell says they’ve been using data in similar ways at Deakin University. Their initiatives include:

  • The Priority Students Program that brings together several different data sets to identify students who need personalised support ranging from phone calls to peer mentoring.
  • The Teacher Dashboard Project that aims to collect data insights for every teacher to help them understand what’s happening with the students in their group.

The impact of the pandemic on the use of data analytics

According to Sacha Nouwens, OES’s way of operating didn’t change much because most of their programs were already online or blended. But COVID was a game changer because:

  • Universities had to move online quickly, providing OES with an opportunity to assist them with analytics and digitising other elements of the student experience.

Marcus O’Donnell agrees that day-to-day operations didn’t change much at Deakin University because most of their courses were already online, but the pandemic did raise a major question around student engagement:

  • How do we keep students engaged when there are things going on in both students’ and teachers’ lives that might mean education isn’t their number one priority?
“[The pandemic] brought a new sense of focus on what digital education really means.”
Marcus O’Donnell, Deakin University

The combination of data is key to student success

Both Sacha and Marcus agree that there isn’t one specific element of data that can be pinpointed as the most important in determining student success. It’s the combination of data or the interaction between data sets that really matters. 

According to Sacha, the collection of data requires:

  • A high-quality learning management system (LMS) to understand the interactions students are having in the systems.
  • A great customer relationship management system (CRM) to know when students are reaching out to staff members and what they’re asking.
“[Within] a matter of weeks of observing their behaviour, we can quickly get a quite accurate prediction [of] which students are going to fly on their own and which ones need an extra helping hand. But we really find that it’s not one single data point, it’s the combination of all of it together.”
Sacha Nouwens, Online Education Services (OES)

The invisibility of analytics to students

Sacha Nouwens explains that analytics are invisible to students and they generally wouldn’t realise they’re being used. Here’s how it works at OES:

  • Their analytics identify when a student is struggling and a helping hand reaches out.
  • They use a high-touch model that connects 25 students to one tutor for each class.
  • A tool also serves up analytics to teachers, but OES doesn’t use automated responses because they want teachers to be able to overlay their knowledge. If they’ve already been in touch with a student, a message that’s not in sync with the previous conversation wouldn’t work.
“With our analytics, we’re really trying to focus on the group that’s not going to be successful without the extra help.”
Sacha Nouwens, Online Education Services (OES)

Marcus O’Donnell believes that automated systems can help, but teachers need to be able to modify the responses. At Deakin University:

  • An intelligent agent can send automated messages to students when they do something unexpected, but their new Teacher Dashboard Project aims to provide teachers with a series of template responses they can adapt depending on their communication history with a given student.
  • Their goal with the dashboard is to use analytics to make life easier for both the teachers and the students.

The challenges of developing analytics

According to Sacha, the adaption of analytics in universities comes with several challenges:

  • Even though OES works closely with teaching staff to design systems and processes that work for them, what teachers think they need doesn’t always turn out to be helpful and they frequently have to start over.
  • Building a bespoke system is a long iterative process and everyone has to accept that.

Marcus O’Donnell agrees and points out that:

  • Analytics is a new area, so we have to be patient and give teachers time to understand how it can integrate into their work.
“Analytics is about individual personalised attention when you’re doing things at scale. That’s the new area.”
Marcus O’Donnell, Deakin University

How analytics have improved retention rates

Marcus O’Donnell says that while they’re still in the early stages of analytics at Deakin University, the foundations are in place and they have noted:

  • Improved retention and a positive effect.

Sacha Nouwens explains that OES does slow rollouts of new technologies to be able to measure their impact. They’ve found that:

  • Before OES used analytics in their interventions, retention rates improved by 2 percent for one of their partners.
  • Now that they’re using analytics, the retention rates for that same partner are 5 percent better.
“When we got the data behind it, it [made] an even bigger difference.”
Sacha Nouwens, Online Education Services (OES)

The pros and cons of analytics that feed directly to the student

According to Marcus O’Donnell, giving students direct feedback about their performance has advantages and disadvantages:

  • They get a really good idea of their self-regulating processes, motivations and engagement. This information could help them perform better in an online environment.
  • At the same time, giving students the right feedback requires taking individual differences and psychometric data into account. We can’t give a student advice on what to do next without considering what type of learner they are and how they approach their goals.
“Matching the analytics we get and the feedback we give to students with specific learning designs… is really coming up. [It’s] the way that we can actually get better value out of the data.”
Marcus O’Donnell, Deakin University
EduGrowth Victorian Global EdTech & Innovation Expo - Day 2 Session Marcus O'Donnell

Sacha Nouwens agrees that giving students direct feedback is tricky because:

  • Students need to understand the data and the next steps they need to take.
  • OES conducted an experiment early on where they told students how much time they spent in the LMS and how that compared to top-performing students. They ended up pulling back on it because it didn’t help the students who needed the most help and it didn’t tell students what they needed to change. OES is now researching next steps in this area.

The challenges of data analytics in universities

According to Sacha Nouwens, the main challenges are that:

  • Universities have legacy systems.
  • With all the different departments working independently, it’s extremely challenging for the central data analytics team to service the needs of the whole university.

Marcus O’Donnell believes that:

  • Universities must educate staff to become analytically data proficient if all this data is to be useful.
  • We have to accept it’s a long and slow process for universities to build their analytics and make the most of them.

Advice for universities about how to approach analytics

Sacha Nouwens says that:

  • Universities are behind other sectors such as banks, social media companies and gambling companies when it comes to analytics.
  • These companies invest a lot of money in analytics because it brings big returns and she would love to see universities make that same investment. 

As for Marcus O’Donnell, he believes that:

  • Students need to be brought along on the analytics journey.
“Data is an incredibly contested area because of a range of things that have been happening, so it’s really important to work directly with students, explain to them what we’re doing and get informed consent to move forward.”
Marcus O’Donnell, Deakin University