Our recent online roundtable – Can engagement data help to address risks to equality of opportunity? was designed to find ways to open up conversations within the higher education sector around whether student engagement data can help universities to be more agile and flexible when it comes to evaluating mid-cycle progress against Access and Participation plan activity.
We started from a set of very simple premises:
- We know that risks to equality of opportunity exist for different groups of students and that these risks differ for each institution
- Universities who want to charge above the basic tuition fee cap are required by the Office for Students to detail how they plan to improve equality of opportunity for students from disadvantaged backgrounds to access, succeed in and progress from higher education
- Those resulting Access and Participation Plans (APPs) detail the intervention strategies and associated activities that universities will implement over a four-year period to address those risks to equality of opportunity, alongside ways to measure and evaluate the impact of those interventions.
Based on these premises, we wanted to drive discussions around how the sector knows if the actions in their APP plans are working, and whether real-time engagement data, like those available from StREAM, Kortext’s student analytics platform can support universities to do this.
Addressing risks to equality of opportunity
According to the Office for Students, universities are required to use their Equality of Opportunity Risk Register to interrogate their own data and explore:
- Who is at risk within the student population at each institution
- How students may be affected
- How universities can contribute to addressing the risk either within their own student population or nationally
We undertook an informal ‘survey’ with audience members exploring how they felt in principle about using data to address risks to equality of opportunity were largely positive, reinforcing our initial supposition, while not focussing on student engagement data in particular.
Figure 1: Word cloud exploring how attendees felt about using data to address risks to equality of opportunity
However, we then undertook a quick poll with attendees, to ask how confident they are in understanding the requirements around work to address risks to equality of opportunity. Those results were more ambiguous, suggesting that there is still some work to be done in terms of fully getting to grips with the new approach to access and participation:
Figure 2: Participants confidence in understanding the requirements around addressing risks to equality of opportunity
Note: This confidence analysis reflects the roles and responsibilities of the audience and cannot be considered representative of the sector as a whole.
Current mechanisms in use by universities represented on the roundtable include, not surprisingly, official end-of-cycle data sources such as that from the OfS and UCAS, as well as internal, real-time sources which, by their nature are more agile.
From lag to lead indicators
Through ongoing conversations with our clients, we are always exploring ways to make student engagement data more accessible and meaningful to those staff who are working on the frontline of student support. The difference between end-of-cycle data and the near real-time data that you see in StREAM is that the data visible in StREAM helps universities to know which students need help at any given moment in time, thereby providing lead indicators to universities that can help them be more agile in their student support framework and processes and to get a mid-year, mid-cycle sense of progress. Action to refocus activity and purpose can follow.
Why ‘engagement’ works
Since the initial development of the StREAM platform, in conjunction with Nottingham Trent University back in the early 2010s, it has always been our approach that engagement is the most valuable indicator of risk – more so than relying solely upon demographic or other factors as a presumption of risk.
Using engagement as a risk indicator works for the following reasons:
- It provides a universal measure of participation that can be consistently applied across different areas of university life
- Engagement (or participation) is a positive and non-biased indicator
- Engagement is based on current effort, not on factors that students cannot affect
- Engagement can be measured in real-time, providing actionable lead indicators
- Categories of engagement are neutral, but permit a nuancing of outreach and support activity tailored to each category
- Personalised interventions follow, in agreement with students
- Management analysis at cohort, programme, subject and faculty level can facilitate targeted outreach campaigns at key points in the academic lifecycle and inform resource-allocation.
Our approach is validated by research findings. A study conducted by Foster and Siddle (2019) at Nottingham Trent University found that alerting students on the basis of their non-engagement with their studies was more efficient at spotting students not progressing than when using demographic data alone when using widening participation status. The study concluded that:
The odds of students with widening participation status generating an alert were on average 43% higher, demonstrating the potential of [StREAM] to preferentially target support at disadvantaged groups without needing to target directly based on immutable factors such as their socio-economic background.
Other universities have undertaken their own research which also supports the value of using engagement as the identifier of risk. Engagement was also found to be widely understood and accepted by students. Our 2023 research with Wonkhe discovered that the concept was meaningful for student participants, and they were comfortable with judging what engagement behaviours carry meaning and value.
It’s good to talk!
As I write, we continue to explore ways to enhance the level of insights available to institutions from student engagement data, particularly in the context of access and participation activity where more management level insights that show how students are collectively participating in their studies are required. The ability to interrogate data based on the objective analysis of student engagement with their studies and overlay that analysis with an exploration of the impact that some of those protected demographic characteristics may have on individual participation, or collectively across the university, can help to identify structural barriers to engagement that can then be dismantled across the university.
Such insights can also help with the wider evaluation piece that is so important to the sector – irrespective of the regulatory imperative driving current evaluation activity. In answering the ‘what works’ question, considerations of transferability, scalability and affordability must also be factored in and meaningfully addressed in any responses. And this is why we need to continue to talk and partner with the sector to develop and create solutions that improve the student experience and enhance their outcomes.
Here at StREAM by Kortext, we maintain that student engagement data can help in the identification of risks to equality of opportunity. While data can never explain the reasons why risks to equality of opportunity exist (and they may be multiple in nature), it is indicative of the existence of those risks. Further (human) investigation is therefore required to understand and take appropriate action in partnership with students, to address those risks.
If you’re keen to continue the conversation, please do get in touch with us. And if you’d like to find out more about how StREAM could add value to your institution, you can book a call with our team.
References:
Ed Foster & Rebecca Siddle (2020) The effectiveness of learning analytics for identifying at-risk students in higher education, Assessment & Evaluation in Higher Education, 45:6, 842-854, DOI: 10.1080/02602938.2019.1682118
