[et_pb_section fb_built=”1″ admin_label=”section” _builder_version=”4.16″ global_colors_info=”{}” theme_builder_area=”post_content”][et_pb_row admin_label=”row” _builder_version=”4.16″ background_size=”initial” background_position=”top_left” background_repeat=”repeat” global_colors_info=”{}” theme_builder_area=”post_content”][et_pb_column type=”4_4″ _builder_version=”4.16″ custom_padding=”|||” global_colors_info=”{}” custom_padding__hover=”|||” theme_builder_area=”post_content”][et_pb_text admin_label=”Text” _builder_version=”4.27.0″ background_size=”initial” background_position=”top_left” background_repeat=”repeat” hover_enabled=”0″ global_colors_info=”{%22gcid-8048ee98-ce8c-41b5-876c-e0d5362f7c6b%22:%91%22link_text_color__hover%22%93}” link_text_color__hover_enabled=”on|hover” link_text_color__hover=”#2d0962″ theme_builder_area=”post_content” sticky_enabled=”0″]<\/p>\n
The Office for Students (OfS) launched its revised approach to <\/span>Access and Participation Plans<\/a><\/strong> (APPs) last year to redefine the expectations for higher education providers. Within the requirements are details of the interventions providers must make to ensure the improvement of equality of opportunity for underrepresented groups to access, succeed and progress within higher education.<\/span>\u00a0<\/span><\/p>\n Making higher education accessible doesn\u2019t stop at getting in. Accessibility encompasses the entire student journey, from getting in to getting on, and going on to complete a course.<\/span>\u00a0<\/span><\/p>\n Universities aim to provide consistent support to students throughout their time at an institution. Determining whether the support and outreach initiatives that have been devised and then implemented are working can be challenging. Here at StREAM, we are wondering whether student engagement analytics data can help.<\/span>\u00a0<\/span><\/p>\n In particular, whether mid-cycle insights from student engagement data could help determine progress against targets and objectives as well as provide an opportunity for mid-cycle revisions in response to feedback, evidence of impact (or otherwise) and resource availability.<\/span>\u00a0<\/span><\/p>\n <\/span><\/p>\n <\/span><\/p>\n In the associated guidance to universities writing their APP, the Office for Students (OfS) <\/span>explains<\/a><\/strong> the role and importance of developing initiatives that are underpinned by a theory of change:<\/span>\u00a0<\/span><\/p>\n A theory of change model is a strategic planning tool \u2026 to ensure that long-term goals are defined and understood. \u2026 [A] theory of change model can provide a clear rationale for how short-term activities are connected to longer term improvements in access and participation, and the evidence practice which underpins them. By identifying the different intermediate outcomes that an intervention may produce, the activity can be designed to meet these outcomes.<\/span>\u00a0<\/span><\/p>\n A natural corollary of this approach is a robust evaluation that provides high-quality evidence about effective practice when it comes to access and participation activity undertaken by universities. Within UK HE, this work is supported primarily by the work of the Centre for Transforming Access and Student Outcomes in Higher Education (TASO), an OfS-funded independent charity.<\/span>\u00a0<\/span><\/p>\n <\/span><\/p>\n <\/span><\/p>\n In his keynote address to the Universities UK Access, Participation and Student Success Conference in November 2023, John Blake, OfS Director for Fair Access and Participation, was clear that in the 20 years since APPs were introduced, it is still unclear exactly \u2018what works\u2019, concluding that the evidence simply isn\u2019t there. Blake further spoke of the need to support institutions to take measured risks in the types of interventions that they create even where there is no guarantee that the intervention will improve the relevant metrics. But he was clear that if improvements in metrics are seen, then it is important to know <\/span>how<\/span><\/i> and <\/span>why<\/span><\/i> the intervention was a success.<\/span>\u00a0<\/span><\/p>\n His comments set us thinking about whether student engagement data has a role to play here in helping universities to measure the impact of their interventions, particularly over the course of the APP duration so as to make appropriate enhancements or revisions to the intervention in response to feedback.<\/span>\u00a0<\/span><\/p>\n <\/span><\/p>\n <\/span><\/p>\n Student engagement analytics, is helping universities create a clearer picture of the ways in which students are engaging with their studies. Our <\/span>student engagement analytics platform, StREAM<\/strong><\/span><\/a>, explores how students are engaging day-to-day with their education, marked by a scale of low to high categories of engagement, in a secure and detailed manner. Codifying engagement provides a simple way of viewing and understanding the data so it is easier to understand and work with.<\/span>\u00a0<\/span><\/p>\n <\/span><\/p>\n <\/span><\/p>\n If engagement is consistently low or a student\u2019s engagement changes significantly \u2013 especially on the lower end \u2013 universities will be notified of a potential at-risk student in line with their own operational requirements. Unexplained changes in engagement patterns or engagement behaviours that are inconsistent with the rest of the cohort suggest that there is value in having an outreach conversation with a student to help explain and understand any factors that may be impeding their ability to participate fully in their studies. Identifying and collaboratively seeking to address issues in real-time provides clarity and consistency that allows universities to gain meaningful insight to help support progression at the individual student level.<\/span>\u00a0<\/span><\/p>\n <\/span><\/p>\n <\/span><\/p>\n The cumulative impact of effective support for individual student engagement is a positive impact on retention and continuation metrics. Rather than wait until the end of the APP cycle, we believe that mid-cycle insights can also be gleaned from student engagement data. Cohort, programme, subject, faculty and university level insights can help universities to understand the changing nature of demand on core student services across the academic year, identify fluctuations in demand and thereby enable an agile in-year response to help balance demand for a service with supply.<\/span>\u00a0<\/span><\/p>\n <\/span><\/p>\n <\/span><\/p>\n Reviewing the interventions implemented for individual students, for cohorts of students and groups of students as determined by characteristic or a group of characteristics, in conjunction with their outcomes can provide universities with a level of insight as to \u2018what works\u2019. However, these insights, or standards of evidence\u2019 have different values from an OfS\/TASO evalution perspective. Briefly,<\/span>\u00a0<\/span><\/p>\n Seeking to provide causal evidence can be challenging. While the OfS are happy for universities to take risks when planning outreach interventions, I doubt any university would sanction an intervention whereby students who have clearly been identified as being \u2018at risk\u2019 were randomly divided into two groups, one of whom received an outreach intervention and the other did not. Moreover, will it be possible to determine whether an intervention is successful for students with a particular set of personal or demographic characteristics? If a student successfully engages with an initiative to support re-engagement with their studies, can that success be attributed to gender, race, sexual orientation, being a care-experienced student or something else? Could it simply be that the intervention captured the attention of the student because it provided some level of intrinsic motivation or appeal?<\/span>\u00a0<\/span><\/p>\n <\/span><\/p>\n <\/span><\/p>\n In spite of these challenges, we continue to believe that student engagement data can nevertheless prove useful to universities through providing mid-cycle insights on the value of different outreach interventions. In recent years, we have worked with a range of clients who are all seeking to understand the impact of their interventions in respect of student engagement, continuation and progression and published the findings of their empirical research. Here is a sample of what our clients have found:<\/span>\u00a0<\/span><\/p>\n <\/strong><\/p>\n <\/span><\/p>\n The research findings detailed above show that universities are successfully using student engagement data to help address a range of use cases around student success, retention and wellbeing, among others. It is clear that more work needs to be done to better understand the relationship between the data and the outcomes and to answer the question posed by John Blake at the UUK conference around knowing <\/span>how<\/span><\/i> and <\/span>why<\/span><\/i> certain interventions are proving successful.<\/span>\u00a0<\/span><\/p>\n A deep dive into student engagement data has the potential to reveal a range of previously unknown insights about the student cohort, including insights on the relationship between personal\/demographic characteristics, engagement and student outcomes. The determination of \u2018risk\u2019 within StREAM is based solely on what students do i.e. how they engage with their studies, rather than on who they are, a decision based on our key principals around ethical and transparent use of data. However, triangulation with those personal and demographic insights provide valuable context when it comes to getting to know your students and agreeing how best to support each and every individual.\u00a0<\/span><\/p>\n Gathering data throughout the student journey allows universities to create a complex picture of what consists of their student population, and the unique requirements of individuals to access support and ultimately, succeed.\u00a0<\/span>\u00a0<\/span><\/p>\n Using this readily-available data can help universities ensure that their support services are relevant and accessible to all student cohorts and encourage widespread student participation, thus nurturing a truly accessible culture and learning environment for all.<\/span>\u00a0<\/span><\/p>\n \u00a0<\/span><\/p>\n To share your opinion, ideas and explore these questions further with us, book a demo<\/strong><\/a>.\u00a0<\/em><\/p>\n [\/et_pb_text][et_pb_divider color=”#16749D” divider_weight=”3px” _builder_version=”4.27.0″ _module_preset=”default” max_width=”80px” module_alignment=”center” custom_margin=”||5px||false|false” global_colors_info=”{}” theme_builder_area=”post_content”][\/et_pb_divider][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":" With the focus on access and participation plans, how can engagement data help the higher education sector evidence their impact?<\/p>\n","protected":false},"author":7,"featured_media":63013,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":" The Office for Students (OfS) launched its revised approach to <\/span>Access and Participation Plans<\/span><\/a> (APPs) last year to redefine the expectations for higher education providers. Within the requirements are details of the interventions providers must make to ensure the improvement of equality of opportunity for underrepresented groups to access, succeed and progress within higher education.<\/span>\u00a0<\/span><\/p>\r\nMaking higher education accessible doesn\u2019t stop at getting in. Accessibility encompasses the entire student journey, from getting in to getting on, and going on to complete a course.<\/span>\u00a0<\/span>\r\n\r\nUniversities aim to provide consistent support to students throughout their time at an institution. Determining whether the support and outreach initiatives that have been devised and then implemented are working can be challenging. Here at Solutionpath, we are wondering whether student engagement analytics data can help.<\/span>\u00a0<\/span>\r\n\r\nIn particular, whether mid-cycle insights from student engagement data could help determine progress against targets and objectives as well as provide an opportunity for mid-cycle revisions in response to feedback, evidence of impact (or otherwise) and resource availability.<\/span>\u00a0<\/span>\r\n The importance of evaluation\u00a0<\/strong><\/p>\r\nIn the associated guidance to universities writing their APP, the Office for Students (OfS) <\/span>explains<\/span><\/a> the role and importance of developing initiatives that are underpinned by a theory of change:<\/span>\u00a0<\/span>\r\n\r\nA theory of change model is a strategic planning tool \u2026 to ensure that long-term goals are defined and understood. \u2026 [A] theory of change model can provide a clear rationale for how short-term activities are connected to longer term improvements in access and participation, and the evidence practice which underpins them. By identifying the different intermediate outcomes that an intervention may produce, the activity can be designed to meet these outcomes.<\/span>\u00a0<\/span>\r\n\r\nA natural corollary of this approach is a robust evaluation that provides high-quality evidence about effective practice when it comes to access and participation activity undertaken by universities. Within UK HE, this work is supported primarily by the work of the Centre for Transforming Access and Student Outcomes in Higher Education (TASO), an OfS-funded independent charity.<\/span>\u00a0<\/span>\r\n\r\nThe need for evidence\u00a0<\/strong>\r\n\r\nIn his keynote address to the Universities UK Access, Participation and Student Success Conference in November 2023, John Blake, OfS Director for Fair Access and Participation, was clear that in the 20 years since APPs were introduced, it is still unclear exactly \u2018what works\u2019, concluding that the evidence simply isn\u2019t there. Blake further spoke of the need to support institutions to take measured risks in the types of interventions that they create even where there is no guarantee that the intervention will improve the relevant metrics. But he was clear that if improvements in metrics are seen, then it is important to know <\/span>how<\/span><\/i> and <\/span>why<\/span><\/i> the intervention was a success.<\/span>\u00a0<\/span>\r\n\r\nHis comments set us thinking about whether student engagement data has a role to play here in helping universities to measure the impact of their interventions, particularly over the course of the APP duration so as to make appropriate enhancements or revisions to the intervention in response to feedback.<\/span>\u00a0<\/span>\r\n Why student engagement data matters\u00a0<\/strong><\/p>\r\nStudent engagement analytics, is helping universities create a clearer picture of the ways in which students are engaging with their studies. Our <\/span>student engagement analytics platform, StREAM<\/span><\/a>, explores how students are engaging day-to-day with their education, marked by a scale of low to high categories of engagement, in a secure and detailed manner. Codifying engagement provides a simple way of viewing and understanding the data so it is easier to understand and work with.<\/span>\u00a0<\/span>\r\n\r\nImpact for individual students\u00a0<\/strong>\r\n\r\nIf engagement is consistently low or a student\u2019s engagement changes significantly \u2013 especially on the lower end \u2013 universities will be notified of a potential at-risk student in line with their own operational requirements. Unexplained changes in engagement patterns or engagement behaviours that are inconsistent with the rest of the cohort suggest that there is value in having an outreach conversation with a student to help explain and understand any factors that may be impeding their ability to participate fully in their studies. Identifying and collaboratively seeking to address issues in real-time provides clarity and consistency that allows universities to gain meaningful insight to help support progression at the individual student level.<\/span>\u00a0<\/span>\r\n\r\nCohort-level impact\u00a0<\/strong>\r\n\r\nThe cumulative impact of effective support for individual student engagement is a positive impact on retention and continuation metrics. Rather than wait until the end of the APP cycle, we believe that mid-cycle insights can also be gleaned from student engagement data. Cohort, programme, subject, faculty and university level insights can help universities to understand the changing nature of demand on core student services across the academic year, identify fluctuations in demand and thereby enable an agile in-year response to help balance demand for a service with supply.<\/span>\u00a0<\/span>\r\n\r\nEvidential standards\u00a0<\/strong>\r\n\r\nReviewing the interventions implemented for individual students, for cohorts of students and groups of students as determined by characteristic or a group of characteristics, in conjunction with their outcomes can provide universities with a level of insight as to \u2018what works\u2019. However, these insights, or standards of evidence\u2019 have different values from an OfS\/TASO evalution perspective. Briefly,<\/span>\u00a0<\/span>\r\n Why we think student engagement data can help\u00a0<\/strong><\/p>\r\nIn spite of these challenges, we continue to believe that student engagement data can nevertheless prove useful to universities through providing mid-cycle insights on the value of different outreach interventions. In recent years, we have worked with a range of clients who are all seeking to understand the impact of their interventions in respect of student engagement, continuation and progression and published the findings of their empirical research. Here is a sample of what our clients have found:<\/span>\u00a0<\/span>\r\n So what?\u00a0<\/strong><\/p>\r\nThe research findings detailed above show that universities are successfully using student engagement data to help address a range of use cases around student success, retention and wellbeing, among others. It is clear that more work needs to be done to better understand the relationship between the data and the outcomes and to answer the question posed by John Blake at the UUK conference around knowing <\/span>how<\/span><\/i> and <\/span>why<\/span><\/i> certain interventions are proving successful.<\/span>\u00a0<\/span>\r\n\r\nA deep dive into student engagement data has the potential to reveal a range of previously unknown insights about the student cohort, including insights on the relationship between personal\/demographic characteristics, engagement and student outcomes. The determination of \u2018risk\u2019 within StREAM is based solely on what students do i.e. how they engage with their studies, rather than on who they are, a decision based on our key principals around ethical and transparent use of data. However, triangulation with those personal and demographic insights provide valuable context when it comes to getting to know your students and agreeing how best to support each and every individual.\u00a0<\/span>\r\n\r\nGathering data throughout the student journey allows universities to create a complex picture of what consists of their student population, and the unique requirements of individuals to access support and ultimately, succeed.\u00a0<\/span>\u00a0<\/span>\r\n\r\nUsing this readily-available data can help universities ensure that their support services are relevant and accessible to all student cohorts and encourage widespread student participation, thus nurturing a truly accessible culture and learning environment for all.<\/span>\u00a0<\/span>\r\n\r\n\u00a0<\/span>\r\n\r\nThe importance of evaluation<\/h2>\n
The need for evidence<\/h2>\n
Why student engagement data matters<\/h2>\n
Impact for individual students<\/h2>\n
Cohort-level impact<\/h2>\n
Evidential standards<\/h2>\n
\n
Why we think student engagement data can help<\/h2>\n
\n
So what?<\/h2>\n
\r\n \t
\r\n \t