What is the future for student assessment in the light of AI/ChatGPT?
ChatGPT is a hot topic in higher education, with concerns about how it will impact on academic integrity and student assessment. We spoke to eminent computer scientist and former Vice-Chancellor and Principal of the University of Edinburgh Sir Tim O’Shea about how universities should be approaching ChatGPT.
What does ChatGPT mean for the future of assessment in higher education?
ChatGPT represents a major transition point in higher education. The assessment questions you might be using currently can be addressed with varying degrees of effectiveness by ChatGPT. However, this is not new. Assessment in higher education has been getting progressively more insecure as a consequence of students’ ability to use a range of online resources. They can do Google searches. They can use open educational resources, databases and digital textbooks. Pre-trained transformers such as ChatGPT are just an enhanced way of text generation from online resources.
A simple comparison might be to the pocket calculator. When they became widespread, there were people who wanted to ban them from schools to ensure children could do calculations. That soon turned out to be nonsensical; pocket calculators were everywhere and even professionals were using them. The key thing was to build from them – devising more sophisticated mathematical exercises that involved the intelligent use of a pocket calculator.
I think the same will apply to ChatGPT. It doesn’t make sense to ban it in an age when you can carry a computer on your wrist. Universities will have to accommodate it. Academics will have to stop setting assignments that can be answered easily by ChatGPT. There’s no harm in that because in the age of the Internet and big tech, those aren’t helpful questions anyway. The key is not to try to steer around it, but to take advantage of it. I see it as an opportunity and a positive progression.
How can ChatGPT be used in higher education?
I think there is tremendous potential to engage students. ChatGPT will lead us to a more sophisticated style of assessment. It allows us to do education at a higher level, to ask higher order questions, to encourage students to reflect. One of the strengths of ChatGPT is that you don’t know if what it’s telling you is true. We can use ChatGPT to enable students to think critically.
It can help students learn how to verify online material using processes like triangulation. ChatGPT looks at lots of online data sources. How does it combine evidence? How important is it if 100 articles on a topic say the same thing? Does it just take a single point of view from a single article? Time series analysis is another interesting use for ChatGPT. How does information on a topic change over time? What was the 2020 view of a topic? What was it in 2021? How is it different? Why did it change?
The next stage is helping students learn how to evaluate an answer. What are the reasons for believing this argument? What are the author’s motives? We have a growing issue with fake news. Lots of information online is made available with a deliberate intent to influence or deceive. For example, in elections we have bots tweeting opinions from bogus accounts. If one outcome of ChatGPT is that we develop a more sophisticated understanding of what a bot is and how it tries to influence us that would be a good thing.
What would your advice be to universities on how to approach ChatGPT?
My basic advice is don’t set students the sort of questions that ChatGPT can answer easily. Banning its use doesn’t seem to me to be a productive response. Don’t discourage it and don’t pass over it in silence. Insist that students are honest and transparent, acknowledging that they have used ChatGPT
As a starting point. I think building on it makes very good sense. Ask what useful things could we do with this? Put ChatGPT in the curriculum, where it is appropriate.
I think it’s a fabulous opportunity for group work. A group of students are interested in a topic. They pose a question to ChatGPT, then they look at the answer and evaluate it. The ChatGPT output isn’t a student output, but an additional resource for the students to use, just like a digital textbook. It gives students lots of opportunities for discussion and critical thinking. If ChatGPT encourages a more sceptical approach to information gleaned on the Internet that would be a very good thing.
