ChatGPT and the future of university assessments

ChatGPT is sending shockwaves through tertiary education and it's difficult for some university lecturers, used to doing things in a certain way, to embrace the rapid change associated with such technologies. Yet change is occurring at pace, and many of us in universities are relishing the opportunities such tools enable.

ChatGPT has garnered considerable media attention in the past few months for its ability to answer questions, provide advice on almost any topic in fluent, well-written English, write computer code and perform various other tasks.

The chatbot, launched in November, has been tested using a broad range of exam questions, including law, medical and business school exams. It passed those exams. Some of the answers provided by ChatGPT are nothing short of magic, and I have seen experts rendered speechless by them. Yet, these uncanny answers were pure luck. ChatGPT does not know whether an answer is correct; it simply predicts the solution based on its massive dataset. Therefore, many answers are not 100 percent accurate and can even be spectacularly wrong. A human is needed to determine the accuracy of its answers.

The reaction of universities to ChatGPT and other similar artificial intelligence (AI) tools has been mixed, falling into three main types: prevention, banning, and embracing.

First, to prevent their use, some universities are falling back to in-person pen and paper exams. However, tests and exams have never been ideal assessment methods. They don't indicate whether someone can work well in teams or present and communicate information orally, and they disadvantage those with often debilitating exam anxiety and so on. Indeed, to accommodate these limitations, many courses have reduced the percentage of course marks given out for tests and exams.

In addition, preventing the use of ChatGPT would work only if all of a course's assessments were for in-person work. To ensure no student could use ChatGPT would require increasing the percentage of marks for old-school tests and exams, which would be a retrograde step.
Second, some tertiary providers have explored banning ChatGPT and other AI tools, and the use of AI-detection tools. These AI detection tools are not 100 percent accurate and can be worked around. My concern is that students will spend more time attempting to circumvent the system than learning the content.

Banning or preventing the use of AI tools for all, or most assessments is counterproductive. People will not, for the foreseeable future, be in competition with AI. Instead, they will be competing with people who are adept and skilled at using such tools. Indeed, people unable to use AI tools may become unemployable in many professional settings as they will be too inefficient and slow.

The key to successfully integrating AI into education lies in understanding that AI tools are not a replacement for human expertise but rather that they are tools that can augment and enhance it.

Universities need to teach students how to use these tools effectively, to provide training and guidance to enhance their learning and prepare them for the workforce.

We have adapted to new tools in the past. For example, the fears that electronic spreadsheets would put accountants out of work did not materialise as the accounting profession pivoted. Similarly, AI tools are forcing a much-needed opportunity to reimagine the role of education in the 21st century.

So where does this leave us with the vexed question of assessment? How do we assess students' knowledge? For most courses, some element of in-person evaluation, whether written, oral, or both, is necessary. The remaining assessments require rethinking, and what may work for one discipline or course may not work for others.

One idea is that instead of providing a question to which the student writes an answer (the traditional approach), both the question and answer could be given. The students could critique the question and answer and explain what they think is correct or incorrect and why.

Alternatively, a student could be assessed on the nature and quality of the prompts they ask an AI tool. This may increase the time required for marking, but it will assist the students' skills with using the tools and provide a good way of assessing their knowledge of the subject matter at hand.  

As with most technology, the challenge is not the technology itself but rather our human emotions, experience and reaction to it.