Recent revelations have unveiled serious concerns regarding the reliability of artificial intelligence (AI) tools used by universities to detect academic misconduct. There are serious allegations that thousands of students were incorrectly identified as cheaters. These allegations were based on imperfect AI-generated reports, raising red flags about the way we uphold academic integrity.
Among Australian universities, the Australian Catholic University (ACU) has been at the forefront of these changes. Yet today, it finds itself on defense for its treatment of almost 6,000 reported instances of supposed academic misconduct in just 2024. According to those reports, about 90 percent of these referrals were a direct result of AI use. This trend is further evidence of the growing dependency on technology in education.
Said deputy Vice-Chancellor Tania Broadley, these numbers are “significantly inflated.” As Lathrop explained, trusting AI detection tools—such as Turnitin’s new AI detector—will fail to accurately flag true instances of misconduct.
The AI Detection Dilemma
Further, the recent introduction of AI detection tools has changed everything when it comes to universities’ efforts to prevent plagiarism and uphold academic integrity. The more we rely on AI-generated content, the greater the risk of harmful inaccuracies that can come from using these tools. Broadley pointed out that roughly one in four of all referrals were cleared after investigation. This was all the more so for cases that were based purely on Turnitin’s AI detection tool.
For AI use cases beyond general data processing, these numbers shot up to 50% at least half of all confirmed breaches. This even extended to situations where AI had created a whole assignment, or students had accessed tools to paraphrase or translate material. This troubling trend raises critical concerns regarding the efficacy of existing guidelines. Perhaps most importantly, they do little to actually safeguard students from false allegations.
“In 2024, approximately half of all confirmed breaches involved the unauthorized or undisclosed use of AI. This included assignments generated by AI, AI-produced references, and the use of AI tools to paraphrase or translate content,” – Tania Broadley
These challenges to AI detection are perhaps best illustrated by her students’ experiences. One paramedic student shared a particularly alarming experience where their essay was marked as mostly written by AI.
“It’s AI detecting AI and almost my entire essay was lit up in blue — 84 percent of it supposedly written by AI,” – Paramedic Student
This raises significant concerns over how universities interpret these reports and the implications for students facing penalties based on flawed technology.
Student Experiences and Consequences
The impact of being wrongly accused can severely impact a student’s future in academia and beyond. Madeleine is a student at ACU, studying nursing. The fallout for her was extreme, when her academic transcript was stamped “results withheld” during a suspended investigation into supposed misconduct.
In a challenging phase of her studies, she found herself frantically applying for graduate positions, only to feel that the incomplete documentation negatively impacted her chances.
“Madeleine is convinced that incomplete document was part of the reason she was not offered a graduate position,” – source not specified.
Worries about what’s become a rubber-stamp process go far beyond specific projects. by Albert Kelly Students are questioning the current state of misconduct investigations and the failure to change that. These processes, as Professor Broadley recognized, have not always moved quickly enough to serve their interests.
“Investigations were not always as timely as they should have been, and significant improvements had been made in the past 12 months,” – Tania Broadley
This sentiment resonates with many students who feel they are caught in a system where they lack clarity regarding what constitutes acceptable use of AI.
Educational Initiatives and Future Directions
ACU has acknowledged the increasing role AI has played in academic spaces. To tackle this, they’ve developed new modules focused on the ethics of AI, available to both staff and students. Our hope is that this initiative helps develop a clearer picture of what high-quality practices look like in this rapidly changing field of educational technology.
UCLA education Professor Jeannie Suk Liu was more concerned about the impulse to punish cheating before it happens, than on what students should actually learn.
“We want to verify whether a student is learning, not whether they’re cheating,” – Professor Liu
The University of Sydney has taken a creative approach. For example, they’ve adopted a “two-lane” approach that permits certain lawful uses of AI in assessments. This cutting-edge framework aims to find the right balance between technology use and academic integrity.
Our universities are rising to meet this challenge. It is important that students and faculty engage in open conversations regarding how AI can and should be used in education.
“For staff and students … there is an opportunity to do the right thing, but it’s not always clear what that is,” – Leah Kaufmann
First, as this dialogue continues, institutions need to sharpen their focus. They must resist censorship and book-banning while continuing to protect academic integrity and the rights and learning opportunities of all students.
