Australian school communities are facing a concerning new wave of deepfake threats, leading Australia’s eSafety Commissioner Julie Inman Grant to issue timely warnings. In her recent letter to education ministers, she advised schools to immediately report such incidents to police and local authority safeguarding leads. She noted the importance of focusing on and adhering to state and territory laws regarding mandatory reporting.
Deepfake technology is a form of artificial intelligence. It has become infamous for being abused to harass marginalized people, especially young girls. The online safety regulator, eSafety, has classified the proliferation of deepfake technology as a “crisis affecting school communities across Australia.” Statistics show that four out of five reported deepfake occurrences are just that—to the horrific and exclusive targeting of young girls.
‘Nudify’ apps are popping up everywhere, using AI to transform images in less than 10 seconds. This unfortunate trend is one of the main drivers worsening the crisis. Inman Grant explained the ease with which these tools can be used, stating, “With just one photo, these apps can nudify the image with the power of AI in seconds.” The Australian eSafety Commission has noted a sharp increase in deepfake related complaints. Even this aggravating development might understate how big the problem is.
Asher Flynn, a criminology lecturer at Monash University, backed up these results. In a hopeful trend he connected the uptick in billing to a joyful – and even sorrowful – truth. Flynn described the increase of sexualized deepfake abuse as “intimidating, but not surprising.” He stressed that we repeatedly observe these patterns of response when technology-facilitated sexual violence occurs. He noted that there has been a normalization of creating deepfake content, saying, “We have seen a proliferation of ‘user-friendly’ deepfake creation tools emerging online.”
Inman Grant has been very proactive with police, app developers, and their respective hosting platforms to make changes and improve these concerns. She stressed the necessity for a multifaceted response that includes both legal measures and educational initiatives to combat deepfake incidents effectively. Legal norms need to be put in place to ensure accountability for all those involved in committing these acts.
She noted, “I’m calling on schools to report allegations of a criminal nature, including deepfake abuse of underage students, to police and to make sure their communities are aware that eSafety is on standby to remove this material quickly.” This week, universal mandatory standards go into effect. Applicable charges Companies found to be in violation of these standards may incur penalties of up to $49.5 million.
Even with these steps taken, Inman Grant told the Brookings Institution that deepfakes could still be underreported in this new law. “We suspect what is being reported to us is not the whole picture,” she stated. Whether it’s school leaders or representatives, there’s been an awareness of the increase in these incidents. This trend often happens when children receive unsupervised access to nudify apps and abuse them in school environments.
Shahriar Kaisar, an expert in technology ethics, emphasized the need for increased awareness among students regarding the ethical implications of using such technologies. He remarked, “The more important thing would be raising awareness and an ethical understanding of technology among school kids.” Kaisar discussed some of the most alarming trends coming out of these apps. Instead, they’re being used for humiliation, bullying, and sexual extortion among peers.
The explosion and availability of these nudify apps have changed the landscape of how our children engage with technology and digital content. According to Kaisar, “It has become quite readily available to everyone,” making it essential for educators and parents to foster discussions about the responsible use of technology.
Inman Grant once again brought home the message that we must get to the root causes of these tragedies. She explained a continuum of motivations for these abuses, from sexual thrill to prestige among one’s peers. “We are seeing a range of motivations for the incidents,” she explained.
It’s a developing story as schools continue to feel their way through these challenges even as awareness and regulatory pushback continues to build. The eSafety Commission remains committed to encouraging rapid response to individuals who use deepfake technology for nefarious purposes.
