A high school student in Sydney, Australia, faces a police investigation and disciplinary action after allegedly using artificial intelligence to create deepfake pornographic images of female classmates, marking one of Australia's first major AI-related incidents in an educational setting.

A deepfake is a fake image created using deep learning algorithms. They are very realistic, and are difficult to distinguish from the real photo depending on the quality. They are usually NSFW, but don't have to be.

The student, who was not named by police, allegedly copied images from social media accounts and school events to create AI-powered candid photos of several female students. He then distributed the content through fake social media profiles, according to emails school officials sent to parents. Local media.

New South Wales Police launched an investigation following reports of "indecent images being produced and distributed online", according to the British Daily Mail. The Guardian. Police are working with both the Electronic Safety Commissioner and the Department for Education to address the incident.

“The school has learned that a 12th grade student has used artificial intelligence to create a profile that looks like your daughters and others,” read the school’s email to affected parents, according to local media. “Unfortunately, innocent images from social media and school events were used.”

This is it It is not an isolated case. on 530 thousand teenagers In the UK, they encountered fake nude photos, according to a study by the London-based non-profit Internet Matters. last year, Local news In Seattle, Washington, a local teen reportedly shared fake photos of his classmates on social media. The year before that, a group of female students in New Jersey Found That their classmates used photos of them fully clothed as a base to create NSFW fake photos of them.

The problem appears to be spreading because generative AI makes it easier to create basically anything. A reconnaissance The Center for Democracy and Technology found that 50% of U.S. high school teachers know of at least one instance in which someone from their school was depicted in explicit AI-generated content.

NSW Education Minister Prue Carr described the incident as "abhorrent" during a press conference on Thursday. “There will be disciplinary measures against the student,” Carr said, praising the school’s vice principal for his quick action in dealing with the situation.

The Ministry of Education stressed its zero-tolerance stance towards such behaviour. “Our top priority is ensuring our students feel safe,” a department spokesperson said. The Guardian. The school provides ongoing health support to affected students.

The incident follows a similar case in Victoria last June, where a 17-year-old student allegedly created AI-generated explicit photos of about 50 female classmates. This student received a warning from the police after an investigation.

Legal experts point to loopholes in current legislation to deal with explicit content generated by artificial intelligence. The Australian Senate passed legislation in August 2023 targeting non-consensual pornography, while advocates in the United States are pushing for a law to prevent deepfakes of intimate images.

Other legal initiatives to address this issue include Deepfakes Accountability Act,Singapore Anti-deepfake legislationand I have a law.