The social media network X has caused major outrage after individuals utilized its AI chatbot Grok to change images of females by stripping their attire.
Numerous instances have been seen where the technology has been used to remove clothing from female subjects to render them in bikinis without permission, as well as to place them in sexualized situations.
Developer Response and Regulatory Position
XAI, the company responsible for Grok, did not respond to questions, other than an standard message stating "traditional news outlets lies".
A Home Office official stated that new laws is being introduced to prohibit image stripping software. Under a upcoming criminal offence, anyone who provided such tools would "be subject to a prison sentence and substantial fines".
"X or Grok has the ability to stop these types of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said platforms must "assess the danger" of people seeing illegal content on their networks, but stopped short of confirming whether it was presently examining X or Grok in regarding artificial intelligence pictures.
Grok is a free AI assistant—with some paid-for features—which responds to X users' prompts when they reference it in a message.
It is often used to provide reactions or further context to other posters' comments, but people on X are also capable of edit an uploaded image through its AI image editing tool.
History of Criticism and Policy
The system has been faulted for permitting users to generate images and videos with explicit content and sexualised content, and it was previously alleged to have been involved in creating sexually explicit content of celebrities.
"The platform has been enabling the creation and distribution of these pictures for a long time without taking any measures and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "showing likenesses of people in a pornographic manner".
In a comment, Ofcom said it was illegal to "make or circulate deepfakes or illegal content" and confirmed this applied to sexual deepfakes generated with AI.
It said platforms such as X were required to take "suitable steps" to "reduce the risk" of users viewing unlawful content on their services and to delete it swiftly when they learn of it.
