Elon Musk's social media platform X has provoked widespread outrage after people used its artificial intelligence assistant Grok to manipulate pictures of females by stripping their clothing.
Numerous instances have surfaced where the system has been employed to virtually strip female subjects to make them appear in swimwear non-consensually, as well as to insert them in sexually explicit scenarios.
Developer Reaction and Regulatory Stance
XAI, the firm that created Grok, did not respond to questions, aside from an auto-reply message stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to outlaw nudification software. Under a new law, anyone who distributed such tools would "receive a jail term and significant fines".
"X or Grok is capable of halting these types of misuse if they desired to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said technology companies must "assess the potential" of people viewing illegal content on their services, but did not specify whether it was presently examining X or Grok in relation to AI-generated pictures.
Grok is a free artificial intelligence—with some paid-for functions—which reacts to X users' prompts when they tag it in a tweet.
It is commonly used to provide commentary or further background to other posters' remarks, but people on X are also able to modify an posted photo through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The chatbot has been criticised for allowing users to create pictures and footage with nudity and sexually suggestive material, and it was in the past alleged to have been involved in making inappropriate content of celebrities.
"X has been enabling the production and sharing of these pictures for months without taking any action and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy bans "depicting images of individuals in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "make or distribute non-consensual intimate images or child sexual abuse material" and stated this applied to sexual deepfakes generated with AI.
It said networks such as X were obligated to take "suitable measures" to "minimize the danger" of users viewing illegal content on their services and to remove it swiftly when they become aware of it.
