The social media platform X has provoked significant backlash after individuals used its AI chatbot Grok to change photos of women by removing their clothing.
Several examples have surfaced where the system has been employed to virtually strip women to make them appear in bikinis non-consensually, as well as to place them in sexually explicit situations.
Company Reaction and Legal Stance
XAI, the company that created Grok, remained silent to questions, aside from an standard message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being brought forward to ban nudification technology. Under a proposed law, anyone who provided such tools would "face a jail term and substantial penalties".
"X or Grok could prevent these types of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said tech firms must "evaluate the risk" of people seeing unlawful material on their networks, but did not confirm whether it was currently examining X or Grok in concerning AI-generated images.
Grok is a no-cost AI assistant—with some additional functions—which responds to X users' queries when they mention it in a post.
It is often used to give commentary or further background to other posters' remarks, but people on X are also able to modify an uploaded picture through its artificial intelligence picture modification feature.
Background of Criticism and Rules
The tool has been faulted for allowing users to create images and videos with explicit content and sexualised material, and it was earlier alleged to have been involved in generating inappropriate content of public figures.
"The platform has been enabling the production and sharing of these pictures for a long time without taking any steps and we have not seen any challenge by regulators," the expert added.
XAI's own terms of service prohibits "depicting likenesses of persons in a pornographic way".
In a statement, Ofcom said it was unlawful to "create or distribute deepfakes or child sexual abuse material" and stated this covered explicit AI images generated with AI.
It said services such as X were mandated to take "appropriate actions" to "lessen the risk" of users encountering illegal material on their platforms and to delete it quickly when they become aware of it.
