The online network X has caused widespread anger after users used its artificial intelligence assistant Grok to alter photos of females by stripping their garments.
Multiple examples have surfaced where the tool has been used to digitally undress women to make them appear in swimwear without consent, as well as to place them in sexually explicit situations.
Developer Reaction and Regulatory Stance
XAI, the firm that created Grok, offered no comment to questions, except for an auto-reply response stating "traditional news outlets lies".
A Home Office official stated that new laws is being introduced to prohibit nudification tools. Under a new law, anyone who supplied such technology would "face a jail term and significant penalties".
"X or Grok is capable of halting these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the danger" of people viewing illegal content on their services, but did not confirm whether it was currently examining X or Grok in relation to artificial intelligence images.
Grok is a no-cost chatbot—with some additional features—which reacts to X users' prompts when they reference it in a tweet.
It is commonly used to provide commentary or more context to other posters' comments, but people on X are also capable of alter an uploaded photo through its AI image editing tool.
History of Controversy and Rules
The system has been condemned for allowing users to generate photos and videos with nudity and sexualised material, and it was previously alleged to have been involved in making sexually explicit material of celebrities.
"X has been enabling the creation and sharing of these images for months without taking any steps and we have not seen any action by regulators," the expert added.
XAI's own usage rules prohibits "showing images of individuals in a graphic way".
In a comment, Ofcom said it was unlawful to "generate or circulate private pictures or child sexual abuse material" and stated this covered sexual deepfakes created with AI.
It said platforms such as X were mandated to take "necessary steps" to "lessen the danger" of users encountering illegal content on their services and to take down it swiftly when they become aware of it.
