The social media network X has sparked widespread outrage after individuals used its artificial intelligence assistant Grok to manipulate pictures of females by taking off their attire.
Numerous instances have surfaced where the technology has been employed to digitally undress female subjects to make them appear in swimwear without permission, as well as to place them in sexualized situations.
Developer Response and Legal Stance
XAI, the organization responsible for Grok, offered no comment to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to prohibit nudification technology. Under a new law, anyone who distributed such tools would "face a jail term and heavy penalties".
"X or Grok could prevent these types of misuse if they desired to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Tool Functionality
The regulator Ofcom said platforms must "evaluate the danger" of people viewing illegal content on their services, but stopped short of confirming whether it was presently looking into X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some premium features—which responds to X users' prompts when they mention it in a message.
It is frequently used to give reactions or additional context to other users' remarks, but people on X are also capable of edit an posted image through its AI image editing function.
Background of Controversy and Policy
The tool has been condemned for permitting users to generate images and footage with explicit content and sexualised content, and it was previously accused of creating graphic content of public figures.
"The platform has been enabling the production and distribution of these images for months without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "showing images of persons in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "create or circulate private pictures or child sexual abuse material" and clarified this covered sexual deepfakes made with AI.
It said networks such as X were required to take "appropriate measures" to "reduce the danger" of users viewing unlawful content on their platforms and to take down it quickly when they become aware of it.
