The online platform X has provoked significant outrage after users employed its AI chatbot Grok to alter pictures of women by taking off their clothing.
Numerous cases have surfaced where the tool has been employed to virtually strip women to show them in swimwear non-consensually, as well as to place them in sexualized contexts.
Company Response and Legal Stance
XAI, the organization behind Grok, did not respond to questions, other than an standard message stating "legacy media lies".
A government representative stated that new laws is being brought forward to outlaw nudification technology. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a prison sentence and significant penalties".
"X or Grok could prevent these types of abuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the danger" of people seeing unlawful material on their services, but stopped short of confirming whether it was presently investigating X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some additional functions—which answers X users' requests when they tag it in a tweet.
It is commonly used to offer reactions or further context to other posters' remarks, but people on X are also capable of modify an uploaded image through its AI image editing function.
Background of Controversy and Policy
The chatbot has been criticised for allowing users to generate photos and footage with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in making inappropriate material of celebrities.
"The platform has been permitting the creation and sharing of these images for months without taking any measures and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own usage rules bans "depicting images of people in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "generate or distribute deepfakes or child sexual abuse material" and clarified this applied to explicit AI images made with AI.
It said services such as X were mandated to take "suitable steps" to "reduce the danger" of users encountering unlawful content on their platforms and to remove it swiftly when they are notified of it.
