The online platform X has sparked widespread outrage after people employed its AI chatbot Grok to manipulate photos of females by taking off their clothing.
Numerous examples have surfaced where the tool has been employed to digitally undress women to show them in bikinis without permission, as well as to place them in sexualized situations.
Developer Response and Legal Position
XAI, the firm responsible for Grok, did not respond to questions, except for an automatically-generated message stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to ban image stripping software. Under a new law, anyone who provided such technology would "receive a jail term and heavy fines".
"X or Grok has the ability to stop these forms of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "assess the potential" of people seeing illegal content on their services, but did not specify whether it was presently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some paid-for features—which responds to X users' prompts when they tag it in a tweet.
It is often used to give reactions or further context to other posters' comments, but people on X are also able to alter an posted photo through its AI image editing tool.
Background of Controversy and Policy
The system has been faulted for enabling users to generate images and footage with nudity and sexualised content, and it was in the past alleged to have been involved in generating graphic material of public figures.
"X has been permitting the production and sharing of these images for a long time without taking any action and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own usage rules bans "portraying images of persons in a pornographic manner".
In a statement, Ofcom said it was unlawful to "create or distribute private pictures or child sexual abuse material" and clarified this covered explicit AI images made with AI.
It said networks such as X were required to take "appropriate steps" to "lessen the risk" of users encountering illegal material on their services and to remove it swiftly when they are notified of it.
