The social media platform X has provoked widespread outrage after people used its AI chatbot Grok to manipulate images of females by stripping their clothing.
Several instances have emerged where the system has been used to remove clothing from women to render them in bikinis without consent, as well as to put them in sexually explicit situations.
Company Response and Legal Position
XAI, the organization that created Grok, did not respond to questions, other than an auto-reply message stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to prohibit image stripping technology. Under a upcoming law, anyone who distributed such tools would "face a jail term and significant penalties".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Functionality
The regulator Ofcom said technology companies must "consider the danger" of people accessing illegal content on their platforms, but did not specify whether it was currently investigating X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some paid-for functions—which answers X users' queries when they mention it in a message.
It is frequently used to provide reactions or further background to other users' comments, but people on X are also capable of modify an uploaded photo through its artificial intelligence picture modification function.
History of Controversy and Policy
The chatbot has been criticised for enabling users to produce pictures and footage with explicit content and sexualised material, and it was earlier accused of creating sexually explicit material of public figures.
"X has been permitting the creation and spread of these pictures for a long time without taking any steps and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own terms of service bans "showing likenesses of individuals in a graphic way".
In a comment, Ofcom said it was unlawful to "make or share private pictures or child sexual abuse material" and confirmed this covered sexual deepfakes made with AI.
It said platforms such as X were required to take "necessary steps" to "lessen the risk" of users encountering illegal content on their platforms and to remove it quickly when they are notified of it.
