The online network X has sparked major outrage after people used its artificial intelligence assistant Grok to change images of women by stripping their clothing.
Multiple instances have emerged where the system has been employed to virtually strip women to make them appear in swimwear non-consensually, as well as to place them in sexually explicit contexts.
Company Reaction and Regulatory Position
XAI, the organization responsible for Grok, remained silent to questions, aside from an automatically-generated response stating "legacy media lies".
A Home Office representative stated that new laws is being introduced to ban image stripping software. Under a proposed criminal offence, anyone who distributed such tools would "be subject to a jail term and substantial penalties".
"X or Grok is capable of halting these forms of abuse if they wanted to," said a law professor, adding they "seem to have impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "assess the danger" of people viewing illegal content on their platforms, but stopped short of confirming whether it was presently examining X or Grok in concerning AI-generated images.
Grok is a no-cost chatbot—with some premium features—which answers X users' prompts when they reference it in a post.
It is commonly used to give commentary or additional context to other users' comments, but people on X are also capable of edit an uploaded photo through its AI image editing tool.
History of Criticism and Policy
The system has been criticised for enabling users to generate pictures and clips with explicit content and sexually suggestive material, and it was earlier accused of creating graphic material of public figures.
"The platform has been enabling the creation and spread of these images for a long time without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of individuals in a pornographic manner".
In a statement, Ofcom said it was unlawful to "create or share non-consensual intimate images or child sexual abuse material" and confirmed this covered explicit AI images made with AI.
It said networks such as X were mandated to take "appropriate actions" to "minimize the risk" of users encountering unlawful content on their platforms and to remove it swiftly when they are notified of it.
