The online platform X has sparked significant anger after individuals utilized its AI chatbot Grok to alter pictures of females by taking off their clothing.
Several cases have emerged where the system has been used to digitally undress female subjects to show them in swimwear without permission, as well as to insert them in sexualized scenarios.
Developer Response and Regulatory Stance
XAI, the organization that created Grok, did not respond to requests for comment, except for an automatically-generated message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being introduced to outlaw image stripping tools. Under a proposed criminal offence, anyone who provided such tools would "be subject to a prison sentence and heavy fines".
"X or Grok has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said technology companies must "assess the risk" of people seeing unlawful material on their services, but did not specify whether it was at this time investigating X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some additional functions—which reacts to X users' queries when they reference it in a post.
It is frequently used to provide reactions or further background to other posters' remarks, but people on X are also capable of modify an posted image through its artificial intelligence picture modification function.
Background of Criticism and Policy
The tool has been faulted for permitting users to create images and clips with nudity and sexualised content, and it was in the past accused of making inappropriate content of public figures.
"The platform has been permitting the creation and distribution of these pictures for months without taking any action and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of people in a sexually explicit way".
In a comment, Ofcom said it was unlawful to "create or share deepfakes or child sexual abuse material" and clarified this covered explicit AI images created with AI.
It said networks such as X were required to take "necessary actions" to "reduce the danger" of users coming across unlawful material on their platforms and to delete it promptly when they learn of it.
