The social media network X has provoked significant backlash after individuals used its artificial intelligence assistant Grok to manipulate images of females by taking off their garments.
Several instances have surfaced where the technology has been used to remove clothing from women to render them in swimwear without consent, as well as to place them in sexualized contexts.
Developer Reaction and Legal Stance
XAI, the firm responsible for Grok, did not respond to inquiries, except for an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit nudification software. Under a upcoming criminal offence, anyone who provided such tools would "face a jail term and heavy penalties".
"X or Grok is capable of halting these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said tech firms must "assess the danger" of people viewing illegal content on their networks, but stopped short of confirming whether it was currently examining X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost chatbot—with some additional functions—which reacts to X users' prompts when they mention it in a tweet.
It is commonly used to give commentary or additional background to other users' comments, but people on X are also capable of edit an uploaded photo through its AI image editing tool.
History of Controversy and Policy
The tool has been condemned for allowing users to create images and clips with nudity and sexually suggestive material, and it was earlier alleged to have been involved in generating sexually explicit content of celebrities.
"The platform has been enabling the production and sharing of these images for months without taking any action and we have yet to see any challenge by regulators," the expert added.
XAI's own terms of service prohibits "showing images of people in a graphic manner".
In a comment, Ofcom said it was unlawful to "generate or circulate private pictures or child sexual abuse material" and confirmed this applied to explicit AI images made with AI.
It said networks such as X were mandated to take "suitable measures" to "minimize the risk" of users coming across unlawful material on their platforms and to remove it swiftly when they become aware of it.
