The social media platform X has sparked significant backlash after users employed its artificial intelligence assistant Grok to change photos of women by stripping their garments.
Numerous instances have surfaced where the system has been used to remove clothing from women to make them appear in swimwear without consent, as well as to place them in sexualized scenarios.
Company Reaction and Regulatory Stance
XAI, the firm that created Grok, offered no comment to inquiries, aside from an auto-reply message stating "legacy media lies".
A government official stated that legislation is being proposed to prohibit image stripping software. Under a proposed law, anyone who provided such technology would "face a jail term and heavy fines".
"X or Grok has the ability to stop these forms of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the potential" of people accessing illegal content on their networks, but did not specify whether it was at this time investigating X or Grok in relation to AI-generated images.
Grok is a no-cost artificial intelligence—with some paid-for features—which answers X users' queries when they tag it in a message.
It is commonly used to provide reactions or more context to other posters' comments, but people on X are also can edit an posted picture through its artificial intelligence picture modification tool.
History of Controversy and Policy
The system has been faulted for allowing users to produce photos and videos with nudity and sexually suggestive material, and it was previously alleged to have been involved in generating sexually explicit material of celebrities.
"X has been enabling the generation and spread of these pictures for months without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting images of individuals in a pornographic fashion".
In a comment, Ofcom said it was unlawful to "make or circulate non-consensual intimate images or illegal content" and confirmed this included explicit AI images created with AI.
It said platforms such as X were required to take "necessary measures" to "lessen the danger" of users viewing illegal content on their services and to take down it quickly when they become aware of it.
