The online network X has provoked widespread backlash after users utilized its AI chatbot Grok to change images of women by taking off their attire.
Several instances have surfaced where the tool has been used to remove clothing from female subjects to show them in swimwear without permission, as well as to put them in sexualized contexts.
Company Reaction and Regulatory Position
XAI, the organization that created Grok, did not respond to inquiries, aside from an automatically-generated response stating "traditional news outlets lies".
A government spokesperson stated that legislation is being brought forward to outlaw image stripping tools. Under a proposed criminal offence, anyone who provided such technology would "receive a jail term and heavy fines".
"X or Grok has the ability to stop these types of misuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said tech firms must "evaluate the risk" of people seeing illegal content on their platforms, but stopped short of confirming whether it was at this time looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free AI assistant—with some paid-for features—which answers X users' prompts when they reference it in a tweet.
It is frequently used to offer commentary or additional context to other users' comments, but people on X are also can edit an posted image through its AI image editing function.
History of Criticism and Rules
The chatbot has been faulted for enabling users to create photos and clips with nudity and sexualised material, and it was previously alleged to have been involved in making inappropriate content of public figures.
"The platform has been allowing the production and distribution of these pictures for months without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "depicting images of persons in a pornographic manner".
In a statement, Ofcom said it was unlawful to "make or circulate non-consensual intimate images or illegal content" and clarified this included explicit AI images created with AI.
It said platforms such as X were required to take "appropriate actions" to "minimize the risk" of users encountering illegal material on their platforms and to delete it quickly when they are notified of it.
