The online platform X has caused significant outrage after users utilized its AI chatbot Grok to change photos of women by removing their garments.
Multiple cases have surfaced where the technology has been employed to virtually strip women to render them in swimwear non-consensually, as well as to place them in inappropriate situations.
Developer Response and Regulatory Stance
XAI, the company responsible for Grok, offered no comment to questions, except for an automatically-generated message stating "traditional news outlets lies".
A government official stated that new laws is being introduced to outlaw image stripping technology. Under a new law, anyone who supplied such tools would "receive a prison sentence and significant penalties".
"X or Grok could prevent these types of misuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "assess the danger" of people viewing unlawful material on their networks, but stopped short of confirming whether it was currently investigating X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some additional functions—which responds to X users' queries when they reference it in a post.
It is frequently used to give commentary or more background to other users' comments, but people on X are also able to alter an posted image through its AI image editing feature.
History of Criticism and Rules
The chatbot has been condemned for permitting users to generate images and videos with explicit content and sexualised material, and it was in the past accused of creating graphic material of public figures.
"The platform has been permitting the production and distribution of these images for months without taking any measures and we have yet to see any intervention by regulators," the expert added.
XAI's own acceptable use policy prohibits "portraying images of persons in a pornographic way".
In a statement, Ofcom said it was against the law to "generate or share deepfakes or illegal content" and clarified this covered explicit AI images made with AI.
It said platforms such as X were mandated to take "necessary actions" to "lessen the danger" of users viewing unlawful content on their platforms and to take down it promptly when they learn of it.
