The online network X has sparked significant anger after people employed its AI chatbot Grok to alter photos of women by taking off their garments.
Several instances have been seen where the system has been used to virtually strip women to show them in swimwear without permission, as well as to insert them in sexually explicit contexts.
Developer Response and Regulatory Position
XAI, the organization responsible for Grok, offered no comment to requests for comment, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office official stated that legislation is being introduced to outlaw nudification software. Under a upcoming law, anyone who distributed such tools would "be subject to a jail term and substantial fines".
"X or Grok could prevent these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the potential" of people seeing illegal content on their platforms, but did not specify whether it was presently looking into X or Grok in regarding artificial intelligence images.
Grok is a no-cost chatbot—with some additional features—which responds to X users' requests when they reference it in a message.
It is frequently used to give reactions or further background to other posters' remarks, but people on X are also capable of alter an uploaded photo through its artificial intelligence picture modification function.
History of Controversy and Rules
The chatbot has been condemned for enabling users to create images and videos with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in making graphic material of celebrities.
"X has been permitting the creation and spread of these images for a long time without taking any measures and we have yet to see any intervention by regulators," the expert added.
XAI's own terms of service prohibits "showing images of persons in a pornographic fashion".
In a statement, Ofcom said it was against the law to "generate or share deepfakes or illegal content" and stated this included sexual deepfakes created with AI.
It said services such as X were required to take "appropriate steps" to "reduce the risk" of users coming across illegal material on their platforms and to delete it quickly when they are notified of it.
