🔗 Share this article

The online platform X has sparked widespread outrage after people employed its AI chatbot Grok to change pictures of women by stripping their clothing.

Several instances have been seen where the tool has been employed to remove clothing from female subjects to make them appear in bikinis without permission, as well as to insert them in sexually explicit situations.

Developer Response and Regulatory Stance

XAI, the firm that created Grok, offered no comment to questions, except for an automatically-generated message stating "traditional news outlets lies".

A government representative stated that legislation is being introduced to outlaw nudification software. Under a proposed law, anyone who distributed such technology would "be subject to a jail term and heavy penalties".

"The platform has the ability to stop these types of misuse if they chose to," said Clare McGlynn, adding they "operate with impunity."

Regulator Stance and Platform Capabilities

The communications watchdog Ofcom said tech firms must "consider the danger" of people viewing unlawful material on their networks, but did not specify whether it was currently examining X or Grok in regarding AI-generated pictures.

Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' queries when they reference it in a post.

It is frequently used to give reactions or additional background to other users' comments, but people on X are also able to alter an posted photo through its AI image editing tool.

History of Criticism and Policy

The chatbot has been criticised for permitting users to produce photos and videos with explicit content and sexualised content, and it was previously alleged to have been involved in creating graphic content of public figures.

"The platform has been enabling the creation and spread of these images for a long time without taking any measures and we have not seen any challenge by regulators," the expert added.

XAI's own terms of service prohibits "showing images of people in a pornographic fashion".

In a statement, Ofcom said it was against the law to "generate or share deepfakes or child sexual abuse material" and stated this covered explicit AI images created with AI.

It said platforms such as X were mandated to take "suitable measures" to "minimize the risk" of users coming across illegal content on their platforms and to delete it swiftly when they are notified of it.