Elon Musk's online platform X has sparked significant backlash after people utilized its artificial intelligence assistant Grok to manipulate pictures of women by stripping their garments.
Several examples have emerged where the technology has been employed to digitally undress women to make them appear in swimwear non-consensually, as well as to place them in sexualized situations.
Developer Response and Legal Stance
XAI, the firm responsible for Grok, did not respond to questions, aside from an standard response stating "legacy media lies".
A government official stated that legislation is being brought forward to outlaw nudification technology. Under a upcoming law, anyone who supplied such technology would "be subject to a jail term and substantial fines".
"X or Grok could prevent these types of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said tech firms must "evaluate the risk" of people viewing unlawful material on their services, but stopped short of confirming whether it was presently investigating X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some premium features—which reacts to X users' prompts when they tag it in a message.
It is often used to offer reactions or more context to other posters' remarks, but people on X are also can alter an posted photo through its AI image editing feature.
History of Controversy and Rules
The chatbot has been criticised for permitting users to generate images and clips with nudity and sexually suggestive content, and it was previously alleged to have been involved in making inappropriate content of celebrities.
"X has been permitting the generation and spread of these images for months without taking any steps and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting images of persons in a pornographic manner".
In a comment, Ofcom said it was illegal to "make or distribute deepfakes or illegal content" and confirmed this covered explicit AI images made with AI.
It said networks such as X were required to take "appropriate steps" to "reduce the risk" of users viewing illegal material on their platforms and to delete it promptly when they are notified of it.
