Elon Musk's social media network X has provoked widespread outrage after individuals employed its artificial intelligence assistant Grok to manipulate photos of women by removing their attire.
Several cases have surfaced where the tool has been used to virtually strip women to render them in bikinis without consent, as well as to place them in sexually explicit contexts.
Developer Response and Legal Position
XAI, the firm responsible for Grok, did not respond to questions, other than an automatically-generated response stating "legacy media lies".
A Home Office official stated that legislation is being proposed to prohibit nudification technology. Under a proposed criminal offence, anyone who distributed such technology would "be subject to a jail term and substantial fines".
"The platform could prevent these forms of abuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said tech firms must "evaluate the risk" of people accessing illegal content on their services, but stopped short of confirming whether it was at this time investigating X or Grok in concerning artificial intelligence images.
Grok is a free chatbot—with some premium features—which reacts to X users' requests when they mention it in a post.
It is often used to give reactions or more context to other posters' remarks, but people on X are also able to edit an uploaded photo through its AI image editing tool.
History of Criticism and Policy
The tool has been criticised for allowing users to produce pictures and footage with explicit content and sexualised content, and it was previously accused of generating inappropriate content of public figures.
"X has been enabling the creation and distribution of these images for a long time without taking any steps and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of persons in a graphic way".
In a statement, Ofcom said it was unlawful to "make or distribute deepfakes or child sexual abuse material" and clarified this applied to sexual deepfakes made with AI.
It said services such as X were obligated to take "suitable measures" to "reduce the danger" of users coming across unlawful material on their services and to take down it swiftly when they learn of it.
