Listen to the article
The British government has intensified its crackdown on AI-generated sexual abuse, confirming a possible ban on Elon Musk’s social media platform X amid a widening investigation into illegal content. The platform could face hefty fines as regulators examine whether it violated the UK’s Online Safety Act.
The formal investigation by Ofcom, the UK communications regulator, follows reports that X’s built-in AI chatbot, Grok, was used to generate and share sexualized deepfake images of women and children. Grok was launched in 2023, but its image generator feature, Grok Imagine, was added in 2025 with a specific mode to generate adult content.
Secretary of State for Science, Innovation and Technology Liz Kendall described the misuse of the AI tool as “deeply disturbing” and promised swift action.
“The content which has circulated on X is vile. It is not just an affront to decent society, it is illegal,” Kendall said in a statement. “No woman or child should live in fear of having their image sexually manipulated by technology.”
Kendall emphasized that sharing or threatening to share deepfake intimate images without consent, including images of people in their underwear, constitutes a criminal offense under UK law. She welcomed Ofcom’s urgency in launching a formal investigation, stressing that victims deserve quick resolution.
The regulatory body has significant enforcement powers at its disposal. Kendall warned that Ofcom can issue “fines worth millions of dollars” or penalties of up to 10% of a company’s qualifying worldwide revenue. However, she noted that X doesn’t need to wait for the investigation’s conclusion to take corrective action.
“They can choose to act sooner to ensure this abhorrent and illegal material cannot be shared on their platform,” she warned.
Ofcom reported it had “urgently made contact” with X on January 5, demanding explanations about steps being taken to protect UK users and setting a response deadline of January 9. While xAI, another company founded by Musk, responded, Ofcom decided to open a formal investigation after reviewing the available evidence.
“Reports of Grok being used to create and share illegal non-consensual intimate images and child sexual abuse material on X have been deeply concerning,” Ofcom said in a statement. “Platforms must protect people in the UK from content that’s illegal in the UK, and we won’t hesitate to investigate where we suspect companies are failing in their duties, especially where there’s a risk of harm to children.”
The chatbot had previously acknowledged in a public post on X that it generated and shared an AI image depicting two young girls in sexualized attire, calling the incident a “failure in safeguards” and apologizing for the harm caused. Amid mounting criticism, Grok confirmed it has begun restricting some image-generation and editing features to paying subscribers to prevent further misuse.
Musk responded to the investigation by accusing the UK government of “fascism” for “arresting thousands of people for social media posts.” The billionaire was responding to a post claiming the country arrests more people for social media posts than “any other country on earth.”
Alongside the Ofcom investigation, the UK government announced that legislation criminalizing the creation of non-consensual intimate images generated by AI would come into force this week. Kendall emphasized that responsibility extends beyond individuals to the platforms hosting such material.
“The platforms that host such material must be held accountable, including X,” she said, adding that she is “prepared to go further” if they do not act appropriately.
The UK Prime Minister’s office confirmed that the government’s presence on the platform is “under review” and that “all options are on the table,” signaling potential for more severe actions against the platform.
International reaction has been swift, with Malaysia and Indonesia becoming the first countries to block Grok entirely, according to reports from the Associated Press.
This case represents one of the most significant regulatory challenges for AI-powered social media tools since the UK implemented its Online Safety Act, highlighting growing global concerns about AI technology’s potential for abuse and the responsibility of platforms to prevent illegal content.
Fact Checker
Verify the accuracy of this article using The Disinformation Commission analysis and real-time sources.
8 Comments
While AI and digital innovation can bring many benefits, the misuse of these tools to violate privacy and create abusive content is unacceptable. I’m glad to see the UK taking strong action to hold platforms accountable and prevent further harm.
This is a complex issue that highlights the need for robust regulation and oversight of emerging technologies. The government must strike a balance between innovation and protecting vulnerable individuals from exploitation.
Agreed. The responsible development and deployment of AI is critical to mitigate these kinds of risks. Proactive policymaking is essential.
The UK is right to investigate this issue thoroughly and pursue legal action if necessary. The creation and distribution of nonconsensual deepfake intimate images is a heinous abuse that cannot be tolerated. Protecting vulnerable individuals should be the top priority.
Absolutely. The government must send a clear message that this kind of exploitation will not be permitted, regardless of the platform or technology involved.
This is a concerning development regarding the misuse of AI technology. The government must take strong action to protect individuals from the threat of deepfake exploitation and abuse. Strict regulations and enforcement are crucial to upholding digital safety and privacy rights.
I agree, the potential for harm from deepfake technology is alarming. Platforms like X need robust safeguards and accountability measures to prevent such egregious violations.
Deepfake technology poses significant risks, and I’m glad the UK is taking this issue seriously. Platforms like X must be held accountable for any failures to safeguard against the misuse of their tools.