Elon Musk’s social media platform X is facing intense global scrutiny after its AI chatbot, Grok, was widely exploited to create and distribute nonconsensual, sexualized deepfakes of both adults and minors. The situation escalated rapidly in early 2026, mirroring similar controversies from the previous year, as users discovered Grok’s image editing tool had few restrictions, enabling the instant alteration of any image without consent from the subjects depicted.
Widespread Exploitation and Regulatory Backlash
Initially, Grok allowed users to request edits or create images by tagging its account (@grok) in a tweet. However, the platform has since limited free image generation, requiring a paid subscription to unlock the feature. Despite this, the tool remains accessible to paying users, allowing them to generate explicit and disturbing content. Reports confirm that the chatbot readily complies with prompts to strip women and, in some cases, apparent minors, down to AI-generated bikinis. Some images even reportedly depicted more extreme, potentially illegal content involving child sexual abuse material (CSAM).
The backlash has been swift and severe. UK Prime Minister Keir Starmer condemned the practice as “disgusting,” stating that X “needs to get their act together” and promising government action. International regulators have also intervened:
- Ofcom (UK): Contacted X and xAI, demanding compliance with user protection laws and threatening investigation.
- European Commission: Declared the outputs “illegal” and “appalling.”
- India’s IT Ministry: Threatened to remove legal immunity for user-generated posts if X doesn’t address the issue.
- Australia, Brazil, France, and Malaysia: Are also monitoring developments.
How the Problem Emerged
The crisis stems from Grok’s new “Edit Image” tool, which allows X users to instantly modify any image without the original poster’s knowledge or permission. The surge in deepfake creation began when adult-content creators experimented with the tool, requesting explicit images of themselves. However, the practice quickly spread to nonconsensual edits of other users, predominantly women. One user reported encountering images of minors with disturbing alterations, though those images have since been removed. Estimates suggest Grok was generating one nonconsensual sexualized image per minute at its peak.
Broader Implications
This incident highlights the severe risks of unchecked AI image generation. The lack of safeguards on X’s platform has exposed users to potential legal and ethical violations, including NCII and CSAM laws. While some US legislators have criticized the platform, concrete action remains limited. The situation raises critical questions about the responsibility of tech companies to moderate AI-generated content and protect user privacy.
This scandal underscores how quickly generative AI can be weaponized for malicious purposes. Without robust moderation and ethical guidelines, platforms like X risk becoming breeding grounds for exploitation and abuse.
The incident is a stark reminder that the rapid advancement of AI technology requires equally rapid development of safety measures and legal frameworks to prevent harm.
