In January 2026, Grok generated nearly 3 million sexualized images in 11 days, including thousands involving minors. The case triggered investigations in a dozen countries and influenced the European Parliament's vote on the Digital Omnibus. This case concretely illustrates what the AI Act prohibits.
In brief
- Grok generated 3 million sexualized deepfakes in 11 days in early January 2026, including thousands of minors.
- "Nudifier" systems have been directly prohibited by Article 5 of the AI Act since February 2, 2025.
- The European Parliament strengthened the prohibition of "nudifier apps" during the IMCO/LIBE vote on March 18, 2026.
- The European Commission ordered X to preserve all internal documents related to Grok until the end of 2026; numerous investigations have been opened.
- Any company using an AI image generator without safeguards faces the same risks, with transparency obligations starting in August 2026.
- The Grok case proves that the AI Act aims to concretely prevent such drift, far beyond theory.
January 2026 marks a turning point in the regulation of generative AI. In less than fifteen days, Grok—the image AI designed by xAI, Elon Musk's company—generated approximately 3 million sexualized images without consent, including those of public figures, politicians, and minors. This content spread massively on X (formerly Twitter), causing a global shockwave. Authorities in countries such as France, India, the UK, Australia, and Brazil immediately opened investigations. Malaysia and Indonesia blocked access to the platform. As for the European Commission, it demanded the preservation of all internal X documents until the end of 2026.
For compaia, this event is not just a technological news item. It concretely illustrates the type of drift that the **AI Act (EU Regulation 2024/1689)** aims to prevent and justifies the existence of these new obligations.
Grok Deepfakes: What Acts, What Rules?
The incriminating feature was terrifyingly simple: users submitted photos of real people to Grok so the AI could digitally "undress" them, place them in sexualized situations, or create explicit content from their images. Grok executed these requests almost systematically. A study revealed that the AI did not refuse any of 60 requests to create deceptive electoral images, whereas ChatGPT, Claude, or Gemini refused these requests in 72% to 97% of cases.
This technology is known as a **"nudifier" application**. From a regulatory perspective, its classification under the AI Act leaves no room for doubt.
Article 5: Prohibited Practices Since February 2025
**Article 5 of the AI Act** lists practices deemed to carry an unacceptable risk, which are formally prohibited in the European Union. These prohibitions have been in effect since **February 2, 2025**, regardless of the rest of the regulation's implementation schedule.
This targets: systems that exploit human vulnerability to cause harmful behavior, as well as those that infringe upon human dignity. A system generating sexualized content of real individuals without their consent falls squarely into this category.
Furthermore, the European Parliament strengthened this prohibition during the IMCO/LIBE vote on March 18, 2026, regarding the Digital Omnibus, by proposing to **explicitly include the prohibition of "nudifier apps"** in the list of prohibited practices. This measure does not create a new obligation but clarifies a prohibition already in force, made necessary by the Grok precedent.
Article 50: Mandatory Transparency on Deepfakes Starting August 2026
Beyond the outright ban, **Article 50(4)** of the AI Act imposes a transparency requirement on all generative AI systems: any audio or visual content created or modified by AI that is intended to resemble a real person must be **clearly labeled as synthetic**.
This obligation takes effect on **August 2, 2026**, and concerns:
- Deepfake videos depicting existing people
- AI-modified images of identifiable individuals
- Audio mimicking real voices
- Any visual creation presented as authentic
The Grok case demonstrates the necessity of this rule: without explicit labeling, such content circulates as if it were authentic, causing damage to the reputation, psychological health, and sometimes physical safety of the victims.
Why Does Grok Differ from Its Competitors?
The Grok case is not limited to a technical failure: it stems from a deliberate choice by the platform. While OpenAI, Anthropic, and Google have established strict filters—systematic rejection of manipulative requests, blocking images of real people, detailed usage policies—Grok chose a claimed "anti-censorship" approach.
Its usage policy, reduced to fewer than 350 words, shifted responsibility to the user. The consequence: where its competitors reject 72% to 97% of problematic requests, Grok accepted almost all of them.
> This is not a simple technical flaw, but a design decision. The AI Act specifically aims to prevent this by imposing risk management obligations from the design phase (Article 9).
This difference is essential for businesses: the AI Act is not limited to sanctioning abuses after the fact. It requires providers to **prevent drift from the design stage**—this is the principle of "safety by design" enshrined in Article 9 on risk management.
An Immediate and Coordinated Regulatory Response
One of the striking aspects of the Grok case is the speed and coordination of the authorities' reaction:
- **European Commission (January 8)**: Injunction to preserve all internal X documents related to Grok until the end of 2026.
- **Paris Public Prosecutor's Office (early January)**: Investigation opened for "manifestly illegal content," confirmed after a search of X's Paris offices in February (with Europol).
- **Ofcom UK (January 12)**: Investigation into X's compliance with user protection rules.
- **Malaysia and Indonesia (January 10)**: Temporary blocking of Grok.
- **Australia, Brazil, India**: Investigations and formal notices in the following weeks.
- **European Parliament (March 18)**: The IMCO/LIBE vote integrates the explicit prohibition of "nudifier apps" into the Digital Omnibus.
This rapid mobilization proves that the regulatory framework already existed: AI Act, DSA (Digital Services Act), and national legislation. What was lacking was systematic enforcement. The strengthening of the AI Office and national authorities aims precisely to fill this gap.
What Are the Consequences for Businesses?
Even if you are not xAI, the Grok case concerns you for several reasons:
1. Similar tools may be present in your organization
Many companies use AI image generation or editing applications—for marketing, creating product visuals, avatars, etc. If these tools can be misused to produce sexualized content of real people without consent, your company—as a **deployer**—bears regulatory responsibility.
2. Your transparency obligations (Art. 50) arrive in August 2026
Even in cases of legitimate use, Article 50(4) requires that any AI-generated content manipulating the image of real people must be clearly identified. This applies to your communication campaigns using avatars, synthetic videos, or any content generated with Midjourney, DALL-E, etc.
3. Reputational impact precedes financial sanctions
The Grok experience demonstrated this: even before potential fines, it was the global reaction—national blocks, investigations, commercial losses—that hit xAI. For an SME or a SaaS publisher, a similar incident, even on a smaller scale, can prove fatal before any intervention by the DGCCRF or the CNIL.
It is imperative to ensure that your generative AI tools respect minimum safeguards: rejection of manipulative requests, inability to generate non-consensual explicit content. The compaia diagnostic allows you to identify your obligations precisely according to your deployer profile.
Key Dates and Sanctions: What to Remember
- **February 2, 2025**: Entry into force of Article 5 on prohibited practices.
- **December 2025 – January 2026**: Explosion of the Grok scandal.
- **March 18, 2026**: European Parliament IMCO/LIBE vote on the Digital Omnibus.
- **August 2, 2026**: Transparency obligation for all AI-generated content (Article 50).
Sanctions for violating Article 5 are the heaviest in the AI Act: up to **35 million euros** or **7%** of total worldwide annual turnover, whichever is higher. For companies in the sector, the financial and reputational risk is real and immediate.
In Summary
The Grok case is not an isolated or extreme instance, but a revelation of what the AI Act intends to prevent. Obligations regarding design, risk management, and transparency are no longer regulatory abstractions: they address real, proven risks and are already subject to active monitoring. Anticipate your obligations, verify your tools, and consult the AI Act timeline to remain compliant at every step from 2024 to 2026.