Shadow AI: When AI rules lag behind practice — and the labs get it wrong too

Generative AI has arrived in German workplaces; internal governance has not kept pace. About 75% of knowledge workers already use AI, mostly without official approval. Nearly all companies — 98% — have an AI strategy, yet only 39% actively steer its use from top management. Microsoft's Work Trend Index counts 60% of leaders who admit they lack a clear implementation plan. Between the strategy document and the desk, a compliance gap has opened that now has its own name: Shadow AI.

Bring-your-own-AI is rarely malicious

78% of AI users bring their own tools to work. The reason is mundane: they want to work faster and find no approved option in-house. According to industry association Bitkom, only 26% of German companies currently provide official AI services to their staff. Letting three-quarters of a workforce work with a technology that only a quarter of employers actually provide doesn't create an innovation culture — it creates shadow IT with a language model attached.

The risk turns serious wherever the data is. HR, finance, tax and legal routinely handle personal and legally sensitive material. The moment application files or employee data get pasted into a public tool, the data-minimization and purpose-limitation principles of Article 5 of the GDPR kick in — and nobody inside the company can say how that data is processed or stored afterward. On top of that, since August, the EU AI Act's transparency and documentation duties apply to use cases affecting people, rights or safety.

• Data protection: which personal or business-critical data may flow into which model?
• Regulation: does the use case fall under the EU AI Act's transparency and documentation duties?
• Accountability: who answers for the output if nobody reviews it professionally?

A year without a filter — at Anthropic

Anyone assuming the labs themselves had closed that gap was proven wrong the very same week. Anthropic disclosed in its own safety report that from May 2025 to April 2026, all data from external contractors passed through its systems without the blocking biological classifiers active. Those filters are meant to stop dangerous knowledge about chemical or biological weapons from being extracted from the models. Around 50,000 people were affected, together running roughly 133 million chats with the models; they were screened solely by the external providers, whose vetting, according to Anthropic, was often insufficient.

After its own investigation, the company found no evidence of actual misuse and has since tightened requirements for its contractors. The irony is still in the detail: this is the same company whose CEO, Dario Amodei, considers chemical and biological weapons risk more dangerous than cyberattacks. It wasn't the rule that was missing — it was the check that it was actually running.

A rule whose effectiveness nobody measures isn't a control — it's a statement of intent with a case number.

OpenAI disbands its Preparedness team

At OpenAI the move happened more in the open. In late July, the company disbanded its Preparedness team, which was tasked with assessing whether its own models posed severe or catastrophic risks — according to the Financial Times, citing several internal sources. Responsibility for biological and cyber risks was distributed across existing teams; the team's former head, Dylan Scandinaro, now investigates the risks of recursively self-improving systems. Co-founder Greg Brockman said safety work had been integrated more deeply into model development.

Internally, the story sounds different. Several employees from the safety division have resigned recently, including ethics lead Chloe Bakalar and Joshua Achiam. One source described a simmering feeling of responsibility and fear that the company isn't doing enough on safety. Especially after the autonomous hacking incident at Hugging Face, staff also spoke out publicly; one said they hoped OpenAI would treat the incident as a warning shot.

What companies should take from this

For AI governance inside any company, both cases teach the same uncomfortable lesson. An approved-tools list, a policy, and a training session produce documentation, not security. What matters is the question neither Anthropic nor OpenAI answered in time: is the control actually running today, and how would we notice if it had failed? Anyone who genuinely wants to push back Shadow AI also has to answer the question 78% of employees have already answered with their own tools — which approved offering actually makes their work faster. Bans without an alternative just move usage to where nobody can see it anymore. For companies outside Germany, the lesson travels well: GDPR's principles and the EU AI Act's transparency duties apply across the whole EU, so any organization handling EU customer or employee data through unsanctioned AI tools faces the same exposure, whether the shadow use started in Frankfurt or anywhere else.