Key Takeaways
- The biggest AI governance challenge under the EU AI Act is visibility, not paperwork or policy-writing.
- Effective AI governance requires continuous visibility across procurement, operations, employees, vendors, and day-to-day business workflows.
- Treating AI governance as a legal-only compliance exercise repeats many of the same mistakes organisations made with GDPR.
- AI governance cannot function as a yearly audit process because AI systems continuously evolve through updates, integrations, retraining, and human interaction.
- Organisations that fail to understand where AI is already operating inside their systems may lose control over how decisions are being shaped.
You probably know about the AI tools your organisation officially approved. But what about the AI nobody formally signed off on? Do you actually know every place AI is already operating inside your business?
Here, I am not talking about the chatbot your customer service team is using or about an AI system your team recently rolled out internally. And I’m not just talking about the systems that appear in your internal documentation either. I’m talking about all of it. Every tool, every process, every little workaround or shortcut your employees started relying on without anyone documenting it.
If your answer is anything other than a confident yes, then this article is for you. But even if your answer is a confident yes, this article is still for you, because confidence without visibility is exactly how we got into the GDPR mess in the first place. This happened because many companies felt confident about GDPR compliance right up until they realised they didn’t actually have full visibility into what was happening inside their own systems.
Why GDPR Should Have Taught Us Better
If we look back at how most companies handled GDPR, it usually went something like this: a deadline appeared on the horizon, the legal team was handed the problem, consultants were hired, policies were written, cookie banners were deployed, and everyone breathed a sigh of relief. After all, compliance was achieved, the box was ticked, so everyone could move on.
Except that it wasn’t compliance. In many cases, it was a sophisticated, well-documented, and often sincere performance of compliance that mostly succeeded in generating paper trails rather than actually changing how organisations handled personal data.
The reason I say this? In many cases, the gap between the privacy policies companies published and the reality of how data actually moved through their systems was huge. And most organisations knew it, but they just didn’t know what to do about it, so they did what companies usually do under deadline pressure: they documented the surface and hoped nobody looked too hard underneath.
Although many organisations learnt their lesson and have since built more mature privacy programmes, others tend to approach the EU AI Act in much the same way they approached GDPR years ago, which means they’re heading straight into the same cycle we saw back then: reactive meetings ➝ rushed documentation ➝ overloaded legal teams ➝ endless policy-writing exercises designed more to survive audits than to actually deal with real operational risk.
The issue is that, with the most demanding requirements for high-risk AI systems set to apply from August 2026, and the first obligations already in force since February 2025, the window for getting ahead of this is narrowing fast.
The Dangerous Misconception Behind Most AI Governance Efforts
The assumption that keeps getting organisations into trouble is a simple one: this is a legal problem, so legal should fix it. And we must admit, it’s an understandable assumption. After all, regulation sounds like a legal matter, doesn’t it?
Part of the reason this assumption persists is that treating AI Act compliance as a legal project feels reassuring. A legal project produces documents, frameworks, checklists, and timelines, so it feels more manageable and creates the illusion that uncertainty can be organised into folders and reviewed later if necessary. That worked, at least partially, for GDPR.
So, it’s no surprise that the responsibility gets handed to legal and compliance teams once again, while the rest of the organisation carries on as usual, waiting to be told what’s allowed and what isn’t.
But unlike GDPR, which was largely about managing personal data contained inside systems, the AI Act isn’t really about static information. It’s about systems that constantly evolve through updates, integrations, retraining, and everyday human interaction. Furthermore, these systems don’t stay neatly inside departments waiting to be audited once per year. They spread across teams, tools, and workflows, often much faster than organisations can properly track them. That’s why so many current conversations around AI governance already feel disconnected from the operational reality.
Therefore, the core challenge of the EU AI Act compliance isn’t only legal but also operational. That’s because it forces organisations to answer questions like: Where exactly is AI operating inside the business? What decisions is it influencing? What happens to real people as a result? These are questions that extend beyond legal interpretation, so you can’t simply hand them over to your legal team and expect meaningful answers back. These questions belong to product managers, engineers, procurement teams, customer service teams, and HR directors. This is especially true for organisations developing or significantly customising AI systems. Under the Act, a provider’s obligations are considerably heavier than those of a business that simply deploys third-party AI tools.
However, the problem goes deeper than that. The biggest AI governance risk facing most organisations right now isn’t just that this technology gets adopted rapidly, evolves continuously, spreads across departments, and will likely reshape business operations far more deeply than GDPR did. The biggest risk comes in three distinct layers, each one less visible and more dangerous than the last. The worrying part? Many organisations are only fully aware of the first one, while visibility into the others remains limited.
Tree Layers of AI You Need to Know About
This is where most AI Act compliance conversations start falling apart. That’s because, in general, frameworks focus entirely on the first layer and barely acknowledge the other two exist. To understand why this matters, let’s look at all three layers separately.
Layer One: Official AI
This is the AI leadership officially approved and everybody talks about. Whether it’s an internal copilot, a customer service chatbot, a fraud detection system, a recommendation engine, or an AI analytics tool, these are the systems that appear in your documentation, get mentioned in meetings, and end up at the centre of most AI Act compliance discussions simply because they’re visible and, therefore, easy to point at.
Governing layer-one AI is important. And compared to everything else, it’s also the easiest part because you know where these systems are, which means you can document and audit them. In many ways, this is the layer most compliance programs are built around, which would be perfectly fine if it reflected the full reality of how AI actually spreads inside organisations.
But it doesn’t. In practice, focusing only on “official” AI is a bit like believing cybersecurity risks only exist inside the systems your IT department installed. This approach is dangerous because it completely ignores how AI spreads across organisations, and by now, we should all know that technology has never spread through formal approval processes alone.
Layer Two: Shadow AI
Employees are already using AI tools everywhere simply because they help them work faster and keep up with growing workloads. The problem is that they’re often using these tools without formal approval. Not because they’re deliberately ignoring company rules, but because, in many cases, nobody has clearly told them not to. As a result, instead of openly discussing their use of AI, employees use different AI solutions discreetly and hope nobody starts asking questions.
This creates a strange organisational paradox: companies publicly announce ambitious AI strategies while simultaneously creating internal cultures where employees feel uncomfortable admitting which AI tools they actually use every day.
That should concern leadership far more than it does. Because once AI usage goes underground, governance loses visibility completely, and the organisation no longer knows what data is being exposed, where it’s going, which outputs are influencing decisions, or how heavily teams are already relying on AI systems nobody officially approved.
And this is where one of the biggest myths around AI governance starts falling apart. Many organisations still believe they can control AI adoption through restrictive policies alone. But they can’t simply because they can’t govern what they can’t see. Instead, they’re trying to manage AI the same way companies once tried to stop employees from using cloud storage, personal devices, or unofficial collaboration tools. We already know how that story ended.
It’s worth noting, however, that most AI tools employees use will likely fall into the minimal- or limited-risk categories under the EU AI Act, where regulatory obligations are fairly limited. But the situation changes considerably when the AI systems being used influence hiring, creditworthiness, access to services, or critical infrastructure.
Layer Three: Inherited AI
This is where visibility starts breaking down even further. Many companies adopt AI not through deliberate strategy, but through the software they already use, as vendors continue adding AI capabilities into existing products.
While many of these capabilities may never qualify as high-risk systems under the AI Act, whether they do is still something organisations need to assess, and that becomes almost impossible if AI solutions have entered the system without proper tracking.
In some cases, vendors clearly communicate newly introduced AI capabilities, but customers don’t review them closely. In others, the communication may be limited or simply overlooked. Either way, AI capabilities can quietly enter critical systems without being fully assessed from a governance or risk perspective.
In practice, that means your organisation may already be using AI systems that influence hiring decisions, shape customer interaction, generate financial reports, flag security threats, or affect business operations in ways nobody fully understands. You didn’t choose those systems, you didn’t evaluate them through any risk framework, you might not even fully understand how they work, and you almost certainly didn’t update your compliance documentation when they arrived, because you didn’t know they had.
This isn’t some theoretical risk we may face in the future, because it’s already happening. But the most concerning part is that organisations can’t even measure how much of this is already taking place inside their systems.
However, there’s a way to understand the scale of the problem: ask your procurement team how many of your existing software vendors have introduced AI capabilities over the last couple of years. Then, compare that number with how many vendor contracts were actually reviewed specifically for AI-related changes during the same period. The gap between those two numbers is your inherited AI problem.
What a Practical Approach to AI Act Compliance Should Look Like
All of the above doesn’t mean organisations are doomed to repeat the same mistakes they made with GDPR. But it does mean they need to approach AI governance differently. To begin with, if your entire compliance strategy starts with policies and documentation before you even understand where AI is already operating inside your business, you’re building governance and compliance in the wrong order.
In case you’re wondering what a practical approach to AI compliance looks like, here are five important steps to focus on:
- The first and most urgent step is building AI visibility before attempting compliance. You can’t govern systems you don’t know about, you can’t classify risks you haven’t identified, and you can’t build meaningful oversight around AI usage that nobody inside your organisation is openly discussing. So before meetings, documentation exercises, policies, and frameworks, you should conduct an honest internal audit across all three layers described above.
The audit should answer a few key questions: What AI systems has leadership formally approved? What AI tools are employees already using in their daily work, including the ones nobody officially signed off on? What AI capabilities have your vendors introduced into the products your organisation depends on? The answers to these questions will tell you more about your actual AI governance situation than any risk classification spreadsheet ever will.
Additionally, the audit shouldn’t only identify where AI is used, but also which use cases carry the highest risk, particularly those affecting sensitive data, decisions, and individuals.
- The second step is treating procurement as an ongoing AI governance function rather than a one-time approval process. Every vendor contract renewal, software update, integration, or platform expansion is now an AI governance checkpoint whether your organisation realises it or not.
That means your procurement team needs to start routinely asking vendors questions that barely existed a couple of years ago: What AI capabilities are now included in the product? What changed since the last review? How do AI-generated outputs influence decisions inside the system? What data is being processed, retained, or used to improve those models? This isn’t about turning your procurement staff into AI engineers. It’s about recognising that software vendors are no longer selling static tools but continuously evolving systems.
- The third step is moving even further away from the idea that AI governance belongs primarily to legal and compliance teams. Effective AI governance only works when product teams, engineers, procurement, security, HR, and leadership all share responsibility for visibility and oversight. Otherwise, organisations end up with governance structures where the people writing policies are disconnected from the people actually using, deploying, purchasing, or maintaining AI systems every day.
Legal absolutely matters here. Compliance matters just as much. But neither function can sit at the centre of AI governance on its own, because the operational reality is spread across the organisation.
- The fourth step is employee education, but not the kind companies usually default to. Your employees don’t need another generic AI compliance training module they click through while answering emails in another tab.
Employees are using AI tools because they help them handle unrealistic workloads and meet growing performance expectations. Trying to ban “unofficial” AI usage completely isn’t only unrealistic, but may also make the visibility problem worse because employees will simply stop talking openly about the tools they rely on while continuing to use them.
What actually works is creating an environment where employees understand:
- which types of AI usage create real risk;
- what data should never be shared with external systems;
- when human review is required;
- and, most importantly, that transparency about AI usage won’t automatically lead to punishment.
This is also where a practical AI acceptable use policy becomes important, not as a restrictive document, but as something that helps make AI usage visible enough to govern.
- The fifth step, and perhaps the hardest for organisations to accept, is that AI governance can’t function as a yearly compliance exercise. Annual AI audits won’t work in an environment where vendors push product updates monthly or even weekly, where employees continuously discover and experiment with new tools, and where AI model behaviour can shift through retraining, ongoing updates, or everyday human interaction.
That means AI governance shouldn’t exist as a disconnected compliance effort running alongside the business, but as part of how the business already operates, inside procurement reviews, vendor on-boarding, security assessments, engineering reviews, product releases, HR workflows, and day-to-day operational decisions.
Concluding Thoughts
If we step back from the regulatory discussion for a moment and look at what the EU AI Act is actually exposing, we begin to see a deeper problem where many organisations no longer fully understand the technology operating inside their own environments.
While similar challenges existed during the rise of cloud computing and SaaS adoption, AI introduces an additional layer of complexity due to its dynamic behaviour, decision-making influence, and rapid rate of integration across tools. This has never happened at this scale, which raises an unsettling possibility: What if the EU AI Act isn’t actually about governing AI? What if it’s really about discovering just how much of it we’ve already lost track of?
Because if we no longer fully understand the technology operating inside our own organisations, then the compliance question almost becomes secondary. The harder question is the one nobody seems willing to ask out loud: If we don’t fully understand what systems we’re operating, how they influence decisions, or why they behave the way they do, who exactly is in control? And what does that mean for the decisions already being made today, not just for tomorrow, but for ten years from now?
Extra sources and further reading
- What Leaders Need to Know about Auditing AI – Harvard Business Review
https://hbr.org/2025/03/what-leaders-need-to-know-about-auditing-ai
The article argues that as AI becomes deeply embedded across organisations, leaders need structured AI audits to identify hidden risks, improve oversight, and ensure AI systems remain transparent, accountable, and aligned with governance expectations. - A Survey of Reinforcement Learning from Human Feedback – Cornell University https://arxiv.org/pdf/2312.14925
In this paper, the authors examine how AI systems become socially and psychologically meaningful through human interaction, proposing the MIRA model to explain AI’s evolving role as both a relational partner and a mediator of human communication. - AI Act – European Commission
https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
This official EU policy page explains the EU AI Act, which uses a risk-based approach to classify AI systems and impose stricter obligations on higher-risk uses of AI in order to ensure safety, transparency, and protection of fundamental rights. - NIST AI RMF Playbook – NIST
https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook
This framework provides practical guidance for ongoing AI governance, risk assessment, testing, evaluation, and oversight across the AI lifecycle.
