Lessons Learned from the EU AI Act: What Organizations Should Know (Part-1)

Oct 9

7 min read

1. INTRODUCTION

AI now screens job applicants, approves loans, answers customers, flags fraud, and supports medical decisions. When it works, it saves time and money. When it fails, real people are harmed, regulators step in, and trust disappears fast.

The EU AI Act is the first comprehensive law written to prevent those failures. It is in the spotlight, and for good reason. But the real value of the Act is the lessons behind it. They match what global standards, regulators, and hard experience have been telling organizations for years.

This two-part series pulls those lessons together in one place. We use the EU AI Act as the anchor, then go deeper into the global standards and practices that help you act on it. Each lesson comes with a real case that shows what happens when it is ignored. Part 1 covers the Act itself and the first four lessons. Part 2 covers the next four, the global standards, a practical roadmap, and our conclusion.

2. THE EU AI ACT IN BRIEF

The Act sorts AI into four levels of risk. At the top are unacceptable-risk uses, which are banned. These include social scoring, manipulative AI that exploits vulnerabilities, untargeted scraping of facial images, and emotion recognition in workplaces and schools. These bans have applied since February 2025. Next are high-risk systems, which face strict rules. These include AI used in hiring, credit scoring, education, law enforcement, border control, and safety parts of products like medical devices and machinery. Below that are limited-risk systems, such as chatbots and deepfakes, which mainly need transparency. Most other AI, like spam filters, is minimal risk and largely left alone. General-purpose AI models, such as large language models, have their own rules, with extra duties for the most powerful ones. These have applied since August 2025.

The Act also assigns roles. Providers build AI or put it on the market, and deployers use it. Importers and distributors have duties too. Your role decides your obligations, and one organization can hold several. Buying AI from a vendor does not remove your responsibility as a deployer.

For high-risk systems, the Act requires risk management, data governance, technical documentation, record keeping, transparency, human oversight, accuracy and cybersecurity, a quality management system, and post-launch monitoring, including reporting of serious incidents. It also already requires AI literacy: staff who use AI must understand it well enough to use it safely.

Penalties are serious: up to €35 million or 7% of global annual turnover for banned practices, and up to €15 million or 3% for most other breaches.

On timing, a provisional political agreement in May 2026 is expected to move the main high-risk obligations to December 2027 for stand-alone systems and August 2028 for AI built into regulated products. Please confirm the final adopted dates. A delay is still no reason to wait, because the work takes many months and customers and partners are already asking about it.

The Act's biggest message is that responsibility does not end at launch. Controls are needed from the first idea to retirement:

Planning: decide whether the use case is allowed, how risky it is, and who owns it.
Data: check quality, sources, bias, and privacy.
Build and training: document your choices and review any vendors or models you rely on.
Testing: check accuracy, fairness, robustness, and security before sign-off.
Deployment: set up transparency, human oversight, and any registration or conformity steps.
Operation: monitor performance, keep logs, and handle incidents.
Retirement: shut systems down properly and keep the records you need.

3. KEY LESSONS FOR ORGANIZATIONS

LESSON 1: KNOW YOUR AI

You cannot manage what you cannot see. Most organizations use far more AI than they realize. There are tools bought from vendors, AI features built into existing software, and tools employees adopt on their own, often called shadow AI. The fix is an AI inventory. For every system, record what it does, who owns it, what data it uses, who supplied it, who it affects, and what risk level it falls into. Then keep it up to date, because a list that is a year old is almost useless.

Case study: A global electronics manufacturer (2023). Reports in 2023 said employees at a large electronics manufacturer pasted internal source code and meeting notes into a public AI chatbot. The company then restricted the use of generative AI tools. There was no clear list of approved tools and no firm rules, so confidential information left the company before anyone noticed.

What would have helped is simple: an approved-tools list, a clear usage policy, staff training, and technical checks for unapproved tools. These are the same things the Act's AI literacy and governance expectations point to.

An inventory goes out of date quickly. Gorisco can help build the register, set up an intake process for every new AI tool or use case, and keep it current through regular managed reviews.

LESSON 2: CLASSIFY AND ASSESS RISK BEFORE DEPLOYMENT

The Act wants risk management to happen before a high-risk system goes live, and to continue through its life. The key idea is proportionality. Not every system needs the same effort. Spend the most time on systems that decide things about people's jobs, money, health, safety, and rights.

A good process has four steps. First, classify each use case by risk level and by your role as provider or deployer. Second, assess the risks: bias, accuracy, security, privacy, and misuse. Third, decide what level of risk is acceptable and who has the authority to accept what is left over. Fourth, retest whenever the data, the model, or the way it is used changes.

Case study: A major online retailer's recruiting tool. Reuters reported in 2018 that a major online retailer had built an experimental hiring tool trained on ten years of past resumes, most of them from men. The system learned to mark down resumes with signals linked to women, and the company dropped it. Under the Act, hiring AI is high-risk. Testing for bias against representative data before use is a basic expectation, and it would have exposed the problem early.

A risk assessment is only as strong as its method. Gorisco helps organizations in running structured AI risk and impact assessments aligned to the EU AI Act, ISO/IEC 23894, and the NIST AI Risk Management Framework, and keep the results in a living risk register rather than a forgotten report.

LESSON 3: BUILD GOVERNANCE WITH CLEAR ACCOUNTABILITY

Every AI system needs a named owner, and the organization needs a clear way of making decisions about AI. Governance is what turns good intentions into consistent action. In practice this means publishing an AI policy that covers acceptable use, prohibited uses, and approval rules. It means forming a small cross-functional group with people from legal, security, data, compliance, HR, and the business. It means using the "three lines" idea: business teams own the risk, risk and compliance teams oversee it, and internal audit provides independent checks. And it means giving senior leaders, and the board, regular and honest reporting on AI risk.

Case study: A national childcare benefits scandal. A national tax authority used a risk-scoring system to flag possible benefit fraud. Tens of thousands of families were wrongly accused and told to repay large sums, and many were pushed into serious financial trouble. Investigations pointed to discriminatory risk factors, weak safeguards, and too little challenge from officials. The government resigned in January 2021. The failure was as much about governance and accountability as it was about the algorithm.

Governance on paper is easy. Governance that works means committees that meet, decisions that are recorded, and owners who are trained. Gorisco helps organizations to design the structure, then support it with ongoing managed services so it keeps working after the launch workshop is over.

LESSON 4: MAKE HUMAN OVERSIGHT REAL

Article 14 of the Act requires that people can understand, monitor, and override high-risk AI. The danger is oversight that exists only on paper. A reviewer who clicks "approve" on everything is not providing oversight. This is known as automation bias, the habit of trusting the machine over our own judgment. Real oversight needs reviewers who are trained, who have enough time, and who have the authority to disagree. They need to see how confident the system is and what its known limits are. There should be a clear path for escalating concerns and a clear way to stop the system. It also helps to measure oversight itself. If humans almost never override the AI, ask why.

Case study: A national welfare debt recovery scheme. Between 2016 and 2019, a national welfare agency used automated income averaging to raise debts against welfare recipients, with little human checking. Many of those debts were unlawful. A Royal Commission later condemned the scheme, and the government paid out around A$1.8 billion in settlements and repayments. Automated decisions that changed people's lives were issued without meaningful review or an easy way to challenge them.

4. CONCLUSION

The first four lessons share one theme: AI problems rarely start with the technology. They start with gaps in basic habits. A manufacturer did not know which AI tools its people used. A retailer did not test for bias before relying on a model. A tax authority lacked clear accountability. A welfare agency had no real human check on automated decisions.

The EU AI Act addresses each of these directly. Know what AI you have, assess its risks before launch, assign clear ownership, and keep humans genuinely in control. These steps are simple to describe, but they take steady effort to maintain.

In Part 2, we cover the next four lessons on documentation, data and vendors, monitoring, and legal red lines. We also look at the global standards that help put all eight into practice, and how Gorisco's managed GRC services can support you across the AI lifecycle.

At Gorisco, we help organizations strengthen resilience by aligning people, processes, and technology. From governance frameworks to remote-work policies, our focus is on building secure, transparent, and continuity-ready operations in today’s connected world.

To read our other blogs, click here. We would love to hear your thoughts, so please share your comments!