Deemed Providers Under the EU AI Act: When Customers Become Providers in Typical Use Cases

Many companies use AI tools to support business processes. In HR especially, such systems have long been part of everyday work: think AI-powered software that automatically screens résumés or analyzes employee data. You save time and resources; after all, you're just a user of the software… But that's exactly where a legal risk can hide. You can go from mere user to responsible provider of an AI system, and suddenly face obligations you never anticipated. That can mean substantial legal and financial risk for your company.  

Why your HR tool can become a case for the EU AI Act

Many companies rely on existing tools such as general-purpose AI systems. But if that tool is used to evaluate job applications or to generate recommendations on hiring and promotions, it can suddenly qualify as a so-called high-risk AI system. At that moment, a simple user legally becomes a provider. In other words: Whoever uses AI in a way that constitutes a high-risk use case is treated as the provider, even if they never developed the software. This shift in roles is known as the provider fiction, or deemed-provider rule.

When does an AI system count as high-risk?

The EU AI Act classifies AI systems by their risk potential, and the strictest rules apply to high-risk AI systems. These include applications that can significantly affect people's health, safety, or fundamental rights. The AI Act names specific fields of application, such as HR management, credit scoring, and access to education. In HR, for example, a tool can be classified as high-risk if it automatically generates recommendations for hiring or promotions. Why? Because decisions like these directly affect people's professional lives, and with that, a fundamental right.

A key distinction: provider vs. deployer

The AI Act fundamentally distinguishes between the provider (the developer) and the deployer (the user) of an AI system. Under Art. 26 of the AI Act, deployers of high-risk AI systems already carry certain obligations, such as human oversight and reviewing input data. But once the deemed-provider rule in Art. 25 of the AI Act comes into play, the range of obligations grows substantially. The deployer must then meet the same obligations that otherwise apply only to providers. These obligations determine how a company uses, documents, and monitors its AI, including setting up a risk management system, ensuring data quality, maintaining technical documentation, and meeting transparency and provisioning obligations toward users. The deemed-provider rule can quickly become a compliance risk when it kicks in unnoticed.

When you become a provider

In most cases, the deemed-provider rule applies when a general-purpose AI system that was not originally classified as high-risk is then used for a high-risk purpose. No technical modification of the AI system is required: a change of purpose alone is enough.

A concrete example: Your company uses a general-purpose AI system such as Microsoft Copilot to draft emails or summarize meetings. So far, unproblematic. But then HR gets the idea to use Copilot to evaluate employee performance. The AI is fed sales figures, customer feedback, and internal chats to generate a list of the best-suited employees for a promotion round. Using AI to evaluate employees, however, is a clearly defined high-risk use case. Even though you're only a user of Copilot, this use makes you a provider in the eyes of the law. Your company must now suddenly meet the full catalog of obligations for providers of high-risk AI.

Very few companies are prepared for that. The risk of unintentionally sliding into the provider role is enormous.

What you can do now

The good news: You can protect yourself against this compliance risk. To avoid stumbling blindly into the provider role, follow these concrete steps:

Create transparency: Run an internal inventory. Which AI systems are already in use in your company? By whom, and for what?

Adopt an internal AI policy: Set binding rules for how all employees may use AI systems.  

Define clear limits (blacklisting): Your policy should include a "blacklist" that explicitly prohibits (or subjects to prior approval) certain forbidden or particularly risky use cases, such as evaluating applicants or employees.

Review the intended use before deployment: Every planned use of an AI system needs legal review up front. Modifications and new use cases should also always be legally reviewed and documented. When in doubt, bring in legal expertise early, before the AI goes into production.

The bottom line

The EU AI Act is more than a regulation for big tech. It affects every company that uses AI. The line between a simple user and a fully responsible provider is thinner than you might think. The deemed-provider rule is not a theoretical footnote; it's a prime example of how quickly and unexpectedly sweeping obligations can arise. Companies that use AI tools flexibly should be aware that even a simple HR use case can suddenly trigger an entirely new set of legal obligations. Acting with foresight here spares you compliance problems down the road while creating transparency and trust in how you work with AI. A clear strategy and binding internal rules for AI use are your company's shield.

‍