Human oversight

Article 14 of the EU AI Act sets out what human oversight of a high-risk AI system must achieve.

What Article 14 requires

A high-risk system must be built, “including with appropriate human-machine interface tools”, so that people can effectively oversee it while it is in use. It must be supplied so that the people overseeing it are enabled, “as appropriate and proportionate”, to:

• understand the system’s limits and spot anomalies
• stay aware of automation bias
• interpret the output correctly
• decide not to use the system, or disregard, override or reverse its output
• stop it with a “stop” button or similar, halting it in a safe state

The measures must be “commensurate with the risks, level of autonomy and context of use”.

Who does what

The provider, who builds the system, builds measures in where technically feasible, specifies measures for the deployer, or both. They go in the instructions for use.

The deployer, whoever uses the system, must assign oversight to people with “the necessary competence, training and authority, as well as the necessary support”. It monitors the system as the instructions direct. If it has reason to think that using the system as instructed may result in a risk, as the Act defines one, it must suspend use and inform the provider or distributor and the authority.

A deployer that owes a fundamental rights impact assessment describes there how it will implement these measures.

Why it matters

Article 14 aims to prevent or minimise risks to health, safety or fundamental rights. It singles out risks that persist despite the other requirements in its section, such as Article 15’s accuracy and robustness. Recital 73 adds that, where appropriate, the system should have built-in limits it cannot override and respond to its operator.

Remote biometric identification

This use gets a stricter rule. The deployer may not act on a match unless at least two people with the necessary competence, training and authority have separately verified and confirmed it. Systems used for law enforcement, migration, border control or asylum are exempt where EU or national law considers the rule disproportionate.

When it applies

After the Digital Omnibus on AI, Articles 14 and 26 apply from 2 December 2027 to high-risk systems listed in Annex III. For products, they apply directly from 2 August 2028 under the laws in Section A of Annex I, such as those on toys and medical devices. For Section B products, such as vehicles, the Act instead amends each sector’s law so that its measures on AI take these requirements into account.

A system placed on the market or put into service before its date is covered only if its design changes significantly afterwards. There are two exceptions. Systems intended for public authorities must comply by 2 August 2030. Components of the EU’s large-scale IT systems must comply by 31 December 2030.

Where definitions disagree

The term is older than the Act. In 2019 the High-Level Expert Group on AI, an independent group set up by the European Commission, listed “human agency and oversight” among seven key requirements for trustworthy AI, all “of equal importance”. The Act’s recital 27 recalls them as non-binding principles. The guidelines say oversight may be achieved through approaches “such as”:

• Human-in-the-loop: a person can intervene in every decision cycle, which “in many cases is neither possible nor desirable”
• Human-on-the-loop: a person can intervene during design and monitor the system’s operation
• Human-in-command: a person can oversee the system’s overall activity and decide when and how to use it

The sources disagree on how close the person must sit to each decision. Article 14 uses none of these labels. Outside biometric identification, it asks for the ability to monitor, override and stop, and to decide not to use the system “in any particular situation”, which echoes human-in-command.

MITRE ATLAS, a catalogue of attacks on AI systems, uses human-in-the-loop for a narrower control: a person approves an AI agent’s actions before the agent takes them. ATLAS scales it to the consequence of the task, with minimal oversight for minor, repetitive tasks.

The US National Institute of Standards and Technology (NIST) makes oversight part of a voluntary framework, not a legal duty. Its AI Risk Management Framework asks organisations to define, assess and document human oversight processes. Among issues it says need further research, it notes: “Some AI systems may not require human oversight, such as models used to improve video compression. Other systems may specifically require human oversight.”

Questions and answers

What does Article 14 of the EU AI Act require?

Article 14 requires high-risk AI systems to be designed so that natural persons can effectively oversee them while they are in use. As appropriate and proportionate, the people overseeing must be able to understand the system's capacities and limitations, stay aware of automation bias, interpret its output, disregard or override that output, and stop the system in a safe state.

Who is responsible for human oversight under the EU AI Act, the provider or the deployer?

Both. The provider builds oversight measures into a high-risk AI system or identifies measures for the deployer to implement, and lists them in the instructions for use. Under Article 26, the deployer assigns oversight to people with the necessary competence, training, authority and support.

Does human oversight mean a human must approve every AI decision?

Not under Article 14 of the EU AI Act, which requires that people can monitor, override and stop a high-risk system rather than approve each output. The exception is remote biometric identification: at least two people must separately verify a match before the deployer acts on it. Systems used for law enforcement, migration, border control or asylum are exempt where EU or national law considers the rule disproportionate.

What is the difference between human-in-the-loop, human-on-the-loop and human-in-command?

The 2019 ethics guidelines of the High-Level Expert Group on AI, set up by the European Commission, give all three as examples of how human oversight may be achieved. Human-in-the-loop means a person can intervene in every decision cycle. Human-on-the-loop means a person can intervene during design and monitor the system's operation. Human-in-command means a person can oversee the system's overall activity and decide when and how to use it.

When does Article 14 of the EU AI Act apply?

After the Digital Omnibus on AI, Article 14 applies from 2 December 2027 to high-risk systems listed in Annex III, and from 2 August 2028 to high-risk products under the EU laws in Section A of Annex I. Most systems placed on the market or put into service before then are covered only if their design later changes significantly. Systems intended for public authorities must comply by 2 August 2030.