Risk · R-162 · Human Factors & Workforce

Unfair algorithmic management

AI is used in recruitment, task allocation, performance assessment or dismissal without fairness testing, explanation, human review or a route to challenge.

Severity: High
Audience: Board · Heads-of · Practitioner
Frameworks: Section 39 · Annex III · Article 26 · Articles 22A to 22D
Responsible AI domain: 07 Fairness & Customer Outcomes
09 Human Oversight & Contestability
Lifecycle stage: 2 Assess & tier
4 Validate & assure
7 Operate & monitor
Reviewed: 2026-11-11

What it is

Employment is among the areas where automated decisions matter most to the person affected. Systems now screen applications, rank candidates, analyse interviews, allocate shifts and tasks, set targets, score performance and identify people for dismissal. The EU AI Act classes AI used for these purposes as high-risk. The risk is that such a system is used as an administrative convenience, with none of the safeguards the stakes demand: no test for discrimination, no reasons given, no person who actually reviews, and no way for the worker or applicant to object.

How it happens

A recruitment or workforce product is bought by HR with no AI assessment.
The system's ranking is followed because there are too many applicants to read.
Performance scores from a tool feed pay or dismissal with no human examination.
Workers are not told that a system is involved.
A feature inside an existing HR platform is switched on.

Why it matters

Discrimination in recruitment and employment is unlawful whether a person or a system does it, and the employer is liable. Significant decisions made with no meaningful human involvement attract the automated decision safeguards. Under the EU AI Act, deployers of high-risk systems at work have duties that include informing workers. Unfair treatment by an opaque system is also corrosive to morale.

Warning signs

HR systems with scoring or ranking features are absent from the AI inventory.
No fairness testing exists for recruitment or performance tools.
Managers cannot explain an individual's score.
Applicants and staff are not told that AI is used.

Who owns it

The executive responsible for people is accountable. HR owns the tools and the decisions. Legal and the data protection officer advise. The second line tests for fairness.

In practice

A logistics firm uses a system to allocate shifts and flag under-performers for review. Workers who take longer routes for safety reasons are flagged repeatedly, and three are dismissed on the system's report. None was told how the score was calculated. The same pattern generalises to any management decision delegated to a tool that those subject to it cannot see into.

Mitigating controls

Primary controls

Bias testing and fairness monitoring Detective: Tests recruitment and management systems for discrimination. A system's performance and outcomes are measured separately for each relevant group before release and on a set cycle in production, against thresholds that trigger investigation.
Human-in-the-loop review Preventive: Keeps a person responsible for decisions about individuals. Consequential AI outputs and decisions are reviewed by a competent person, with the time, information and authority to change them, before they take effect.

Supporting controls

Contestability and appeals process Corrective: Lets staff challenge a decision. People affected by an AI-supported decision can challenge it and have it reviewed afresh by someone with the authority, information and independence to change it.
Explanations for affected individuals Directive: Tells staff how a decision was reached. People subject to an AI-supported decision receive a meaningful explanation of what was decided, the main reasons in their case and how to challenge it.
Speak-up route for AI concerns Detective: Staff can raise concerns about how they are being managed by AI. Staff can raise concerns about how AI is being built or used, in confidence and without fear of reprisal, and each concern is assessed, investigated where warranted and answered.
Workforce impact assessment and involvement Directive: Involves staff before such systems are introduced. Before AI is deployed in ways that change work, its effect on roles, skills, working conditions and monitoring of staff is assessed, with the people affected and their representatives informed and heard.

Framework and clause references

Framework	Clause	Title
Equality Act 2010	Section 39	Employees and applicants Discrimination against employees and applicants.
EU AI Act	Annex III	High-risk AI systems Recruitment and worker management are high-risk uses.
EU AI Act	Article 26	Obligations of deployers of high-risk AI systems Informing workers.
UK General Data Protection Regulation	Articles 22A to 22D	Automated processing and significant decisions Significant decisions about workers.

Sources

EU AI Act: Regulation (EU) 2024/1689 European Union (EUR-Lex) · official text

How to cite

Axiom Verity (2026). Unfair algorithmic management. Reference Library, R-162. https://axiom-verity.com/risks/unfair-algorithmic-management. Reviewed 2026-11-11.

Found an error or a better source? Report a correction.