All articles

Where a person must check AI's work

Why AI output needs checking, which decisions always need a person, and what real checking looks like.

Key figures

Employees who relied on AI output without critically evaluating it
66%
UK businesses applying some human input or oversight to AI output
81%

Sources: University of Melbourne and KPMG, Trust, attitudes and use of AI, 2025 · UK Government, AI adoption survey, 2026

A person must check anything that affects someone's money, job, legal position or safety. That includes anything sent to customers in your name, and any figure or fact you rely on. Checking means reviewing the result and being able to change it, rather than just signing it off.

In the 2025 Ayinde v London Borough of Haringey case, the High Court warned that freely available generative AI tools can produce coherent and plausible responses that turn out to be entirely incorrect, making confident assertions that are simply untrue.

A 2025 survey of 48,340 people in 47 countries by the University of Melbourne and KPMG found that two-thirds of employees (66%) have relied on AI output at work without critically evaluating it.

Automated decisions under UK law

Under the Data (Use and Access) Act 2025, a decision counts as based solely on automated processing if there is no meaningful human involvement. Where significant decisions about people are made that way, safeguards must let the person get human intervention and contest the decision.

The ICO's 2026 draft guidance, currently under consultation, states that human involvement must be active rather than a token gesture. In the UK Government's 2026 AI adoption survey, 81% of businesses reported applying some level of human input or oversight to AI output.

What real checking means

Real checking means someone reviews the output before it takes effect, has the authority to change it, understands what the tool can get wrong, and looks at the underlying information. A tick-box sign-off on hundreds of items an hour is not checking.

Where a person must check

  1. 01Decisions about hiring, pay or someone's job.
  2. 02Decisions about credit, refunds or what a customer owes.
  3. 03Anything sent to customers in your name.
  4. 04Figures in quotes, invoices and reports.
  5. 05Legal, health or safety statements.
  6. 06Facts, sources and references you rely on.
  7. 07Anything using sensitive personal data.

In every system we build, a named person on your team approves what leaves the business. One of the questions in our free scorecard asks whether you've agreed which tasks AI must never handle without a person checking.

Sources

Clinton Onyekwere

Clinton Onyekwere founded Clinton AI in Leeds. We help organisations adopt AI and build lasting internal skills.

Get our newest articles

We only email you when a new piece goes live.

See where AI fits your business.