Source: [Original HTML page](https://stolenorbit.com/en/resources/human-review-for-ai/)

Language: English

A PRACTICAL BUSINESS GUIDE

# Human review of AI: a control should help someone make a decision.

An “Approve” button is insufficient. Reviewers need evidence, an understanding of consequences, time to check and a real ability to reject or correct the proposed action.

[By Stolen Orbit](https://stolenorbit.com/en/about/) Updated 24 September 2026

## Define exactly what is being approved

[Source for this section](https://stolenorbit.com/en/resources/human-review-for-ai/#decision)

Distinguish accepting a draft, authorising a change and checking an action that has already happened. These controls occur at different times. For an external message, a later review cannot prevent the wrong content from reaching the recipient.

Write the decision entrusted to the person: “verify customer, amount and attachment before sending” is actionable; “check the AI” is too vague. Assign an owner and a substitute. If nobody can decide in time, the system needs a declared behaviour: wait, stop or use an alternative process. Make that outcome visible to the team responsible for the case.

## What the reviewer needs to see

[Source for this section](https://stolenorbit.com/en/resources/human-review-for-ai/#interface)

Scroll the table to compare all columns.

Minimum information for an approval request

| Element | Purpose | Example |
| --- | --- | --- |
| Proposed action | Makes the consequence explicit | Send this email to this contact |
| Data and source | Supports an independent check | Extracted amount alongside its document passage |
| Changes | Avoids rereading an entire record | Previous address and proposed address |
| Reason for review | Directs attention | Customer identifier has multiple matches |
| Meaningful options | Allows stopping or correction | Approve, edit, reject, request clarification |

## Illustrative example: a reply containing a commercial offer

[Source for this section](https://stolenorbit.com/en/resources/human-review-for-ai/#example)

AI drafts a reply using an incoming enquiry and an offer in the CRM. The reviewer sees recipient, product, amount, validity and attachment. Differences between the current offer and the one mentioned in the message are highlighted. Until approval, the system retains a draft and does not send.

If the offer changes meanwhile, approval should not apply to a different version. Bind the decision to the checked data and verify relevant conditions again before acting. This prevents a valid review from becoming open-ended permission for subsequent changes. A reviewer should also be able to see whether a message was actually sent or is still waiting.

## Size the human workload and test absence

[Source for this section](https://stolenorbit.com/en/resources/human-review-for-ai/#capacity)

Measure queue arrivals, review time and cases returned for correction. If volume exceeds review capacity, users may accumulate a backlog or approve too quickly. The remedy could be a narrower scope, better data or a clearer interface rather than reduced controls.

Test an absent reviewer, duplicate approval, rejection and expiry. Rejection must still prevent the action if a background process retries. Record who decided, on which version and with what outcome, without collecting more content than needed to reconstruct the process. Include the handling of urgent cases in the operating plan.

## Write a review policy the team can apply

[Source for this section](https://stolenorbit.com/en/resources/human-review-for-ai/#policy)

OWASP recommends limiting autonomy and permissions and involving people in consequential actions. Adapt the practical approach here to the process. Review does not replace access enforcement or automatically make a task with serious consequences suitable for automation. It should be evaluated as part of the complete system.

-   List actions requiring advance approval and actions allowed within explicit limits.
-   Define reviewer evidence, competence and authority.
-   Specify expiry, substitutes and behaviour when nobody intervenes.
-   Enforce that the performed action matches the approved action.
-   Review escaped errors, corrections, waiting times and backlog with the operations team.

FROM IDEAS TO A BRIEF

## A template to work from.

[Source for this section](https://stolenorbit.com/en/resources/human-review-for-ai/#downloads)

### Human review plan

Routing and approval rules with named roles, review evidence, handling windows and a fallback when the reviewer is unavailable.

[Download the Markdown template](https://stolenorbit.com/downloads/human-review-policy-en.md)

### AI permission matrix

An identity–resource–operation matrix with justification, approver, negative tests and a revocation procedure.

[Download the Markdown template](https://stolenorbit.com/downloads/permission-matrix-en.md)

### Automation exception register

An exception queue with status, priority, ownership and closure evidence; repeated issues become inputs to workflow improvements.

[Download the Markdown template](https://stolenorbit.com/downloads/exception-register-en.md)

## Practical questions

[Source for this section](https://stolenorbit.com/en/resources/human-review-for-ai/#faq)

**Is reviewing low-confidence cases enough?**

Not always. Model-reported confidence is not automatically calibrated. Also use observable criteria such as action type, amount, missing data, conflicts and evaluation findings.

**Can review be sampled?**

Sampling can help quality assurance for some tasks, but it does not prevent errors in unreviewed cases. Assess consequences, reversibility and other controls before replacing advance approval.

**How do we avoid creating more work?**

Show the evidence needed for the decision, highlight changes and measure the complete effort including corrections and interruptions. If no benefit emerges, redesign or narrow the process.

## References and method

[Source for this section](https://stolenorbit.com/en/resources/human-review-for-ai/#sources)

[OWASP — LLM06:2025 Excessive Agency](https://genai.owasp.org/llmrisk/llm062025-excessive-agency/)

Reference on bounded autonomy, minimum permissions and approval for consequential actions. The commercial scenario is illustrative.

EXPLORE FURTHER

[Human review plan](https://stolenorbit.com/en/templates/human-review-policy/)

[AI permission matrix](https://stolenorbit.com/en/templates/ai-permission-matrix/)

[Automation exception register](https://stolenorbit.com/en/templates/exception-register/)

[AI agents for business](https://stolenorbit.com/en/services/ai-agents/)

[Turn documents into usable data, with checks at every handoff.](https://stolenorbit.com/en/services/ai-document-data-extraction/)

[Find answers in your company’s approved knowledge.](https://stolenorbit.com/en/services/internal-ai-knowledge-assistant/)

[All practical guides](https://stolenorbit.com/en/resources/)

## Which process should improve first?

[Source for this section](https://stolenorbit.com/en/resources/human-review-for-ai/#growth-closing-heading)

Start with a concrete process, the systems you use and the people who will operate it every day.

[Let’s discuss your process](https://stolenorbit.com/en/contact/)

## Turn the question into a next step.

-   [Evaluating AI reliability](https://stolenorbit.com/en/resources/evaluating-ai-reliability/)
-   [Evaluation lab](https://stolenorbit.com/en/tools/ai-extraction-evaluation/)
-   [AI evaluation dataset](https://stolenorbit.com/en/templates/ai-evaluation-dataset/)