Designing a Human Review Queue That Does Not Become a Bottleneck

“Send it to a human” is not an operating design.

It is a routing instruction. The hard work begins after the handoff.

If every uncertain recommendation, stale source, policy conflict, failed command, and customer-facing draft enters the same queue, review becomes the bottleneck. High-consequence work waits behind low-value cases. Reviewers open incomplete evidence packets. Reassignment resets the visible age. Approvals become a reflex. The automation appears controlled because a person clicked a button, while the underlying decision remains weak.

A useful human-review queue must do more than collect tasks. It must decide what deserves review, preserve why the machine stopped, route the case to a qualified person, expose the consequence clock, constrain work in progress, record a reviewable decision, and keep the case open until execution is reconciled.

This paper proposes that architecture for multi-location self-storage. It is a reference method and fictional exercise—not a description of a deployed product, customer workflow, staffing result, service level, or measured improvement.

Open the full-size human-review queue state machine.

Human review is a service, not a safety label

NIST AI RMF 1.0 says organizations should define and differentiate roles and responsibilities for human-AI configurations and oversight. Its current Core also calls for processes for human oversight to be defined, assessed, and documented. Appendix C warns that human-AI interaction varies, human judgment carries bias, and explanations are interpreted differently by different people.

Those points matter because the presence of a reviewer does not prove meaningful oversight. A reviewer may lack:

• the source evidence used by the automation;
• the authority to make the decision;
• the qualifications required for the consequence;
• enough time before the decision becomes unsafe or irrelevant;
• a visible alternative to approve or deny;
• the ability to request missing evidence;
• the ability to stop downstream execution; or
• feedback showing whether the approved action actually occurred.

The queue should therefore be treated as an operating service with an explicit intake contract, state model, capacity boundary, decision authority, and reconciliation obligation.

Keep the queue distinct from the exception ledger

An exception ledger records work that stopped, conflicted, failed, or requires accountable follow-up. A human-review queue is narrower: it manages cases in which a qualified person must adjudicate a bounded decision.

Some exceptions do not need adjudication. A provider timeout may need retry or incident response. A duplicate event may need idempotent suppression. A stale facility identity may need source-owner correction. Conversely, a review case may be created before anything fails: an access-rule change, a public-hours notice, a refund recommendation, or an AI-drafted customer response may require approval by policy.

Link the two systems when needed, but do not collapse them. The exception ledger answers, “What needs accountable resolution?” The review queue answers, “Which decision must this qualified person make, from which evidence, by when, and what state follows?”

Define one reviewable unit

The reviewable unit is not a message, model output, or alert. It is one bounded decision case.

Every case should identify:

• a stable review-case ID;
• the facility, entity, and actor scope;
• the decision the reviewer is being asked to make;
• the proposed action and prohibited alternatives;
• the policy and consequence class that require review;
• why the automation deferred;
• the evidence packet and its freshness limits;
• the required reviewer role and separation-of-duties rule;
• the decision deadline and escalation path;
• the allowable outcomes;
• the execution boundary; and
• the governing readback required for closure.

If two proposed actions can be approved independently, they are two cases. If several model outputs support one indivisible decision, they belong in one evidence packet. The case boundary should follow the operating decision, not the number of technical events.

Admission control comes before prioritization

A queue cannot prioritize its way out of unlimited intake. Decide first which cases are allowed to enter.

Route	Use when	Required behavior
Automatic	Policy permits a bounded, reversible action with complete evidence	Execute, retain evidence, reconcile, and sample for monitoring
Human review	A qualified decision is required and the packet is reviewable	Admit one bounded case with deadline and owner
Needs evidence	The decision is defined but required evidence is missing, stale, or conflicted	Do not occupy active reviewer capacity; route to the source owner
Prohibited	Policy disallows the action regardless of reviewer preference	Deny at enforcement and record the reason
Incident or exception	The workflow is failing rather than awaiting adjudication	Route to the incident or exception owner; create a review case only for a real decision

“Low confidence” alone is not an admission rule. Confidence may be poorly calibrated, irrelevant to the consequence, or high even when a required source is stale. Admission should be based on policy, evidence completeness, consequence, reversibility, and the exact decision required.

Build an evidence packet, not a persuasion packet

The packet should help the reviewer test the proposal. It should not be arranged only to make approval easy.

A balanced packet includes:

• the proposed decision in plain language;
• the machine's reason for deferring;
• the governing source references and observation times;
• missing, stale, conflicting, or excluded evidence;
• the applicable policy version and effective time;
• the consequence and reversibility assessment;
• the model, rule, prompt, retrieval, or workflow version when relevant;
• allowed decision outcomes;
• the effect of taking no action before the deadline;
• privacy or sensitivity markings;
• the execution and rollback boundaries; and
• links to the source records, not only a generated summary.

Show contradictory evidence beside supporting evidence. Show an expired source as expired. If the automation cannot state why it stopped, the case is not ready for adjudication.

W3C PROV-O provides general concepts for entities, activities, agents, derivation, attribution, association, plans, and delegation. Those concepts can help represent the packet's provenance. They do not determine which self-storage source is authoritative or which reviewer may approve an action.

Use a state machine that preserves waiting

A queue state should explain what the case is waiting for.

The proposed core states are:

1. received — a candidate case exists but has not been routed;
2. triaged — consequence, policy, duplication, and destination were evaluated;
3. needs_evidence — required evidence is missing, stale, or conflicted;
4. ready — the packet is complete and a qualified queue can accept it;
5. claimed — one qualified reviewer owns the decision;
6. escalated — the original queue cannot decide within authority or time;
7. approved or denied — the adjudication is recorded;
8. expired or withdrawn — the decision is no longer valid or required;
9. executed_pending_reconciliation — the approved action has a receipt but lacks governing readback;
10. reconciled — the governing source matches the authorized result; and
11. reopened — new evidence, an appeal, a mismatch, or a policy change invalidated closure.

Do not reset the original received_at time when a case moves between queues. Preserve every transition with actor, role, time, reason, and prior state. Reassignment changes ownership, not history.

Priority is a policy decision

First-in, first-out is simple, but it can let a low-consequence draft delay a time-sensitive access decision. A raw model score is worse: it makes the system's confidence a proxy for operating urgency.

A transparent priority rule can consider:

• consequence class;
• time until consequence or opportunity expiry;
• reversibility;
• number and type of affected people, facilities, or records;
• financial, access, privacy, safety, or public-information sensitivity;
• whether an alternative safe state exists;
• current age and prior escalation; and
• the qualified reviewer pool available.

Publish the rule as a versioned policy. Use arrival time as a tie-breaker inside a consequence class. Monitor whether one facility, language, customer cohort, or decision type is repeatedly delayed. A queue can be internally consistent and still produce unfair service.

Preserve four clocks

One average handling-time metric hides why work is late. Preserve four clocks:

• Total age: from first receipt to the current time. This never resets.
• Ready age: time spent with a complete packet waiting for a qualified reviewer.
• Active review time: time a reviewer actually held the case, excluding a clearly recorded evidence wait.
• Consequence deadline: the latest time at which the decision remains useful or safe under the governing policy.

An evidence wait is not invisible. It has a source owner and its own timer. A case that spends two hours awaiting a missing facility-hours source and three minutes in active review did not have a three-minute cycle time.

Use percentile distributions and oldest-case age by consequence class. A portfolio average can remain stable while one high-consequence case becomes dangerously old.

Capacity is part of the safety design

Review capacity is finite. The design should say what happens when intake exceeds qualified service capacity.

John D. C. Little's 1961 queueing result relates the long-run average number of units in a stationary system, the effective arrival rate, and the average time in the system: L = λW. The result has conditions; it is not a staffing formula or a promise about a transient queue. It does provide a useful conservation check. If completed throughput remains roughly fixed while work in progress grows, average time in the system cannot remain unchanged indefinitely.

For an operating queue:

• measure arrivals, exits, reopens, and active work in progress by class;
• set a work-in-process limit for each qualified reviewer pool;
• stop low-value admission before the queue is saturated;
• route incomplete cases out of active review capacity;
• reserve capacity for high-consequence or time-critical work;
• make upstream automation degrade safely when review capacity is exhausted; and
• rehearse overload, absence, reassignment, and provider-failure conditions.

Google SRE guidance on cascading failures describes load shedding and queue-length controls as ways to prevent overload from consuming the service. A human-review queue is not a server, but the operating lesson transfers carefully: unbounded waiting is not graceful degradation. The safe degraded mode may be to stop proposing low-consequence work, preserve the current facility state, or require manual initiation.

A reviewer needs authority, qualification, and independence

Assign to a role, then verify the person currently satisfies it.

The assignment check should answer:

• Is the reviewer authenticated?
• Is the reviewer scoped to this facility, region, or decision type?
• Is the policy and training current?
• Does the decision require a specialist qualification?
• Did this reviewer create the proposal or control the evidence source?
• Does separation of duties require a second person?
• Is a conflict, recusal, or temporary delegation recorded?
• Can the reviewer stop execution and request more evidence?

NIST SP 800-53 Revision 5 includes general controls for separation of duties, least privilege, and audit review. These security and privacy controls can inform role design. They do not define self-storage business authority or make every decision a two-person approval.

The interface must make disagreement possible

A review screen should not present one large “Approve” button and a buried alternative.

At minimum, expose:

• the decision and consequence at the top;
• the original queue age and deadline;
• supporting, contradictory, stale, and missing evidence;
• the policy and reviewer authority;
• approve, deny, request evidence, escalate, recuse, and withdraw paths when allowed;
• the effect of no decision;
• the exact action that approval authorizes;
• the rollback owner and limit;
• a required reason code plus bounded notes; and
• the post-decision receipt and reconciliation state.

Do not use a default selection for a consequential decision. Do not let a generated explanation hide the original records. Do not present confidence as certainty. A mobile reviewer must see the same decision boundary and missing-evidence state as a desktop reviewer.

Decision, execution, and closure are three states

An approval authorizes an action. It does not prove that the action occurred.

Preserve:

1. the review decision and evidence packet version;
2. the authorized action request;
3. the receiving system's acceptance or rejection;
4. execution evidence;
5. governing-source or public-user-path readback;
6. match, mismatch, missing, or stale reconciliation state; and
7. reopen, rollback, or escalation when the result does not match.

A queue that closes at approval optimizes the click, not the outcome. A queue that closes at provider acceptance optimizes the receipt. The operating case closes only at its declared terminal state.

Measure the queue without rewarding rubber stamps

A review dashboard should not rank people by approvals per hour. That encourages speed and agreement regardless of consequence.

Use a balanced set of indicators:

• candidate cases by route: automatic, review, needs evidence, prohibited, incident;
• admitted cases by consequence and decision type;
• duplicate suppression and invalid-case rate;
• evidence-complete rate at admission;
• total age, ready age, active time, and oldest case;
• deadline breach and escalation rate;
• approve, deny, request-evidence, recuse, expire, and withdraw outcomes;
• reopen and appeal rate;
• execution and reconciliation completion;
• mismatch, rollback, and unresolved-result counts;
• work in progress, arrival rate, and completed throughput; and
• measurement coverage and unknown cases.

Segment by consequence, facility, workflow, reviewer pool, and policy version. Do not publish an “accuracy” or “productivity” result unless the ground truth, population, period, exclusions, reviewer independence, and limitations are defined.

Five failure tests

1. Evidence flood: remove one required source and generate 100 candidates. Do incomplete cases consume active reviewer capacity?
2. Priority inversion: place a low-consequence draft ahead of a time-sensitive access case. Does policy reorder them transparently?
3. Age reset: reassign an old case twice. Does the original receipt time remain visible?
4. Authority gap: assign a qualified reviewer outside the facility scope. Does enforcement reject the decision?
5. False closure: approve a public-hours change but make the public readback disagree. Does the case reopen instead of closing green?

A fictional multi-location tabletop

The worksheet includes five fictional cases. They describe no real customer, facility, provider, policy, deployment, or result.

A temporary-hours notice has complete evidence and waits in a regional operations queue. An access-rule proposal is time-sensitive but lacks a current authority record, so it remains needs_evidence rather than occupying an access reviewer. A duplicate refund recommendation collapses under one business case. A maintenance dispatch is approved but remains open pending governing work-order readback. A low-consequence response draft expires during a simulated capacity event without displacing higher-consequence work.

Run the exercise with an operator, a source owner, a reviewer, and a technical owner:

1. Validate the admission route for each case.
2. Confirm the evidence packet supports disagreement.
3. Calculate which clock is running.
4. Apply the priority policy without reading the model confidence first.
5. Remove one reviewer and enforce the work-in-progress limit.
6. Approve one case, then fail execution.
7. Execute another, then create a reconciliation mismatch.
8. Reassign the oldest case and confirm its age does not reset.
9. Ask another operator to reconstruct every state transition.

The queue contract

Before releasing a human-review queue, require a written contract that names:

• the decisions the queue may contain;
• the decisions that remain prohibited;
• admission and evidence-completeness rules;
• consequence classes and priority policy;
• qualified reviewer roles and separation of duties;
• state transitions and terminal states;
• the four clocks and escalation deadlines;
• work-in-progress and overload behavior;
• decision outcomes, reason codes, and appeal paths;
• execution, reconciliation, rollback, and reopening rules;
• privacy, retention, and audit access;
• measurement definitions and unknown-data treatment; and
• the owner authorized to change the contract.

Run the worksheet

Download the human-review queue design worksheet. It contains five fictional cases and 56 governed fields from beginner intake through architect-level capacity, provenance, execution, and reconciliation controls.

A beginner can start with the decision, facility, consequence, evidence state, deadline, reviewer role, and next action. An operator can add queue state, age, escalation, and source ownership. An architect can complete the policy, model, provenance, separation-of-duties, work-in-progress, execution, reconciliation, privacy, and retention fields.

Sources and limitations

Observed August 22, 2026:

• NIST AI RMF Core, excerpted from AI RMF 1.0, for documented human-AI roles, oversight, prioritization, monitoring, appeal, override, incident response, and recovery. NIST states that AI RMF 1.0 is being revised.
• NIST AI RMF Appendix C, for limitations and variability in human-AI interaction, role differentiation, context loss, and human bias.
• NIST AI 600-1, Generative AI Profile, published July 26, 2024 and updated April 8, 2026, for cross-sectoral GAI risk actions including documented roles, monitoring, incident response, and ownership.
• NIST SP 800-53 Revision 5, for general separation-of-duties, least-privilege, and audit-review control concepts.
• Google SRE: Addressing Cascading Failures, for technical queue-length control, overload, load shedding, and graceful degradation concepts.
• John D. C. Little, “A Proof for the Queuing Formula: L = λW”, published in Operations Research in 1961, for the stationary-system relationship among average work in progress, effective arrival rate, and average time in the system.
• W3C PROV-O, a W3C Recommendation dated April 30, 2013, for general provenance relationships among entities, activities, agents, plans, roles, attribution, association, and delegation.

These sources inform the proposed method but do not define a self-storage review queue, staffing level, decision deadline, service commitment, product architecture, or certification. NIST AI resources are voluntary and cross-sectoral; AI RMF 1.0 is under revision. SP 800-53 addresses security and privacy controls, not self-storage business authority. Google SRE describes software-service overload, not human staffing. Little's result has stationary and finite-mean conditions and does not by itself determine transient capacity or queue discipline. PROV-O represents provenance but does not choose authoritative sources or decision rights. The states, clocks, admission routes, metrics, worksheet, and fictional tabletop are proposed here and require local operational, technical, source-owner, privacy, security, legal, accessibility, and workforce review.

Disclosure

I am Chief Operating Officer of modSTORAGE and CEO and Co-Founder of Facily.ai. This paper is an authored operating method. It is not a description of a live review queue, product capability, customer deployment, staffing result, service level, legal conclusion, certification, or measured operating outcome.