The Agent Said It Was Done. I Still Need to See the Result.

Part II: Verification, human control, and an architecture that keeps errors from multiplying

What this article covers

An agent can finish a task and leave behind a mistake that the next agent treats as a fact. I look at how that happens, what evidence I want before accepting the work and where human control can actually change the outcome. Model review helps, but I want the system to check its work against something outside its own explanation.

Full article

How fast can one reasonable mistake travel?

In Part I, I wrote about contracts, ownership, and state outside the conversation. Those boundaries make the system understandable. They do not prove the output is correct.

Now imagine one agent writes a plausible rule into a module. A second agent writes tests around that behavior. A third updates the documentation. The result enters a knowledge base. A week later, another agent retrieves the document and cites it as project policy.

By then, finding the original assumption is difficult. The code, the tests, and the documentation all seem to support one another, even though they grew from the same mistake.

This is the real scaling problem. Models make mistakes, and people make mistakes too. The difference is the speed at which an agent can turn one wrong assumption into code, tests, reports, records, and instructions. Generation takes minutes. Careful verification usually takes longer.

When output grows faster than checking capacity, you accumulate verification debt: artifacts that look finished but whose meaning nobody has actually confirmed.

Microsoft Research built DELEGATE-52 to examine long delegated editing workflows across 52 professional domains. In its reported experiment with 19 models, even frontier systems corrupted an average of 25% of document content by the end of long workflows. Larger documents, longer interaction, and distractor files made the degradation worse. The point is not that every office document will lose exactly one quarter of its meaning. The point is that locally acceptable edits can compound into a damaged result while the interface keeps looking calm.

The same problem can fit inside a perfectly typed JSON response. The schema confirms that status exists; checking whether that status describes the work is another task.

What counts as proof of completion?

Asking an agent to "check your work" can be useful as one additional pass. It is weak as the foundation of reliability because the model may reuse the same mistaken picture that produced the answer.

A critical survey in TACL found little evidence that prompted models reliably correct themselves without a useful outside signal, except in tasks unusually suited to self-correction. Results improve when the system can use reliable external feedback.

I would start by asking what evidence could expose the mistake without relying on the explanation that produced it.

First, use deterministic checks where they exist: schemas, types, arithmetic, compilation, access rules, business constraints, duplicate detection.

Then inspect the external state. Run the code and its relevant tests. Open the document the agent actually created. Query a changed database record and compare it with the earlier state. A submitted form needs a receipt or confirmed status; a factual claim needs a primary source. For a sent message, inspect the recipient, final body, and delivery state.

After that, a model reviewer can scale preliminary semantic review. It can find omissions, contradictions, and scope drift. Give it explicit criteria and evidence. Calibrate it against human judgments. A review needs something to challenge the original answer with. A different prompt alone may leave the same mistaken assumption in place.

Then use a human where reliable ground truth is missing, values conflict, consequences are serious, or somebody must accept residual risk.

When I read "task complete," I want to be able to follow it to the result: the test, the changed record, the actual file, the source, or the delivery confirmation. The report helps me find the evidence. Then I can decide whether that evidence answers the task.

Work passes through verification before entering durable state; control and human review govern the transition.

Can several agents argue their way to truth?

The idea is attractive: let several agents debate, vote, and correct one another.

Sometimes multiple attempts help. Independent review can expose different failure modes. But agreement is not truth. Similar models using similar sources can repeat the same bias. A majority vote shows consistency among samples, not contact with reality.

The 2024 ICML paper "Should We Be Going MAD?" tested several multi-agent debate protocols and found that they did not reliably beat simpler strategies such as self-consistency and ensembles of reasoning paths. Some debate systems improved after careful tuning, which makes the result more interesting, not less: debate is a method with conditions and costs, not a universal accuracy switch.

A 2026 EACL study examined problem drift in longer debates. The discussion can move away from the original question through low-quality feedback, lack of progress, and unclear reasoning. More rounds can produce more text while making the actual task smaller in everyone's attention.

If the disagreement concerns a number we can calculate or a record we can inspect, I would do that before another round of debate. It gives the discussion something concrete to work from.

Use a council only when a real tradeoff has several defensible answers. Keep it bounded:

Ask one precise question.
Give every participant the same decision criteria.
Collect independent initial positions before cross-reading.
Separate claims, evidence, assumptions, and contradictions.
Allow one targeted rebuttal on material disagreements.
Use an arbiter with a declared decision rule.
Stop after a fixed round and preserve the minority position.

Do not force consensus. Sometimes the correct output is a clear disagreement and a human decision.

Where does a human actually help?

Putting an approval button before every tool call sounds safe. Soon the person clicks it mechanically because the system asks too often and explains too little.

At that point, the button is still there, but the person is no longer examining each decision. I would rather spend that attention where it can change the outcome.

A useful human checkpoint has four properties:

The decision can materially change the result.
The action is difficult to reverse or affects people, money, permissions, private data, or public channels.
The reviewer receives a compact evidence package, not the entire execution log.
The reviewer can edit, reject, roll back, or stop the process.

Drafting an email can be autonomous. Sending it to a named person crosses a different boundary. Research can run for hours. Publishing the conclusion under somebody's name requires a separate decision. A system may prepare a payment, permission change, or deletion plan. Execution belongs behind a hard gate.

OWASP's guidance on excessive agency makes the engineering version of the same point: minimize tool functionality and permissions, avoid open-ended tools where possible, and enforce authorization in downstream systems. Anthropic's published containment work gives an even sharper example. In one controlled red-team exercise, a malicious user-delivered prompt caused credential exfiltration in 24 of 25 retries of that same prompt. Their conclusion concerned the environment: filesystem and network boundaries must block the action regardless of what the model or user appears to intend.

The written instruction and the environment have different jobs here. The instruction explains what should happen; the environment limits what the operation can actually do.

The reviewer also needs enough independent practice to recognize when the whole chain is answering the wrong question. How that competence will be formed and maintained deserves a separate article.

How would I build the system today?

For a system like the accounting project, this is the sequence I would work through.

Decision	What must be explicit
Define the accepted outcome	Do not start with "research the topic" or "build the application." Name the artifact, its user, required properties, prohibited effects, and the evidence that will prove completion.
Check whether an agent is necessary	If the rule can be written as code, write code. If the path is known, build a workflow. Use an agent when the next step genuinely depends on observation and cannot be fully specified in advance.
Divide work by artifacts	Each stage should create an inspectable object: a source ledger, plan, patch, report, table, diff, or decision record. "The agent worked on it" is not an artifact.
Assign one owner at every shared boundary	Use one writer for mutable state, one merge owner, and one process for resolving conflicts. Shared memory without ownership becomes a room where everybody left a note and nobody knows which note is current.
Enforce permissions outside the prompt	Separate read, write, send, publish, pay, and delete. Use scoped credentials, isolated workspaces, network rules, and explicit downstream authorization. The prompt can explain a boundary. The environment must enforce it.
Verify material transitions	Do not place an expensive verifier after every sentence. Check where an error would enter shared state, trigger an external effect, contaminate the next stage, or make rework costly.
Bound every loop	Set maximum iterations, time, cost, no-progress conditions, final states, and an escalation path. "Try again" is not a recovery policy.
Compare against a simple baseline	A multi-agent system should beat one good workflow or one tool-using agent on accepted outcome, coverage, time, or total cost. If it does not, the extra architecture has no job.
Turn incidents into tests	Every rejected output, real human correction, and production incident should add a case to the regression suite. That preserves the correction in a form the next version can be checked against, instead of leaving it buried in an old conversation.

Why is architecture the work now?

When I started that accounting system, I thought the main value of agents was faster code. Later I understood that speed was only the entry condition.

The real work was defining module boundaries, preserving sources of truth, separating permissions, recording decisions, checking the transitions where mistakes spread, and preventing the same agent from writing reality and certifying that reality as correct.

That adds work at the beginning. But once an assumption has spread into several modules, finding and correcting all of its consequences takes work too.

What I want from that system is fairly practical. I want to see what changed, what supports the change, and what remains unresolved. Then I can let the next stage continue without having to reconstruct the whole history again.

Practical takeaways

Define completion through external evidence, not the agent's report about itself.
Use deterministic checks first, then inspect the actual environment, then add model and human review where needed.
Treat model debate as a specialized method, not a source of truth.
Put human approval at consequential, reversible-by-design boundaries and provide a compact evidence package.
Enforce permissions in tools, credentials, sandboxes, and downstream systems.
Limit every loop by time, cost, iterations, progress, and final state.
Convert every serious failure and human correction into a regression test.

In brief

I want to follow a completion report to a result I can inspect. As the system grows, that means keeping evidence with the work, containing errors before the next stage uses them and leaving a person a real opportunity to stop or change a consequential decision.

Denis Ostapenko · ORCID 0009-0006-2630-7280