Table of Contents
- The Compounding Arithmetic
- Where Agents Actually Break
- Workflows Beat Agents Most of the Time
- Tool Design Determines Success
- Making Actions Safe to Retry
- Approval Gates and Where to Place Them
- Observability for Non-Deterministic Systems
- Scoping an Agent That Ships
- Common Pitfalls
- Conclusion
- Frequently Asked Questions
Key takeaway: Reliability multiplies rather than averages across steps. The most effective interventions are reducing the number of autonomous decisions, designing tools that fail loudly, and placing human approval before irreversible actions.
The Compounding Arithmetic
An agent that performs each step correctly 95 percent of the time sounds reliable. Chain ten steps and the probability of completing all of them correctly is 0.95 to the tenth power — roughly 60 percent.
Twenty steps brings it to 36 percent. Thirty steps to 21 percent.
This is the central fact of agent engineering and it is frequently discovered empirically rather than anticipated. Teams build a demonstration that works, deploy it, and find that it fails a third of the time for reasons that seem to vary. The reason does vary — a different step fails on each occasion — while the aggregate failure rate is entirely predictable from the arithmetic.
Two consequences follow immediately.
First, per-step reliability must be extremely high for long chains to work at all. Reaching 90 percent end-to-end across ten steps requires 99 percent per step, which is a substantially harder engineering target than it appears.
Second, and more usefully: reducing step count improves reliability faster than improving per-step accuracy. Cutting a workflow from twelve steps to five does more for success rate than any amount of prompt refinement on the twelve.
This reframes the work. The goal is not a better agent. It is a smaller one.
Where Agents Actually Break
The failure modes are specific and mostly not what teams expect.
Wrong tool selection. With many available tools, the agent picks a plausible but incorrect one. This worsens sharply as the tool count grows, because tool descriptions begin to overlap.
Malformed arguments. The right tool called with wrong parameters — a misformatted date, a hallucinated identifier, a required field omitted. Schema-constrained tool calling addresses much of this.
Silent misinterpretation of results. The tool returns data the agent misreads, then proceeds confidently on a wrong premise. This is the most dangerous category because nothing errors.
Loops. The agent repeats an action, sometimes with minor variations, without progressing. Without a step limit this consumes budget indefinitely.
Premature completion. The agent declares success having done part of the task. Particularly common when the completion criterion is vague.
Context loss over long chains. Early decisions and constraints fall out of effective attention, so later steps contradict earlier ones.
Cascading errors. An early mistake produces plausible intermediate output, and subsequent steps build correctly on a wrong foundation. The final result is coherent and wrong.
That last pattern is worth dwelling on, because it explains why agent failures are hard to detect. A crashed pipeline is obvious. An agent that produced a complete, well-formatted, internally consistent wrong answer requires someone to check the work.
Workflows Beat Agents Most of the Time
An important distinction that determines architecture:
A workflow has predetermined steps. The model performs specific tasks within a structure you defined — classify this, extract that, draft this text. Control flow is code.
An agent decides its own steps. It plans, selects tools, evaluates results, and determines when it is finished. Control flow is model output.
Agents are more flexible and dramatically less reliable, because every decision they make is a place to be wrong. The reliability arithmetic applies to the decisions, and a workflow has none.
The practical guidance is to use a workflow wherever the steps are knowable, and reserve agent autonomy for the parts that genuinely cannot be enumerated in advance.
| Use a workflow when | Use an agent when |
|---|---|
| Steps are known in advance | The path genuinely varies per input |
| Reliability is important | Flexibility outweighs predictability |
| Debugging matters | Exploration is the task |
| Cost must be predictable | Variable cost is acceptable |
Most tasks presented as requiring an agent turn out, on examination, to have four or five knowable steps with one genuinely variable decision inside. Encoding the known structure in code and using the model for the single variable decision produces something that works, and it is less interesting to build than an autonomous agent — which is a real reason teams avoid it.
Tool Design Determines Success
Tool interfaces affect reliability more than prompt wording, and this is consistently underappreciated.
Keep the tool count low. Beyond roughly ten tools, selection accuracy degrades noticeably. If you have thirty, group them behind a smaller number of higher-level tools, or route to a subset based on task type before invoking the agent.
Make descriptions disambiguating rather than merely descriptive. The description exists to distinguish this tool from the others. search_orders and find_purchases will be confused; stating explicitly when each applies resolves it.
Constrain arguments with schemas. Enums over free strings, required fields marked, formats specified. This eliminates the malformed-argument failure class structurally.
Return errors the model can act on. Error 400 teaches nothing. Invalid date format. Expected YYYY-MM-DD, received "next Tuesday". Use the resolve_date tool first. enables recovery on the next attempt.
Include result metadata. Whether the result set was truncated, how many total matches existed, whether the operation was partial. Agents misinterpret silently-truncated results as complete ones.
Fail loudly on ambiguity. A tool that guesses when input is unclear propagates the guess. Returning an explicit ambiguity error lets the agent clarify.
# Poor: the agent cannot recover from this
def get_order(order_id: str) -> dict:
return db.query(order_id) # raises, or returns None silently
# Better: actionable errors and explicit metadata
def get_order(order_id: str) -> dict:
"""Retrieve a single order by its exact ID (format: ORD-NNNNNN).
For searching by customer or date, use search_orders instead."""
if not re.match(r"^ORD-\d{6}$", order_id):
return {"error": "invalid_format",
"message": f"Expected ORD-NNNNNN, got '{order_id}'. "
"Use search_orders to find an ID."}
order = db.query(order_id)
if order is None:
return {"error": "not_found",
"message": f"No order {order_id} exists.",
"suggestion": "Verify the ID or use search_orders."}
return {"status": "ok", "order": order}Making Actions Safe to Retry
Agents retry. If retrying an action repeats its side effects, retries cause damage.
Make write operations idempotent. Accept a client-supplied idempotency key and return the original result on repeat rather than performing the action again. This single property converts retries from dangerous to safe.
Separate reads from writes in tool design. Read tools can be retried freely. Write tools need protection. Mixing both in one tool means the safe operation inherits the unsafe one’s constraints.
Prefer proposals to executions. A tool that returns “here is the email I would send” and a separate tool that sends it splits the decision from the action, which permits inspection between them.
Implement compensating actions where possible. If an agent can create something, it should be able to undo it. Not every action is reversible, and knowing which are is necessary for designing the approval gates.
Enforce hard limits. Maximum steps, maximum tool calls, maximum spend, maximum wall-clock time. An agent without limits can consume unbounded resources, and loops are common enough that this is not hypothetical.
Approval Gates and Where to Place Them
Full autonomy is rarely the right design for anything with consequences. The question is where to require a human.
Place a gate before any action that is irreversible, externally visible, financially material, or legally consequential. Sending communications to customers, moving money, deleting data, modifying production configuration, and committing to obligations all qualify.
Do not gate reversible internal operations — reading data, drafting content, computing analysis, populating a staging area. Gating everything produces approval fatigue, and a human clicking approve on forty low-stakes actions will click approve on the forty-first without reading it.
Design of the gate matters as much as its placement. Show what will happen and why, not merely a request to confirm. Show the reasoning chain that led here. Make rejection informative, capturing why the human declined so the failure can be analysed. And batch related approvals rather than interrupting repeatedly for each step of one logical operation.
The progression that works in practice is graduated autonomy: begin with approval on every action, measure which categories are approved essentially always, and remove gates from those categories specifically. This is data-driven rather than optimistic, and it converges on the right set of gates rather than guessing them upfront.
Observability for Non-Deterministic Systems
Debugging an agent requires records that conventional application logging does not produce.
Capture per step: the model’s reasoning output, which tool was selected and why, the exact arguments passed, the complete result returned, the elapsed time, and the token cost. Then link all steps to a single trace identifier for the overall task.
This is more verbose than typical logging and it is the only way to answer the question that matters — where did this go wrong? An agent that produced a wrong final answer gives no indication of which of its twelve steps introduced the error. The trace does.
Metrics worth tracking at the aggregate level:
Task completion rate. Genuinely completed, verified against the actual goal rather than the agent’s self-report.
Steps per task, distributed. A rising tail indicates inefficiency or looping.
Tool error rate, per tool. Identifies which tools are confusing or poorly specified.
Human intervention rate. How often a person had to correct or take over.
Cost per completed task. The number that determines viability.
Self-reported success requires particular scepticism. Agents declare completion when they believe they are finished, and that belief is exactly the thing that fails. Verification must be external — check the actual state of the world rather than asking the agent whether it succeeded.
Scoping an Agent That Ships
Characteristics of tasks where agents work:
Bounded action space. A small number of tools, all well-specified.
Verifiable success. You can check completion programmatically rather than by judgement.
Reversible or gated actions. Mistakes can be undone or are caught before taking effect.
Tolerance for latency. Multi-step reasoning is slow. Interactive use cases fit poorly.
Genuine path variation. If the steps are fixed, a workflow is strictly better.
Characteristics that predict failure: unbounded tool access, subjective success criteria, irreversible actions without gates, real-time latency requirements, and tasks where a wrong answer is expensive and hard to detect.
The most reliable design pattern remains a narrow agent operating inside a broader workflow — code handles the known structure, the agent handles one genuinely variable decision, and the result is verified before proceeding.
Common Pitfalls
Building an agent where a workflow suffices. Every autonomous decision is a failure opportunity.
Too many tools. Selection accuracy degrades past roughly ten.
Unactionable tool errors. The agent cannot recover from a message it cannot interpret.
Non-idempotent write operations. Retries cause duplicate side effects.
No step or cost limits. Loops consume unbounded resources.
Trusting self-reported completion. Verify against actual state.
Gating everything. Produces approval fatigue and defeats the purpose of the gates that matter.
Conclusion
Agent reliability is governed by multiplication. Ten steps at 95 percent is 60 percent, and no amount of prompt engineering changes the arithmetic — only reducing the step count or raising per-step reliability does.
That makes the highest-return interventions structural. Replace autonomous decisions with code wherever the steps are knowable. Keep the tool count small and the descriptions disambiguating. Return errors the model can act on. Make writes idempotent so retries are safe. Gate the irreversible actions and only those. Trace every step, and verify completion against the world rather than the agent’s opinion of itself.
The agents that work in production are narrower than the demonstrations suggest. That is not a limitation to be overcome; it is what reliability at multi-step autonomy currently requires.
Frequently Asked Questions
How many steps can an agent reliably handle? With well-designed tools, roughly five to ten before compounding failure dominates. Beyond that, decompose into separate verified stages rather than one long chain.
Should agents be allowed to write to production systems? Only through idempotent operations, with approval gates on anything irreversible, and with complete audit logging. Unrestricted production write access is difficult to justify.
How is an agent tested? A fixed set of tasks with programmatically verifiable outcomes, run repeatedly to measure the success distribution rather than a single pass. Non-determinism means one successful run proves very little.
What causes agents to loop? Usually a tool returning results the agent cannot interpret as progress, so it retries with variations. Better error messages and result metadata resolve most cases; step limits contain the rest.
Is a multi-agent architecture more reliable than one agent? Generally less reliable, because inter-agent communication introduces additional failure points. It can help by narrowing each agent’s tool set and responsibility, which is the actual benefit — specialisation rather than the multiplicity.
How should agent costs be controlled? Hard caps on steps, tool calls, and tokens per task, enforced in code rather than requested in the prompt. Monitor cost per completed task, since cost per call can look reasonable while tasks require many calls.
When is full autonomy appropriate? When actions are reversible, success is verifiable, and the cost of an undetected error is low. Content drafting, data exploration, and internal analysis fit. Anything customer-facing or financial generally does not.
