One axis, not three generations
The marketing arranges these as a timeline: rules-based automation, then generative AI, then agents, each obsoleting the one before. That framing is wrong in a way that costs money, because it suggests the newest option is the best option and the older ones are legacy.
What actually separates them is two properties of the work itself.
- How much variance the input contains. A fixed-format file arriving in the same place every night has none. An email from a customer describing a problem in their own words has a great deal.
- How many steps must complete without a person checking in between. Reading a document and returning a summary is one step. Reading it, deciding which of four systems it belongs in, creating a record, notifying someone and following up is five.
Rules-based automation handles no variance and any number of steps, deterministically. Generative AI handles enormous variance in a single step. An agent attempts both at once — variance and multiple steps — and that combination is where the difficulty lives, not in the model.
What agents actually score
Carnegie Mellon researchers built TheAgentCompany, a simulated software firm, and set agents to ordinary knowledge work: browsing, writing code, running applications, messaging colleagues. The best model in that evaluation completed 30.3% of tasks outright, rising to 39.3% on a metric giving partial credit. Several widely deployed models scored in single digits. Salesforce's own CRMArena-Pro benchmark found roughly 58% success on single-turn scenarios and about 35% on multi-turn ones.
Both benchmarks were published in mid-2025 and frontier models have improved substantially since. Quoting them as current scores would be dishonest, and anyone who shows you a 2025 agent benchmark as though it describes 2026 systems is making an argument rather than a measurement. What has not changed is the shape of the finding: multi-step performance falls off far faster than single-step performance, across every model and every benchmark. The absolute numbers move. The gradient does not.
The arithmetic nobody puts on the slide
Here is why that gradient exists, and it is not a limitation of any particular model. If each step in a task succeeds independently with probability p, and every step has to land for the task to be complete, end-to-end success is p raised to the number of steps.
At 95% per step — which is very good, better than most production agents manage on real systems — a five-step task completes 77% of the time and a twelve-step task 54%. At 90% per step, twelve steps gets you 28%. The chart at the top of this article is that function, nothing more.
Three consequences follow directly, and they are more useful than any vendor comparison table.
- Shortening the chain beats improving the model. Going from twelve steps to four does more for reliability than a model upgrade usually will, and you control the step count.
- A checkpoint resets the compounding. A person approving at step four means the next sequence starts from certainty rather than from 0.63. This is why human-in-the-loop designs outperform fully autonomous ones in practice, and it is a mathematical reason rather than a cultural one.
- Deterministic steps should stay deterministic. Every step you hand to a model is a step with a failure probability. A step done by a rule has, for practical purposes, none. Mixing the two is not a compromise; it is the correct design.
When plain automation is still the right answer
This is the unfashionable part of the conversation and usually the most valuable. A large share of what gets proposed as an agent project is a scheduled job, an API integration, or a form with validation on it.
- → The input arrives in a known format — a file, an API payload, a form submission
- → The logic is expressible as conditions, even if there are a lot of them
- → You need the same input to produce the same output every time, and to prove it did
- → The process is regulated and every decision must be reconstructable years later
- → Screen-scraping bots break when a vendor moves a button. API-level integration does not — prefer it wherever an API exists
- → Every new exception becomes another branch, and the rule set eventually outgrows anyone's understanding of it
- → It cannot read a sentence. Anything that requires interpreting unstructured language is out of scope, however many rules you add
- → Maintenance is permanent and usually unbudgeted
The honest summary: deterministic automation is cheap to run, trivially auditable, and fails in obvious ways at obvious moments. Those are valuable properties, and in finance, compliance and anything with a regulator attached they are often non-negotiable.
When generative AI alone is enough
This is the most under-used of the three, because it is less exciting to announce than an agent and more capable than most people assume.
A single model call, with retrieval where it needs facts, handles an enormous amount of real work: classifying an inbound message, extracting structured fields from an unstructured document, drafting a reply for a person to send, summarising a long thread, translating free text into a structured record that a deterministic system then processes.
The property that makes this safe is that a human sees the output before anything acts on it, or a deterministic system validates it against a schema. There is one step, so nothing compounds. When it is wrong, it is wrong visibly and once, rather than silently and five times in a chain.
A pattern worth naming, because it covers more ground than either pure approach: use the model to turn messy input into structured data, then let conventional code do the work. The variance is absorbed at the boundary, and everything downstream stays deterministic, testable and auditable.
When an agent earns its cost
Agents are the right answer when the task genuinely requires choosing what to do next based on what just happened — not merely executing a known sequence, which is automation with extra latency and a bill attached.
The conditions that make one work are specific, and projects that skip them are the ones in Gartner's 40%.
| Condition | Why it matters | If it is missing |
|---|---|---|
| The path genuinely varies | Otherwise you are paying a model to rediscover a fixed sequence on every run | Write the sequence down and automate it |
| Few steps, or checkpoints between them | Compounding is the dominant failure mode, and step count is the lever you control | Split the process and put a person or a validation gate at the seam |
| Errors are visible and recoverable | A wrong draft is cheap. A wrong payment, a wrong deletion or a wrong message to a customer is not | Keep the final action human, and let the agent prepare it |
| Tools are real APIs with real error handling | Most agent failures in production are tool failures, not reasoning failures | Fix the integrations first — that work is required either way |
| You can evaluate it | Without a test set of real cases you cannot tell improvement from luck | Do not start. This is the one that is genuinely disqualifying |
If you are building one, the architecture and governance layers are a subject in themselves — we covered them in building enterprise AI agents, and the broader comparison with predictive machine learning sits in agentic AI versus traditional AI.
Agent washing, and the questions that cut through it
Gartner uses the term agent washing for vendors rebranding existing assistants, chatbots and RPA products as agentic without the underlying capability, and estimates that only around 130 of the thousands of vendors claiming agentic AI are genuinely doing it. That ratio means the default assumption about any given claim should be scepticism.
Four questions separate the two quickly, and none of them require you to be technical.
Running the decision on a real process
The fastest way to settle this for any specific process is to map it rather than to debate it.
Write down the steps, honestly
Every one, including the lookups people do without noticing and the judgement calls nobody documented. The step count you end up with is the single most predictive number in this exercise, and it is always higher than the first draft.
Mark each step structured or unstructured
Structured steps go to code or rules. Unstructured steps are candidates for a model. Most processes turn out to be mostly structured with two or three genuinely messy points, which is a very different project from the one that was proposed.
Mark the irreversible steps
Payments, deletions, anything that reaches a customer, anything a regulator would ask about. These stay human, or behind a hard limit the system cannot exceed. Decide this before the architecture, not after a demo.
Build the evaluation set, then the smallest thing
A few dozen real cases with known-good outcomes. Then automate the structured spine, add a model at the messy points, and only reach for an agent if the sequence genuinely varies. Measure against the set at each stage.
Done properly, this exercise most often produces a hybrid: deterministic code for the spine, a model call at the two or three points where language has to be understood, a person at the irreversible step, and no agent at all. That is not a disappointing result. It is cheaper to build, far cheaper to run, auditable, and it will still be working in two years — which is more than can be said for a meaningful share of the agent projects started this year.
Frequently asked questions
Not Sure Which One Your Process Needs?
We map the process step by step, mark what is structured, what is messy and what is irreversible, and recommend the simplest thing that works — which is often not an agent. Book a free 30-minute call with our engineering team. NDA first, no obligation.