In June 2025 Gartner counted the vendors presenting themselves as agentic AI providers and arrived at a number that should stop any buyer in their tracks: only around 130 of the thousands are the real thing. It named the practice directly - agent washing, the rebranding of a product that already existed, whether an AI assistant, a chatbot or a software robot, without adding genuine agentic capability. RPA was named explicitly. The same analysis predicted that more than 40 percent of agentic AI projects will be cancelled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls.
The press release those figures come from is dated 25 June 2025, so it is over a year old. In that year the market grew, the vocabulary did not settle, and the buyer’s problem stayed exactly where it was - and it is a practical one: in a demo, every system looks agentic. A demo shows a case the vendor prepared, rehearsed and stripped of the variants nobody anticipated. What settles the question is everything outside that case: an unfamiliar variant, ten times the volume, and an error nobody notices.
So I wrote down ten questions you can ask in a meeting, without technical knowledge and without looking at the architecture. They do not call for a demo. They call for answers.
Where these ten questions come from
Not from a feature list, but from the layers that in our own delivery practice genuinely separate an agentic system from automation: tool selection at run time, a completion criterion expressed as a case state, a distinct identity and permission scope, an approval gate triggered by a condition, reproducible reasoning behind a decision, handling of silent failure, a variable component in the cost, and an evaluation set instead of a recording.
Each question has two possible answers. Award 1 point for an answer in the “agentic system” column and 0 points for one in the “repackaged bot” column. An evasive answer scores zero - not out of spite, but out of realism: a vendor who has run one of these in production answers these questions on the spot.
The ten questions
1. What happens when a case arrives that was never in the specification
| Agentic system | Repackaged bot |
|---|---|
| Recognises the case as novel, decides on the basis of its objective, and can explain the decision | Raises an exception and routes the case to a human queue, or fails outright |
This is the decisive question. Everything below simply refines it.
2. Who determines the order of steps - the designer, or the system at run time
| Agentic system | Repackaged bot |
|---|---|
| The sequence is formed on each run, according to the state of the case | The sequence is drawn into a workflow and identical on every run |
A follow-up worth asking for: two runs of the same case with different data, side by side. Identical paths across materially different inputs answer the question by themselves.
3. Is there an explicit set of tools the system chooses from
| Agentic system | Repackaged bot |
|---|---|
| A declared tool set with descriptions, permissions and usage boundaries, selected at run time | System calls hard-wired into steps, with no scope for selection |
4. How is success defined for a single run
| Agentic system | Repackaged bot |
|---|---|
| As a resolved case state, measured independently of the path taken | As every step completing without error |
“The job succeeded” and “the case was resolved” are two different statements. It is worth checking which one the vendor’s dashboard actually reports.
5. Does the system hold its own identity, or run on a service account
| Agentic system | Repackaged bot |
|---|---|
| A distinct identity with its own permission scope, lifecycle and a way to revoke access | A shared service or robot account, with broader permissions than the work requires |
Control question: who revokes this system’s access on a Monday morning, and in which system do they do it. If the answer is “we shut the machine down”, that is not an identity - it is a service account.
6. Where exactly does the human approval gate sit, and what triggers it
| Agentic system | Repackaged bot |
|---|---|
| A gate defined by a condition: value threshold, confidence level, case class, unfamiliar counterparty | No gate, or a gate at the end once the action has already executed |
Approval after the fact is not a control. It is a report. We wrote about this at length in where an agent’s autonomy ends and a human decision begins.
7. Can you reconstruct why a particular decision was made
| Agentic system | Repackaged bot |
|---|---|
| A record of the reasoning and the tools selected, reproducible for a single case from last month | An activity log: what was clicked and when, with no reason attached |
8. How does the system behave when it gets something wrong
| Agentic system | Repackaged bot |
|---|---|
| The vendor has an answer for silent failure - a confident, incorrect decision - and describes detection and rollback | The answer covers visible failure only: retries, alerts, exception handling |
A software robot fails loudly. A system with a language model in the loop fails quietly and with confidence - it does what it “thinks” you meant, and nothing raises an alarm. A vendor unfamiliar with that distinction has not run one of these in production.
9. What happens to cost at ten times the case volume
| Agentic system | Repackaged bot |
|---|---|
| A variable cost stated openly, with a billing unit and the effect of model call volume | A flat licence fee with no variable component, despite claimed reasoning |
This is a question about the price list, but the answer is about the product. A system that genuinely reasons consumes compute on every case, so it has a variable component. The absence of one alongside claimed reasoning tells you what is inside the box, not that you negotiated well.
10. How does the vendor measure quality outside the demo
| Agentic system | Repackaged bot |
|---|---|
| An evaluation set including edge cases, a numeric result, and a history of changes across versions | A recorded demo and a list of reference deployments |
Reading the score
| Score | What it means | What to do about it |
|---|---|---|
| 8-10 | A genuine agentic system. The conversation moves on to oversight, variable cost and permission scope | Proceed to questions on identity, gates and the cost model |
| 4-7 | Automation with elements of reasoning. Sometimes the right choice, but not what the label promises | Establish which steps are genuinely agentic and price them separately |
| 0-3 | A rebranded bot, chatbot or assistant | Buy it as rule-based automation, at rule-based automation prices |
The part without which this list would be dishonest
A low score does not mean a poor product. Deterministic automation is cheaper, more predictable and easier to control - for many processes I would choose it over an agent. Wherever the rule is stable and an error is expensive, a rule-based robot is the better investment than a system that reasons.
What a low score does mean is something else: price, risk and timeline should match what you are actually buying, rather than what it was called in the proposal. Rule-based automation sold as an agent costs what an agent costs, is delivered on an agent’s timeline, and promises a flexibility it will not deliver - and that is the cost you only see six months after signing.
It is also worth remembering that the boundary moves in the opposite direction to the one usually assumed. Agentic systems do not replace robots - a mature architecture assigns roles: agents reason, robots execute, people lead. I wrote about that in reflections on RPA, agents and the skills landscape, and about the distance between something that works and something that runs in production in a coding agent writes the automation in 10 minutes; getting it into production takes 10 weeks.
The question the other way round
I sit on the vendor side myself, so this list works against us too - and that is the point. If you put any of these ten questions to the SNOK team during a conversation about automation and AI, the answer should come back on the spot, with a number and an example.
And now the question the other way round: which question would you add to this list from your own buying experience - the one that settled it fastest in your organisation?
The Gartner figures come from a press release dated 25 June 2025. Check for a more recent edition before citing them.