Can you safely let AI agents take actions in your systems?
Autonomy fails on permissions, evaluation and rollback long before it fails on model quality. Score the six foundations that decide whether agents reach production.
Score each dimension
Answer as your organisation operates today, not as it is described in a strategy deck. Scores update the result live.
Are the actions an agent needs exposed as reliable, documented, idempotent APIs?
1 = screen scraping and manual steps · 5 = documented, idempotent APIs with sandboxes
Do agents run under scoped, auditable identities with least-privilege access?
1 = shared service accounts · 5 = per-agent identities, scoped tokens, full audit trail
Is there a regression suite of real cases that gates every prompt, model or tool change?
1 = manual spot checks · 5 = automated eval suite in CI with pass thresholds
Are approval thresholds, escalation paths and rollback defined for high-impact actions?
1 = no approval model · 5 = risk-tiered approvals with tested rollback
Can you trace an agent run end to end — inputs, tool calls, decisions, outcome?
1 = logs only, no traces · 5 = full run traces with outcome attribution
Do you know cost per completed task, and can you route models by cost and risk?
1 = no per-task cost visibility · 5 = cost per task tracked with cheap-first routing
You can put agents into production on scoped workflows. Tighten evaluation and cost controls before widening the action surface.
Weighted average 3.00 of 5. Scores stay in the page link and the PDF — nothing is stored and no email is required.
Your prioritised next 90 days
Generated from your three weakest weighted dimensions.
- 01Identity & permissions — scored 3/5
Issue per-agent identities with least-privilege scopes and short-lived tokens, and log every action with the initiating user and business context.
- 02Tool & API surface — scored 3/5
Publish an action catalogue for the target workflow: each tool documented, idempotent, rate-limited and available in a sandbox before agents touch it.
- 03Evaluation & testing — scored 3/5
Build a golden set of 200+ real cases with expected outcomes and gate every release on automated eval scores, not vibes.
Identity, evaluation, human-in-the-loop and tool surface carry the highest weight because they are the controls auditors and risk committees ask about first.
Common questions
- What makes agentic AI different from a chatbot programme?
- Agents take actions against real systems. That shifts the hard problems from language quality to permissions, evaluation, rollback and cost control — which is what this assessment scores.
- Do we need every dimension at level 4 before starting?
- No. Identity, permissions and human-in-the-loop should be strong before any write action reaches production; the rest can mature alongside a narrow first agent.
- How does this relate to the Agentic AI ROI calculator?
- This assessment tells you whether you can run agents safely; the calculator tells you what they are worth net of inference and orchestration cost. Most boards want both.