Building your first production-grade agentic workflow

Why most agentic pilots stall

The stall pattern is consistent: an impressive demo, an unclear intent boundary, a tool layer that the model cannot call reliably, and no evaluation harness. Six months later, the pilot is quietly retired.

None of the failure modes are fixed by a bigger model. Every one is an architecture or ownership problem.

The six-layer agentic reference architecture

Standardize every enterprise agent on six layers. Each is a decision point with two or three defensible options and a clear anti-pattern.

• Intent boundary — what the agent will and will not attempt, expressed as guardrail-checkable policy
• Tool layer — function contracts, idempotency, retries, side-effect containment
• Memory — session, task, entity, long-term, each with a hard bound and a PII policy
• Orchestration — single-model, cascade or multi-agent, with a chosen fallback pattern
• Guardrails — input, tool-call and output policies enforced independently of the model
• Evaluation — golden sets, LLM-as-judge and production traces feeding CI

Design tools LLMs can actually call

Reliability comes from tool design, not from prompting. One verb, one noun, narrow argument shapes, typed enums, idempotency keys.

Side-effect containment matters as much as shape. Tools that write to customer-facing systems should route through an approval layer, never called directly from the model in an unbounded loop.

Memory that scales without leaking

Four memory types show up in every enterprise agent. Session, task, entity, long-term. Each belongs in its own store with its own retention, encryption and access-control profile.

PII should never enter the vector store in raw form. Upstream tokenization plus a redaction pass is the pattern that survives audit.

Guardrails and human-in-the-loop that work in production

Guardrails must be enforced independently of the model they guard. Three layers: input, tool call, output. Each runs a separate small model or rule engine with veto power.

Human-in-the-loop is a design choice, not a fallback. Pick from the four patterns — sample-based review, exception routing, pre-action approval, post-action audit — and place them deliberately.

Evaluation in CI, not on a schedule

Every prompt or tool change runs against a golden set plus an LLM-as-judge in CI. A change that regresses task success, hallucination bound or cost per resolution blocks the deploy.

Cost per resolved interaction is the single most useful metric. It composes containment, latency, model choice and tool efficiency into one number the CFO can read.

The operating model that keeps agents in production

Three roles, all required. Product owner accountable for the outcome. Platform team accountable for the substrate. Safety review board with veto authority.

Programs missing any of the three eventually stall. Usually within six months of first ship.