- Intent boundary: Users, systems, upstream events
- Orchestration: Planners, routers, multi-agent graphs
- Tools: APIs, RPA, retrieval, code execution
- Memory: Short-term, long-term, episodic, semantic
- Guardrails: Input · tool · output policies
- Evaluation: LLM-as-judge, golden sets, red-team
Selecting use cases that survive real data
Generative workloads succeed where the source content is structured enough to retrieve accurately and the output is checkable. Document summarisation, drafting with human review, knowledge answering over curated corpora and structured extraction clear that bar. Open-ended reasoning over messy, contradictory enterprise content does not — and it is where most PoC graveyards start.
Retrieval quality is the whole game
In production incidents we review, the model is rarely at fault. The retrieval is: stale documents, conflicting policy versions, chunking that severs tables from their headers, and permission filters applied after retrieval instead of within it. Budget more for content remediation and retrieval evaluation than for model work, and measure retrieval precision separately from answer quality.
Evaluation before scale, in CI
A golden set of 100–300 real questions with graded reference answers, run on every prompt, model or index change, with a pass threshold that blocks deployment. Without this you cannot upgrade a model, change a prompt or re-index a corpus without risking silent regression — which is why so many generative systems freeze on the day they go live.
Production gates that risk functions accept
Permission-aware retrieval, input and output policy checks, full audit trail linking answer to sources and tool calls, human review on any action with financial or clinical consequence, and documented model-change control. In regulated sectors these are not enhancements — they are the difference between production and an indefinite pilot.
Cost control at production volume
Model routing by task complexity, caching on repeated retrieval, context trimming, and per-workload cost budgets with alerting. Cost per successful task is the metric — cost per token tells you nothing about whether the workload is economic. Model all of it at peak volume before you commit to a rollout.
- Pick workloads where retrieval is accurate and output is checkable
- Retrieval quality, not model choice, explains most production failures
- A golden set in CI is what lets you change models and prompts safely
- Track cost per successful task, not cost per token, at peak volume
Questions leaders ask us
- What does generative AI consulting include?
- Use case selection against real data, retrieval and content remediation, evaluation harness design, production gates for risk and compliance, cost control at production volume, and delivery of the first workloads into production.
- Why do generative AI proofs of concept fail to reach production?
- Usually retrieval quality — stale or conflicting content, chunking that breaks structure, and permission filters applied after retrieval — combined with no evaluation harness, so nobody can prove the system is safe to scale.
- How do we control generative AI costs?
- Route by task complexity, cache repeated retrieval, trim context, and set per-workload budgets with alerting. Measure cost per successful task at peak volume rather than cost per token at pilot volume.