- Product squads (Outcomes): Own the KPI, prioritize agents by ROI, run experiments
- AI platform (Capability): Tools, memory, orchestration, evaluation harness
- Safety function (Policy): Guardrails, red-team, audit trail, model risk
- CoE / Chief AI Officer (Governance): Portfolio, funding model, cross-BU standards
Three archetype org models
CoE-led, embedded product, platform-first. Each fits a different starting point. The one that fails most often is the fourth — 'let every BU figure it out' — because it produces 40 pilots and zero platforms.
Who owns what
Product owns outcomes. Platform owns tools, memory and evaluation. Safety owns policy, red-teaming and audit. Procurement, legal and risk are consulted early, not late.
Funding the platform without starving the outcomes
Central funding for the platform, BU funding for outcomes, shared FinOps discipline across both. The formula that keeps AI from becoming another shared-services line item nobody trusts.
Talent bench and retention
Roles that matter: AI product manager, ML platform engineer, prompt engineer, evaluation lead, safety analyst. Career paths and retention practices for a market where every senior IC has three open offers.
The operating model is the bottleneck
Most enterprises can build a working prototype in weeks and then take a year to put it into production. The delay is almost never modelling capability; it is the absence of an operating model that answers five questions: who decides what gets built, who can approve a production release, who owns the workflow once it is live, how spend is attributed, and how change is governed. Until those five answers exist in writing, every use case renegotiates them from scratch, and that renegotiation is the real cost of enterprise AI.
Decision rights, written down
Publish a simple decision matrix. Portfolio prioritisation sits with a business-led forum. Architecture and platform standards sit with the centre. Release approval sits with the business owner for low tiers and with a review body for high tiers. Model and vendor selection sits with the centre with business input. Budget sits with the business unit for workflows and centrally for platform. Ambiguity in any row shows up two quarters later as a stalled release nobody can unblock, and the fix is a page of text agreed before the first build rather than an escalation during it.
Roles the model requires
Six roles carry an AI operating model: a business owner accountable for the outcome metric; a product manager who shapes the workflow and its scope; platform and integration engineers who own the tool layer; an evaluation and quality specialist who owns the golden sets; a governance partner who translates risk requirements into checklists; and an operations owner who handles run, monitoring and incidents. In smaller organisations one person may carry two of these, but if any role has no name against it, that responsibility is not happening.
Funding, attribution and portfolio discipline
Treat AI as a portfolio with quarterly reallocation rather than an annual project budget. Every live workflow reports its metric and its fully loaded cost, including inference. Underperforming workflows are retired or re-scoped, and the released funding moves to what is working. This discipline is what keeps a program from accumulating a long tail of live-but-ignored assistants that quietly consume budget and support attention.
The cadence that runs it
Weekly: delivery squads review escalations, quality and cost, and ship changes. Monthly: business owners review scorecards with the platform team and agree the change backlog. Quarterly: portfolio reallocation, governance re-certification sweep, vendor and model portfolio review, and a refresh of the intent or use-case inventory. Annually: operating model retrospective and standards refresh. The cadence matters more than the org chart, because it is what forces decisions to be made on evidence at a predictable time.
Signals the model is failing
Watch for these: prototypes accumulating without production releases; the same integration built by three teams; governance review times growing quarter on quarter; no single view of AI spend; business owners who cannot state their workflow's metric; and shadow AI appearing in business units. Each has a specific remedy in the model — release authority, shared tool layer, tiered self-certification, cost attribution, scorecard discipline and a usable intake path — and each gets worse if treated as a communications problem.
Intake: how work enters the model
Every operating model needs a front door. Define a short intake that captures the business problem, the owner, the metric, the data and systems involved and the risk tier, and commit to a response time measured in days. Route intake to one of three outcomes: proceed to shaping, redirect to an existing capability, or decline with a reason. Enterprises without a usable intake path get shadow AI, because a business unit with a real problem and no front door buys a tool with a credit card.
Release management for AI workloads
AI releases differ from ordinary software releases because behaviour can change without a code change — new model versions, updated retrieval content, shifted input distributions. Version prompts, tools, retrieval configuration and model selection together as a release unit; require evaluation results to accompany a promotion; support progressive rollout with traffic splitting; and keep rollback to a single action. Enterprises that treat prompt edits as content changes rather than releases discover the gap during their first production regression.
Run, support and on-call
Someone must own production. Define monitoring and alert thresholds, first-line triage, escalation into the platform team, an AI-specific incident classification, and a communication path to business owners when a workflow is suspended. Document the run cost of each workflow including support effort. Many programs fund the build and not the run, then quietly starve support until reliability degrades and confidence with it.
Skills, sourcing and the build-versus-hire question
Map the six roles against your actual bench and decide deliberately what to hire, develop internally or source from a partner. Platform and integration engineering is usually worth hiring or building because it is continuous. Evaluation and quality can start with a partner and transfer. Business ownership and workflow knowledge must always be internal. Publish the plan; ambiguity about which capabilities are permanent is a leading cause of attrition among exactly the engineers you need to keep.
Evolving the model as the portfolio grows
The operating model that suits three workflows fails at thirty. Expect to formalise intake, delegate release authority further into business units as the tiering proves itself, split the platform team as the tool layer grows, and automate governance evidence collection. Review the model annually against the failure signals — accumulating prototypes, duplicated integrations, lengthening reviews, shadow AI — and change it before those signals become the reason the program loses funding.
Documenting the model so it survives people leaving
An operating model that lives in the heads of three people fails the first time one of them changes role. Write it down in a small set of maintained artefacts: a decision matrix covering prioritisation, architecture, release, model and vendor selection and budget; role definitions with named holders for business owner, product manager, platform engineer, evaluation specialist, governance partner and operations owner; the risk tiers with their evidence requirements and approvers; the intake path with its response commitment; the release process with its promotion criteria and rollback expectations; the run model with monitoring, triage and on-call; and the meeting cadence with each forum's purpose, membership and standing agenda. Keep the whole set short enough that a new joiner reads it in an hour, and version it so changes are visible. Review it annually against the failure signals — accumulating prototypes, duplicated integrations, lengthening governance queues, absent spend visibility, business owners who cannot state their metric, and shadow AI appearing in business units — and change the model rather than exhorting people to follow it better. Most enterprises discover the model needs restructuring somewhere between the tenth and the thirtieth production workflow: intake becomes formalised, release authority delegates further into business units as tiering proves itself, the platform team splits by domain, and governance evidence collection automates. Anticipating that evolution and scheduling the review makes it a planned change rather than a crisis triggered by a stalled release or an incident nobody was resourced to handle.
Funding the model without stalling delivery
Split funding into three lines and govern them differently. Platform — the shared tool layer, retrieval infrastructure, evaluation harness, observability and governance tooling — is funded centrally on an annual basis and judged on adoption and unit cost, not on individual business cases, because requiring each workflow to justify shared infrastructure guarantees it is never built. Workflow delivery is funded by the business unit that owns the outcome, with the metric named in the funding request and reviewed at ninety days. Run cost — inference, platform consumption, monitoring and the operations time to keep quality steady — is attributed to the owning workflow from the first day of production, so the true unit economics are visible before scale makes them expensive to change. Enterprises that fund everything through individual project cases build the same integrations repeatedly; those that centralise everything create a queue. The split keeps the shared layer funded and the delivery accountability local.
- Three archetypes work — the 'let every BU figure it out' non-model does not
- Product owns outcomes, platform owns capability, safety owns policy
- Central platform funding + BU outcome funding is the durable model
- Safety is a function, not a document
- Write down decision rights for prioritisation, release, ownership, spend and change before the first build.
- Name a person for each of the six roles; unnamed responsibilities are not being carried.
- Run AI as a quarterly reallocated portfolio where every live workflow reports metric and fully loaded cost.
- Cadence beats org chart: weekly delivery review, monthly scorecards, quarterly reallocation and re-certification.
Questions leaders ask us
- What is an AI operating model?
- An AI operating model is the org design, funding model and governance that turns AI from a series of pilots into a compounding capability — typically a CoE or Chief AI Officer function plus embedded product squads, a platform team and a safety function.
- Do we need a Chief AI Officer?
- Not always a titled role, but you do need a single accountable executive for AI outcomes, platform and safety. In many enterprises this sits with the CIO or COO with a Head of AI Platform reporting in.
- How should AI be funded across BUs and central?
- Central funding for the shared platform (tools, memory, evaluation, safety), BU funding for outcomes, and shared FinOps discipline across both. This keeps AI from becoming another shared-services line item nobody trusts.
- How many people do we need on the AI platform team?
- For a mid-large enterprise, a nucleus of 8–15 covering AI product management, ML platform engineering, evaluation, safety and applied engineering — scaling with the number of live agent workloads, not the number of pilots.
- Why do prototypes stall before production?
- Because release authority, entitlement and ownership are renegotiated per use case. Publishing decision rights and tiered release criteria removes the recurring bottleneck.
- How should AI spend be governed?
- As a portfolio reallocated quarterly, with every live workflow reporting its outcome metric and fully loaded cost including inference, and underperformers retired rather than left running.
- What are the warning signs of a broken operating model?
- Prototype accumulation, duplicated integrations, lengthening governance queues, no consolidated view of spend, business owners without metrics, and shadow AI in business units.
Sources
- [1] Organisations that redesign workflows around AI, not just tools, report the strongest bottom-line impact. The State of AI — McKinsey & Company, 2025
- [2] Pilot-to-production failure is an operating-model problem more than a model-quality problem. The GenAI Divide: State of AI in Business 2025 — MIT NANDA, 2025