- CRM alignment: Salesforce · Dynamics · ServiceNow
- Incumbent economics: Sunk contracts, upgrade credits
- Regulated constraints: Healthcare · PCI · residency
- Geographic coverage: Regions, PSTN, languages
- Outcome: two-vendor shortlist, scored bake-off
The five layers you are actually buying
Telephony and routing (the CCaaS), the automation and orchestration layer, the knowledge and retrieval layer, the agent desktop surface, and the analytics and QA layer. Vendors bundle these differently, which is why feature-by-feature comparison produces nonsense. Score by layer, and be explicit about which layers you intend to keep swappable.
Scoring criteria that separate vendors
Six criteria carry the decision: quality of the evaluation tooling, depth of write-back integration into your CRM and core systems, transparency of the audit trail on tool calls, latency under real concurrency, data residency and retention controls, and the commercial model's behaviour as volume grows. Conversation design tooling and prebuilt intent libraries — the demo's centrepiece — rarely change the outcome.
Integration reality check
Ask every vendor to complete one real write-back transaction against your sandbox during evaluation: create a case, update an order, post a claim note. This single test eliminates more shortlists than any RFP section. Read-only assistants are commodity; the ability to act is what you are paying for.
Total cost including inference
Model four lines beyond licence: per-conversation inference, integration build and maintenance, evaluation and QA tooling, and the run team. Inference cost per contained conversation should be modelled at three volume scenarios — automation economics that work at pilot volume can invert at peak.
Build, buy or layer
Buy packaged automation for commodity intents on commodity data. Layer a platform such as Kore.ai, Amazon Bedrock or Azure AI Foundry when the differentiating workflow runs on your own data and needs to outlive your current CCaaS contract. Build in-house only where the workflow is a competitive asset and you have a standing platform team to run it.
The five software layers you are actually buying
A call center automation stack is rarely one purchase. There is the contact platform that owns telephony, routing and the agent desktop; the conversational layer that handles intent, dialogue and voice; the knowledge and retrieval layer that grounds answers; the integration layer that reaches systems of record; and the analytics and quality layer that scores what happened. Vendors bundle these differently, which is why feature-by-feature comparisons mislead. Map each layer to who owns it in your enterprise and where your gaps are, and evaluate vendors on the layers you actually need rather than on the completeness of their diagram.
Evaluation criteria that predict success
The criteria that correlate with successful deployments are unglamorous: quality and latency of the voice path under real conditions, depth and reliability of integration with your systems of record, the fidelity of the evaluation and testing tooling, how configuration is versioned and promoted between environments, the observability available to your own engineers, and the commercial model's behaviour as volume grows. Demos test none of these. A two-week paid proof against your own data, your own telephony and your own top three intents tests all of them.
Pricing models and where the surprises hide
Per-seat pricing rewards automation and penalises growth in agents; per-minute or per-session pricing does the opposite; consumption pricing on tokens shifts risk to you as volume rises. Model total cost at current volume, at peak, and at the automation rate you are targeting, and ask explicitly about charges for sandbox environments, additional languages, analytics retention, data egress and premium support. The common surprise is not the headline rate — it is the environment, retention and integration line items that appear after signature.
Build, buy or assemble
Buying a suite is fastest to a first result and slowest to differentiation. Assembling best-of-breed layers on your own orchestration gives control over cost and model choice at the price of owning integration and upgrades. Building end to end only makes sense when the workflow is genuinely proprietary. Most enterprises land on assembly: keep the contact platform, own the orchestration, tool layer and evaluation, and treat models and speech services as replaceable components behind an interface.
Implementation risks to price into the plan
Budget explicitly for knowledge remediation, integration work against systems that were never designed for real-time access, identity and entitlement plumbing, evaluation set creation, and the change management required for supervisors and agents. These items, not licences, are where implementations overrun. A plan that shows the vendor's timeline without these workstreams is a plan that will slip.
Integration depth is the deciding factor
Automation software is only as capable as its reach into your systems of record. Assess the availability of real APIs versus screen-scraping, support for your identity and entitlement model, event streaming for real-time use cases, write-back reliability and error semantics, and whether integrations are configuration or bespoke development. Ask vendors to demonstrate against your systems during evaluation. Integration depth is the factor most likely to determine whether the deployment reaches its business case, and it is the factor least visible in a scripted demo.
Testing, evaluation and release tooling
Ask how a change is tested before it reaches production, whether conversation flows and prompts are version-controlled, whether there are separate environments, whether regression suites can be run automatically, and whether evaluation results are exportable. Platforms that treat configuration as a live database with no promotion path force enterprises into risky change management for years. This capability rarely appears on requirement lists and is regularly the thing operations teams complain about eighteen months later.
Observability and who can debug production
When an interaction goes wrong, can your team reconstruct it — inputs, retrieval, model calls, tool invocations, decisions — without a vendor support ticket? Can logs be exported to your own observability estate? Are latency and error metrics available in real time? Enterprises that cannot answer yes end up dependent on vendor turnaround for every incident, which is an operational risk that grows exactly as the automation becomes business-critical.
Roadmap risk, lock-in and exit
Assess how portable your investment is: are intents, flows and prompts exportable in a usable form, is the knowledge corpus yours, can models be substituted, and what happens to your data on exit. Consider vendor viability and roadmap direction, especially in a market consolidating rapidly. The mitigation is architectural — keep orchestration, evaluation and knowledge under your control and treat the platform as a replaceable component — rather than contractual.
Running a proof of value that actually proves something
Define success criteria before starting: named intents, a published evaluation set drawn from your real interactions, latency thresholds, integration against real systems in a test environment, and a fixed timebox. Have your engineers, not the vendor's, drive part of the configuration to test usability. Score the outcome against the criteria in writing. Proofs run without pre-agreed criteria always conclude positively and predict nothing.
A structured selection process
Run selection in five stages with fixed timeboxes. First, define requirements from your own intent inventory and integration reality rather than from an analyst grid, and separate the requirements that are genuinely differentiating from the table stakes every vendor satisfies. Second, longlist against the layers where you actually have a gap — contact platform, conversational layer, knowledge and retrieval, integration, analytics and quality — and discard vendors whose strength lies in layers you already own. Third, run scripted demonstrations against your scenarios, not theirs: bring three real interactions, including one messy one, and ask for them to be handled live. Fourth, run a paid proof of value with pre-agreed success criteria — named intents, an evaluation set drawn from your real interactions, latency thresholds, integration against real systems in a test environment, and your engineers doing part of the configuration to test usability. Fifth, negotiate on the terms that matter over a multi-year horizon: pricing at your expected volume and automation rate, consumption treatment, environment and sandbox costs, data export and retention, support commitments and exit assistance. Throughout, weight integration depth, testing and release tooling, and observability heavily, because those three determine whether your team can operate the platform independently a year after go-live. Score in writing against the criteria you set at the start, and record the gaps you are knowingly accepting. Selections run without written criteria conclude positively for whichever vendor demonstrated most recently, and the accepted gaps resurface eighteen months later as the reasons the programme is stuck.
Questions to put to every shortlisted vendor
Ask how a change moves from a developer's environment to production, and who can make that change. Ask what the platform costs at your volume with your expected automation rate, including environments, storage and analytics. Ask to see the observability an operations team uses daily — traces, failure attribution, latency at the 95th percentile, cost per interaction. Ask how the intent model is tested and how a regression is detected before customers meet it. Ask what happens when an integration times out mid-conversation. Ask which parts of the estate remain portable if you leave, and what exit assistance is contractual rather than goodwill. Ask for two references running at your scale in your industry, and speak to their operations lead rather than their programme sponsor. The answers separate platforms that a team can own from platforms that require the vendor's professional services for every meaningful change, and that distinction costs far more over three years than any line in the licence schedule.
- Score by layer, not by feature list — vendors bundle the five layers differently
- Evaluation tooling, write-back depth and audit trail carry the decision
- Make every vendor complete one real write-back in your sandbox
- Model inference cost at three volume scenarios, not at pilot volume
- You are buying five layers, not one product; evaluate only the layers where you have a real gap.
- A paid proof against your own data, telephony and top intents predicts success better than any demo.
- Model pricing at current, peak and target automation volumes, and probe environment and retention charges.
- Assembly beats suite purchase for most enterprises: own orchestration, tools and evaluation.
Questions leaders ask us
- What should we score when evaluating call center automation software?
- Evaluation tooling quality, depth of write-back integration, audit-trail transparency on tool calls, latency under real concurrency, data residency and retention controls, and how the commercial model behaves as volume grows.
- Should call center automation come from our CCaaS vendor?
- Only if you are prepared to re-platform the automation when you re-platform the contact center. Most enterprises keep the automation and orchestration layer separate so it survives CCaaS change and can run across multiple platforms at once.
- What costs do buyers usually miss?
- Per-conversation inference at peak volume, integration maintenance, evaluation and QA tooling, and the standing run team. Licence is rarely the largest line by year two.
- How do we compare call center automation vendors fairly?
- Score them per layer against your actual gaps, then run a paid two-week proof on your own data, telephony and top three intents with a published evaluation set.
- What pricing model is safest as automation scales?
- Whichever one does not punish the outcome you are pursuing. If you intend to automate aggressively, avoid models that shift cost per interaction upward as agents leave the equation, and cap consumption exposure.
- Should we keep our existing contact platform?
- Usually yes. Replace it only when its routing, telephony or data model actively blocks the roadmap; otherwise layer the conversational and analytics capability on top through APIs and event streams.
- What is most often missing from vendor implementation plans?
- Knowledge remediation, entitlement and integration work, evaluation set creation, and supervisor and agent change management. These drive overruns far more often than licensing.