AI Glossary · Applied AI Engineering
Cost per Task
End-to-end token, retrieval and infrastructure cost to complete one unit of work — the correct unit economics for LLM systems, not price-per-token.
Definition
What is Cost per Task?
Cost per Task is end-to-end token, retrieval and infrastructure cost to complete one unit of work — the correct unit economics for LLM systems, not price-per-token.
- Category
- Applied AI Engineering
- Glossary set
- 10 related terms
- Audience
- Enterprise AI leaders
Why does Cost per Task matter in enterprise AI?
Cost per Task matters in enterprise AI programs because it helps business and technology leaders align vocabulary, scope, ownership, and measurable outcomes.
Related terms in Applied AI Engineering
- AI Engineer
- A software engineer who ships production LLM systems — prompt design, RAG, tool integration, evals, cost/latency tuning. Distinct from ML engineer.
- Copilot
- An AI assistant embedded in a workflow (email, CRM, IDE, contact center desktop) that suggests actions in-line while a human remains in control.
- Evals
- Automated tests that measure model or agent quality on task-specific criteria — accuracy, safety, groundedness, latency, cost — usually run in CI.
- Guardrails
- Runtime filters and policies that block unsafe inputs or outputs — PII, jailbreaks, toxicity, off-topic responses, unauthorized tool calls.
- IDP (Intelligent Document Processing)
- Extracting structured data from unstructured documents (claims, invoices, contracts) using OCR, layout models and LLMs.
- Latency Budget
- Allocation of end-to-end response time across retrieval, model calls and tool hops. Voice AI budgets are typically <1.2s per turn.