AI Glossary · Applied AI Engineering
Latency Budget
Allocation of end-to-end response time across retrieval, model calls and tool hops. Voice AI budgets are typically <1.2s per turn.
Related terms in Applied AI Engineering
- AI Engineer
- A software engineer who ships production LLM systems — prompt design, RAG, tool integration, evals, cost/latency tuning. Distinct from ML engineer.
- Copilot
- An AI assistant embedded in a workflow (email, CRM, IDE, contact center desktop) that suggests actions in-line while a human remains in control.
- Cost per Task
- End-to-end token, retrieval and infrastructure cost to complete one unit of work — the correct unit economics for LLM systems, not price-per-token.
- Evals
- Automated tests that measure model or agent quality on task-specific criteria — accuracy, safety, groundedness, latency, cost — usually run in CI.
- Guardrails
- Runtime filters and policies that block unsafe inputs or outputs — PII, jailbreaks, toxicity, off-topic responses, unauthorized tool calls.
- IDP (Intelligent Document Processing)
- Extracting structured data from unstructured documents (claims, invoices, contracts) using OCR, layout models and LLMs.