AI Glossary · Foundations
Benchmark
A standardized task suite (e.g., MMLU, GSM8K, HELM) used to compare model quality on reasoning, coding, safety or domain knowledge.
Definition
What is Benchmark?
Benchmark is a standardized task suite (e.g., MMLU, GSM8K, HELM) used to compare model quality on reasoning, coding, safety or domain knowledge.
- Category
- Foundations
- Glossary set
- 28 related terms
- Audience
- Enterprise AI leaders
Why does Benchmark matter in enterprise AI?
Benchmark matters in enterprise AI programs because it helps business and technology leaders align vocabulary, scope, ownership, and measurable outcomes.
Related terms in Foundations
- Alignment
- The discipline of making an AI system's behavior match human intent, values and organizational policy — both at training time and at runtime.
- Artificial Intelligence (AI)
- Software systems that perform tasks normally requiring human intelligence — perception, reasoning, language, decision-making — by learning patterns from data.
- Artificial General Intelligence (AGI)
- A hypothetical AI capable of performing any intellectual task a human can, across domains. Not a shipping product category today; treat vendor claims with scrutiny.
- Chain-of-Thought (CoT)
- A prompting or training pattern where a model produces intermediate reasoning steps before its final answer, improving accuracy on math, planning and multi-hop tasks.
- Classification
- A supervised learning task that assigns inputs to discrete categories (spam vs. not spam, churn risk tier). The workhorse of predictive AI.
- Context Window
- The maximum amount of text (measured in tokens) a model can consider in a single call, including prompt, tools, retrieved documents and prior turns.