39 benchmarks
AI coding agents (Claude Code, Codex, Gemini CLI, …) ranked by pass rate on real end-to-end terminal tasks — scored per agent × model.
View agent leaderboard →Humanity's Last Exam — extremely challenging questions designed to test the upper limits of AI capability across diverse domains.
Graduate-level multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult.
Long Context Retrieval benchmark testing ability to find and use information in very long documents.
Instruction Following Benchmark measuring LLM ability to adhere to nuanced writing constraints and formatting requirements.
Tau2 benchmark testing multi-turn agent capabilities in airline and retail domains.
Terminal-based benchmark testing AI ability to interact with command-line interfaces and solve system tasks.
Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains.
Real-world coding benchmark with problems from competitive programming contests, testing code generation and problem-solving abilities.
American Invitational Mathematics Examination 2025 problems testing olympiad-level mathematical reasoning.
AGIEval English — human-level reasoning tasks from standardized exams like SAT, LSAT, and civil service exams.
Big-Bench Hard — challenging subset of BIG-Bench focusing on tasks where language models previously underperformed.
Competition mathematics problems requiring multi-step reasoning, covering algebra, geometry, number theory, and calculus.
American Invitational Mathematics Examination 2024 problems testing olympiad-level mathematical reasoning.
Science coding benchmark measuring AI ability to solve scientific computing tasks.
Multimodal Understanding benchmark testing vision-language models on expert-level tasks.
Massive Multitask Language Understanding — tests knowledge across 57 subjects.
Software Engineering benchmark testing ability to resolve real GitHub issues.
OpenAI HumanEval benchmark measuring Python code generation from function docstrings.
Mostly Basic Python Problems Plus — tests Python code generation with enhanced test cases.
Accounting and audit benchmark testing financial reasoning capabilities.
Simple question answering benchmark testing factual accuracy and knowledge retrieval.
Grade School Math 8K — 8,500 high quality grade school math word problems.
Big Bench Extra Hard — even more challenging reasoning tasks pushing the limits of language model capabilities.
AI2 Reasoning Challenge (Challenge set) — grade-school science questions requiring complex reasoning.
BIRD-CRITIC — multi-turn benchmark testing SQL generation and database interaction.
AGIEval Chinese — reasoning tasks from Chinese standardized exams (Gaokao, civil service).
AI2 Reasoning Challenge (Easy set) — grade-school science questions.
Berkeley Function Calling Leaderboard v3 — testing function/tool calling accuracy.
Instruction Following Evaluation benchmark testing how well LLMs follow detailed formatting and content constraints.
Medical question answering benchmark from USMLE-style questions.
Mathematics benchmark covering algebra, geometry, number theory, and calculus problems.
Stock market benchmark testing financial analysis capabilities.
Logic puzzle benchmark based on knights (truth-tellers) and knaves (liars) puzzles.
Extended formal logic benchmark testing deductive and propositional reasoning.
GAIA — General AI Assistants benchmark testing multi-step real-world tasks.
Weapons of Mass Destruction Proxy — benchmark testing knowledge safety boundaries.
DarkBench — benchmark testing model safety and resistance to adversarial attacks.
Workshop on Machine Translation 2014 — multilingual translation quality benchmark.
MultiChallenge — general knowledge benchmark with diverse challenge types.
Built by @aellman
Also from the founder: BlobCap, a price index for AI training data deals, showing what AI labs actually pay for data.
2026 68 Ventures, LLC. All rights reserved.