AI benchmarks explained

How to read this table

Trust grade (A–D) answers one question: when this benchmark says two models differ, how seriously should you take it? An A means the benchmark actively defends itself — refreshed items, contamination controls, a sample large enough that small gaps still carry signal. B is the workhorse grade: no known structural problem, but nothing an A has to defend it either. C means age or a thin sample has started eroding what a score can tell you. D means the ceiling has been reached or the test set has leaked — scores still get printed everywhere, but once a benchmark’s status reads saturated or contaminated, every comparison on it here is labeled Tainted rather than ranked.

Meaningful gap is the point difference below which two scores are statistically indistinguishable on that benchmark — mostly a function of sample size. An 89-item suite simply cannot resolve a 3-point difference the way a 2,500-item one can, which is why the thresholds in this table range from ±2 to over ±10. Between two independently-run scores, any gap under the threshold gets called a tie here, whatever order the leaderboards print.

Category groups benchmarks by what they ask a model to do, because that is what determines whether a score transfers to your work. Reasoning tests hand the model a question with a known answer; coding tests grade a patch by running the project’s own tests; agentic tests turn it loose on a long task with real tools and score the end state. A model can be genuinely excellent at one and ordinary at another — comparing across categories is the single most common way benchmark tables mislead.

The grouping is by what form the work takes, not by which subjects it touches — Terminal-Bench’s tasks span software engineering, sysadmin, data processing and security, but every one of them is graded the same way: turn a model loose on a real terminal and check the end state. That is what makes it agentic, regardless of how many fields its task prompts happen to mention.

Status tracks where the benchmark is in its life: current ones still separate frontier models, aging ones are losing resolution as the field climbs toward their ceiling, and saturated or contaminated ones no longer produce comparisons worth trusting at all.

The full grading criteria live in the methodology; the grades and thresholds themselves ship as CC BY JSON on the open data page. Each benchmark name above links to a deeper page: what it measures, its caveats, and current scores — or put any of it to work in the comparison tool.