Claude Opus 4.8 vs Gemini 3.1 Pro Preview

Independent runs now give Claude Opus 4.8 three clean real wins on agent work, each confirmed on both sides: Terminal-Bench 2.1 (78.9 vs 65.8), Toolathlon-Verified (76.2 vs 61.1), and Agents' Last Exam (27.0 vs 16.4 overall pass rate). The Terminal-Bench one is the weakest: both rows are tbench.ai's, but with different agent scaffolds (Claude Code for Opus, Gemini CLI for Gemini), and vals.ai's single Terminus 2 harness puts them at 71.91 and 70.79, a tie. The reasoning and coding picture: HLE without tools is a statistical tie. GPQA Diamond and, since 2026-09-29, LiveCodeBench no longer belong in that group — both are graded saturated (grade D), so the close gaps there (92.42 vs 95.45, 87.82 vs 88.48) reflect ceiling-out tests, not evenly-matched models. (An earlier version of this page said Agents' Last Exam favored Gemini; that was our error — we had mixed the board's partial-credit score with its pass rate. Corrected 2026-08-17.) LiveBench adds one more tie (76.2 vs 77.0), and AA-AnalystAgent, added 2026-09-29, another: 45.0 vs 41.25 pass^5, both run by Artificial Analysis, inside that benchmark's 11.2-point band. ARC-AGI-2 again doesn't yield a comparable line, since Opus 4.8's tracked score sits at its High tier (72.1) while Gemini's is a flat, untiered 77.1 — different conditions, no matched tier to compare. HMMT Feb 2026 doesn't add a usable line either: both scores (95.45 vs 94.70) carry MathArena's own disclosed contamination-warning, since every model this site tracks was released after the competition happened, so the whole benchmark is graded tainted rather than read as a real gap or a tie. Gemini's remaining case is price: $2/$12 vs $5/$25.