benchgap
Calibration

Vals MMLU-Pro → AA MMLU-Pro

AA MMLU-Pro is estimated from Vals MMLU-Pro with a Michaelis–Menten + offset curve fitted on 10 models measured on both: y = 0.2649 + 2.0000·x / (1.97410 + x), R² = 0.68, cross-validated error 1.1 pp. It is used for 13 estimates.

Estimated modelVals MMLU-ProAA MMLU-ProSource
Gemini 3.1 Pro91.0%89.6%estimated ± 1.1 pp, medium confidence
Qwen3.8 Max88.6%88.4%estimated ± 1.1 pp, high confidence
Claude Sonnet 4.5 Thinking87.4%87.9%estimated ± 1.1 pp, high confidence
Grok 4.2086.3%87.3%estimated ± 1.1 pp, high confidence
Gemini 3.1 Flash-Lite86.2%87.3%estimated ± 1.1 pp, high confidence
GLM-5.3-Flash86.1%87.2%estimated ± 1.1 pp, high confidence
Qwen3.5 Flash84.1%86.2%estimated ± 1.1 pp, high confidence
MiMo-V2.582.9%85.6%estimated ± 1.1 pp, high confidence
GLM-4.581.2%84.8%estimated ± 1.1 pp, high confidence
Claude Haiku 4.578.7%83.5%estimated ± 1.1 pp, medium confidence
o3-mini78.7%83.5%estimated ± 1.1 pp, medium confidence
Claude 3.5 Sonnet78.4%83.3%estimated ± 1.1 pp, medium confidence
Gemini 1.5 Pro75.3%81.7%estimated ± 1.1 pp, medium confidence

Fitted on

ModelVals MMLU-ProAA MMLU-Pro
Claude Opus 4.5 Thinking87.3%89.5%
Gemini 3 Pro90.1%89.8%
GLM-4.782.7%85.6%
GPT-586.5%87.1%
GPT-5.186.4%87.0%
GPT-5.286.2%87.4%
Grok 485.3%86.6%
Grok 4.1 Fast (Reasoning)84.2%85.4%
Grok 4 Fast (Reasoning)79.7%85.0%
o385.6%85.3%