benchgap
Calibration

ARC-AGI-3 → Vals MMLU-Pro

Vals MMLU-Pro is estimated from ARC-AGI-3 with a Michaelis–Menten curve fitted on 13 models measured on both: y = 0.8979·x / (0.00004 + x), R² = 0.36, cross-validated error 1.6 pp. It is used for 5 estimates.

Estimated modelARC-AGI-3Vals MMLU-ProSource
GPT-6.1 Sol52.7%89.8%estimated ± 1.6 pp, low confidence
GPT-6 Sol4.6%89.7%estimated ± 1.6 pp, medium confidence
Grok 4.71.8%89.6%estimated ± 1.6 pp, medium confidence
Claude Haiku 5.50.5%89.0%estimated ± 1.6 pp, medium confidence
GPT-6 Luna0.1%86.6%estimated ± 1.6 pp, medium confidence

Fitted on

ModelARC-AGI-3Vals MMLU-Pro
Claude Opus 4.70.2%89.9%
Claude Opus 4.81.5%89.6%
Claude Opus 530.2%91.6%
Gemini 3.1 Pro0.4%91.0%
Gemini 3.8 Flash10.4%90.2%
GPT-5.40.2%87.5%
GPT-5.50.4%88.1%
GPT-5.6 Luna0.2%86.0%
GPT-5.6 Sol7.8%89.1%
GPT-5.6 Terra0.8%86.7%
Grok 4.200.1%86.3%
Grok 4.50.3%89.2%
Grok 4.62.1%89.4%