benchgap
Calibration

HLE → Vals MMLU-Pro

Vals MMLU-Pro is estimated from HLE with a linear curve fitted on 32 models measured on both: y = 0.1647·x + 0.7954, R² = 0.42, cross-validated error 2.4 pp. It is used for 6 estimates.

Estimated modelHLEVals MMLU-ProSource
Hy459.1%89.3%estimated ± 2.4 pp, medium confidence
Apodex 1.156.1%88.8%estimated ± 2.4 pp, medium confidence
Hy4 flash55.2%88.6%estimated ± 2.4 pp, medium confidence
Agents-A147.6%87.4%estimated ± 2.4 pp, medium confidence
Ling 3.1 Flash37.4%85.7%estimated ± 2.4 pp, medium confidence
MiniCPM5-2B8.9%81.0%estimated ± 2.4 pp, low confidence

Fitted on

ModelHLEVals MMLU-Pro
Claude Fable 5.165.0%92.4%
Claude Opus 4.530.8%85.6%
Claude Opus 4.754.7%89.9%
Claude Opus 4.857.9%89.6%
Claude Opus 564.7%91.6%
Claude Sonnet 4.649.0%87.3%
Claude Sonnet 557.4%87.5%
DeepSeek V4 Flash 073134.8%86.2%
DeepSeek V4 Pro42.7%87.0%
Gemini 3.5 Flash40.2%89.5%
GLM-4.724.8%82.7%
GLM-5.131.0%86.9%
GLM-5.240.5%86.7%
GPT-5.452.1%87.5%
GPT-5.4 mini41.5%84.6%
GPT-5.4 nano37.7%77.2%
GPT-5.552.2%88.1%
Grok 4.335.0%85.8%
Inkling46.0%86.3%
Inkling-Small47.8%85.6%
Kimi K2.634.7%87.6%
Kimi K356.0%88.0%
Ling 3.0 Flash22.7%82.0%
MiMo-V2.5-Pro48.0%84.6%
MiniMax M2.728.0%80.4%
Muse Spark50.4%87.3%
Muse Spark 1.162.1%88.7%
Nemotron 3 Ultra26.7%85.8%
Qwen3.6 Plus28.8%87.7%
Qwen3.7 Max41.4%89.3%
Qwen3.8-27B30.8%84.3%
Qwen3.8 Max43.6%88.6%