Calibration
HLE → Vals MMLU-Pro
Vals MMLU-Pro is estimated from HLE with a linear curve fitted on 32 models measured on both: y = 0.1647·x + 0.7954, R² = 0.42, cross-validated error 2.4 pp. It is used for 6 estimates.
| Estimated model | HLE | Vals MMLU-Pro | Source |
|---|---|---|---|
| Hy4 | 59.1% | 89.3% | estimated ± 2.4 pp, medium confidence |
| Apodex 1.1 | 56.1% | 88.8% | estimated ± 2.4 pp, medium confidence |
| Hy4 flash | 55.2% | 88.6% | estimated ± 2.4 pp, medium confidence |
| Agents-A1 | 47.6% | 87.4% | estimated ± 2.4 pp, medium confidence |
| Ling 3.1 Flash | 37.4% | 85.7% | estimated ± 2.4 pp, medium confidence |
| MiniCPM5-2B | 8.9% | 81.0% | estimated ± 2.4 pp, low confidence |
Fitted on
| Model | HLE | Vals MMLU-Pro |
|---|---|---|
| Claude Fable 5.1 | 65.0% | 92.4% |
| Claude Opus 4.5 | 30.8% | 85.6% |
| Claude Opus 4.7 | 54.7% | 89.9% |
| Claude Opus 4.8 | 57.9% | 89.6% |
| Claude Opus 5 | 64.7% | 91.6% |
| Claude Sonnet 4.6 | 49.0% | 87.3% |
| Claude Sonnet 5 | 57.4% | 87.5% |
| DeepSeek V4 Flash 0731 | 34.8% | 86.2% |
| DeepSeek V4 Pro | 42.7% | 87.0% |
| Gemini 3.5 Flash | 40.2% | 89.5% |
| GLM-4.7 | 24.8% | 82.7% |
| GLM-5.1 | 31.0% | 86.9% |
| GLM-5.2 | 40.5% | 86.7% |
| GPT-5.4 | 52.1% | 87.5% |
| GPT-5.4 mini | 41.5% | 84.6% |
| GPT-5.4 nano | 37.7% | 77.2% |
| GPT-5.5 | 52.2% | 88.1% |
| Grok 4.3 | 35.0% | 85.8% |
| Inkling | 46.0% | 86.3% |
| Inkling-Small | 47.8% | 85.6% |
| Kimi K2.6 | 34.7% | 87.6% |
| Kimi K3 | 56.0% | 88.0% |
| Ling 3.0 Flash | 22.7% | 82.0% |
| MiMo-V2.5-Pro | 48.0% | 84.6% |
| MiniMax M2.7 | 28.0% | 80.4% |
| Muse Spark | 50.4% | 87.3% |
| Muse Spark 1.1 | 62.1% | 88.7% |
| Nemotron 3 Ultra | 26.7% | 85.8% |
| Qwen3.6 Plus | 28.8% | 87.7% |
| Qwen3.7 Max | 41.4% | 89.3% |
| Qwen3.8-27B | 30.8% | 84.3% |
| Qwen3.8 Max | 43.6% | 88.6% |