Calibration
HLE w/o tools → Vals MMLU-Pro
Vals MMLU-Pro is estimated from HLE w/o tools with a Michaelis–Menten curve fitted on 19 models measured on both: y = 0.9970·x / (0.05462 + x), R² = 0.76, cross-validated error 1.9 pp. It is used for 3 estimates.
| Estimated model | HLE w/o tools | Vals MMLU-Pro | Source |
|---|---|---|---|
| Claude Haiku 5.5 | 45.9% | 89.1% | estimated ± 1.9 pp, high confidence |
| Gemma 4 26B A4B | 8.7% | 61.2% | estimated ± 1.9 pp, medium confidence |
| Pareto 26.9 | 49.0% | 89.7% | estimated ± 1.9 pp, high confidence |