Calibration
HLE w/o tools → Vals GPQA Diamond
Vals GPQA Diamond is estimated from HLE w/o tools with a Michaelis–Menten curve fitted on 19 models measured on both: y = 1.0644·x / (0.07487 + x), R² = 0.66, cross-validated error 3.0 pp. It is used for 5 estimates.
| Estimated model | HLE w/o tools | Vals GPQA Diamond | Source |
|---|---|---|---|
| Claude Haiku 5.5 | 45.9% | 91.5% | estimated ± 3.0 pp, high confidence |
| Claude Mythos 5 | 59.0% | 94.4% | estimated ± 3.0 pp, high confidence |
| Gemma 4 26B A4B | 8.7% | 57.2% | estimated ± 3.0 pp, medium confidence |
| Gemma 4 31B | 19.5% | 76.9% | estimated ± 3.0 pp, medium confidence |
| Pareto 26.9 | 49.0% | 92.3% | estimated ± 3.0 pp, high confidence |