Calibration
Toolathlon-Verified → HLE w/ tools
HLE w/ tools is estimated from Toolathlon-Verified with a linear curve fitted on 14 models measured on both: y = 0.8340·x + -0.0303, R² = 0.78, cross-validated error 5.8 pp. It is used for 1 estimate.
| Estimated model | Toolathlon-Verified | HLE w/ tools | Source |
|---|---|---|---|
| Laguna S 2.1 | 49.7% | 38.4% | estimated ± 5.8 pp, medium confidence |
Fitted on
| Model | Toolathlon-Verified | HLE w/ tools |
|---|---|---|
| Claude Opus 5 | 80.6% | 64.7% |
| Claude Opus 5.5 | 77.8% | 67.7% |
| Claude Sonnet 5.5 | 77.8% | 64.5% |
| DeepSeek V4 Flash 0731 | 70.3% | 45.1% |
| DeepSeek V4 Pro 0813 | 74.1% | 60.0% |
| dots3-note Preview | 55.6% | 52.6% |
| GLM-5.3 | 73.0% | 62.5% |
| GLM-5.3-Flash | 78.4% | 55.3% |
| Hy4 preview | 74.1% | 55.4% |
| Muse Spark Open | 72.8% | 61.0% |
| Ornith-1.5-35B-A3B | 48.7% | 33.4% |
| Ornith-1.5-397B | 71.2% | 56.1% |
| Ornith-1.5-9B | 41.2% | 30.5% |
| Qwen3.8 Max | 72.5% | 56.2% |