Calibration
HLE w/o tools → Artificial Analysis Intelligence Index
Artificial Analysis Intelligence Index is estimated from HLE w/o tools with a Hill curve fitted on 28 models measured on both: y = 0.1345 + (1.2000 − 0.1345)·x^2.15 / (0.77511^2.15 + x^2.15), R² = 0.85, cross-validated error 5.1 pp. It is used for 11 estimates.
| Estimated model | HLE w/o tools | Artificial Analysis Intelligence Index | Source |
|---|---|---|---|
| Claude Mythos 5 | 59.0% | 51.5% | estimated ± 5.1 pp, medium confidence |
| Grok 4.20 | 31.6% | 26.9% | estimated ± 5.1 pp, medium confidence |
| Ornith-1.5-35B-A3B | 25.6% | 22.4% | estimated ± 5.1 pp, medium confidence |
| Ornith-1.5-397B | 44.6% | 38.3% | estimated ± 5.1 pp, medium confidence |
| Ornith-1.5-9B | 20.2% | 19.0% | estimated ± 5.1 pp, medium confidence |
| Pareto 26.9 | 49.0% | 42.4% | estimated ± 5.1 pp, medium confidence |
| Qwen3.8 Max | 43.6% | 37.4% | estimated ± 5.1 pp, medium confidence |
| Beam | 36.2% | 30.8% | estimated ± 5.1 pp, medium confidence |
| Sakana Fugu | 47.2% | 40.7% | estimated ± 5.1 pp, medium confidence |
| Sakana Fugu-Ultra | 50.0% | 43.3% | estimated ± 5.1 pp, medium confidence |
| Solar Open 2 | 28.8% | 24.8% | estimated ± 5.1 pp, medium confidence |