Calibration
HLE w/o tools → ARC-AGI-1
ARC-AGI-1 is estimated from HLE w/o tools with a Hill curve fitted on 12 models measured on both: y = 0.1468 + (0.9791 − 0.1468)·x^6.00 / (0.25505^6.00 + x^6.00), R² = 0.98, cross-validated error 3.7 pp. It is used for 8 estimates.
| Estimated model | HLE w/o tools | ARC-AGI-1 | Source |
|---|---|---|---|
| Claude Haiku 5.5 | 45.9% | 95.5% | estimated ± 3.7 pp, high confidence |
| Claude Mythos 5 | 59.0% | 97.4% | estimated ± 3.7 pp, high confidence |
| Claude Sonnet 5.5 | 56.9% | 97.2% | estimated ± 3.7 pp, high confidence |
| Gemma 4 26B A4B | 8.7% | 14.8% | estimated ± 3.7 pp, medium confidence |
| Gemma 4 31B | 19.5% | 28.5% | estimated ± 3.7 pp, medium confidence |
| MiMo-V2.5-Pro | 34.0% | 85.3% | estimated ± 3.7 pp, high confidence |
| Muse Spark 1.1 | 52.2% | 96.8% | estimated ± 3.7 pp, high confidence |
| Pareto 26.9 | 49.0% | 96.3% | estimated ± 3.7 pp, high confidence |