Calibration
ARC-AGI-2 → HLE w/o tools
HLE w/o tools is estimated from ARC-AGI-2 with a linear curve fitted on 18 models measured on both: y = 0.3391·x + 0.2112, R² = 0.66, cross-validated error 6.6 pp. It is used for 0 estimates.
Fitted on
| Model | ARC-AGI-2 | HLE w/o tools |
|---|---|---|
| Claude Fable 5.1 | 90.0% | 60.9% |
| Claude Haiku 5.5 | 61.3% | 45.9% |
| Claude Opus 4.7 | 75.8% | 46.9% |
| Claude Opus 4.8 | 72.1% | 49.8% |
| Claude Opus 5 | 90.4% | 56.3% |
| Claude Opus 5.5 | 91.7% | 64.4% |
| Gemini 3.1 Pro | 77.1% | 45.4% |
| GPT-5.4 | 74.0% | 39.8% |
| GPT-5.4 mini | 18.9% | 28.2% |
| GPT-5.4 nano | 5.7% | 24.3% |
| GPT-5.4 Pro | 83.3% | 42.7% |
| GPT-5.5 | 85.0% | 41.4% |
| GPT-5.5 Pro | 84.2% | 43.1% |
| Grok 4.20 | 53.3% | 31.6% |
| Inkling 2 Small | 81.7% | 44.1% |
| Inkling-Small | 40.1% | 31.6% |
| Kimi K3 | 60.4% | 43.5% |
| Muse Spark | 42.5% | 42.8% |