Calibration
ARC-AGI-1 → Vals MMLU-Pro
Vals MMLU-Pro is estimated from ARC-AGI-1 with a Michaelis–Menten curve fitted on 21 models measured on both: y = 1.0790·x / (0.19761 + x), R² = 0.76, cross-validated error 1.8 pp. It is used for 7 estimates.
| Estimated model | ARC-AGI-1 | Vals MMLU-Pro | Source |
|---|---|---|---|
| Claude Opus 4.5 Thinking | 80.0% | 86.5% | estimated ± 1.8 pp, high confidence |
| Claude Opus 4.6 (Adaptive) | 93.0% | 89.0% | estimated ± 1.8 pp, high confidence |
| Claude Sonnet 4.5 Thinking | 63.7% | 82.3% | estimated ± 1.8 pp, high confidence |
| Gemini 3 Pro | 75.0% | 85.4% | estimated ± 1.8 pp, high confidence |
| GPT-5.1 | 72.8% | 84.9% | estimated ± 1.8 pp, high confidence |
| GPT-5.4 Pro | 94.5% | 89.2% | estimated ± 1.8 pp, high confidence |
| GPT-5.5 Pro | 95.0% | 89.3% | estimated ± 1.8 pp, high confidence |