Calibration
ARC-AGI-3 → Vals MMLU-Pro
Vals MMLU-Pro is estimated from ARC-AGI-3 with a Michaelis–Menten curve fitted on 13 models measured on both: y = 0.8979·x / (0.00004 + x), R² = 0.36, cross-validated error 1.6 pp. It is used for 5 estimates.
| Estimated model | ARC-AGI-3 | Vals MMLU-Pro | Source |
|---|---|---|---|
| GPT-6.1 Sol | 52.7% | 89.8% | estimated ± 1.6 pp, low confidence |
| GPT-6 Sol | 4.6% | 89.7% | estimated ± 1.6 pp, medium confidence |
| Grok 4.7 | 1.8% | 89.6% | estimated ± 1.6 pp, medium confidence |
| Claude Haiku 5.5 | 0.5% | 89.0% | estimated ± 1.6 pp, medium confidence |
| GPT-6 Luna | 0.1% | 86.6% | estimated ± 1.6 pp, medium confidence |
Fitted on
| Model | ARC-AGI-3 | Vals MMLU-Pro |
|---|---|---|
| Claude Opus 4.7 | 0.2% | 89.9% |
| Claude Opus 4.8 | 1.5% | 89.6% |
| Claude Opus 5 | 30.2% | 91.6% |
| Gemini 3.1 Pro | 0.4% | 91.0% |
| Gemini 3.8 Flash | 10.4% | 90.2% |
| GPT-5.4 | 0.2% | 87.5% |
| GPT-5.5 | 0.4% | 88.1% |
| GPT-5.6 Luna | 0.2% | 86.0% |
| GPT-5.6 Sol | 7.8% | 89.1% |
| GPT-5.6 Terra | 0.8% | 86.7% |
| Grok 4.20 | 0.1% | 86.3% |
| Grok 4.5 | 0.3% | 89.2% |
| Grok 4.6 | 2.1% | 89.4% |