Calibration
Vals MMLU-Pro → ARC-AGI-2
ARC-AGI-2 is estimated from Vals MMLU-Pro with a linear curve fitted on 24 models measured on both: y = 6.0745·x + -4.6954, R² = 0.69, cross-validated error 13.0 pp. It is used for 4 estimates.
| Estimated model | Vals MMLU-Pro | ARC-AGI-2 | Source |
|---|---|---|---|
| Muse Spark 1.2 | 88.3% | 66.8% | estimated ± 13.0 pp, low confidence |
| Laguna M.1 | 68.8% | 0.0% | estimated ± 13.0 pp, low confidence |
| Laguna XS.2 | 69.1% | 0.0% | estimated ± 13.0 pp, low confidence |
| Mistral Medium 3.5 128B | 75.3% | 0.0% | estimated ± 13.0 pp, low confidence |
Fitted on
| Model | Vals MMLU-Pro | ARC-AGI-2 |
|---|---|---|
| Claude Fable 5 | 91.5% | 89.2% |
| Claude Fable 5.1 | 92.4% | 90.0% |
| Claude Opus 4.8 | 89.6% | 72.1% |
| Claude Opus 5 | 91.6% | 90.4% |
| Claude Sonnet 4.6 | 87.3% | 58.3% |
| DeepSeek V4 Flash 0731 | 86.2% | 61.4% |
| DeepSeek V4 Pro 0813 | 87.0% | 61.3% |
| Gemini 3.1 Pro | 91.0% | 77.1% |
| Gemini 3.5 Flash | 89.5% | 72.1% |
| Gemini 3.6 Flash | 89.3% | 60.4% |
| Gemini 3.7 Flash | 90.1% | 84.6% |
| Gemini 3.8 Flash | 90.2% | 89.2% |
| GPT-5.4 mini | 84.6% | 18.9% |
| GPT-5.4 nano | 77.2% | 5.7% |
| GPT-5.5 | 88.1% | 85.0% |
| GPT-5.6 Luna | 86.0% | 59.5% |
| GPT-5.6 Sol | 89.1% | 92.5% |
| GPT-5.6 Terra | 86.7% | 83.9% |
| Grok 4.20 | 86.3% | 53.3% |
| Grok 4.5 | 89.2% | 52.6% |
| Grok 4.6 | 89.4% | 67.1% |
| Inkling-Small | 85.6% | 40.1% |
| Kimi K3 | 88.0% | 60.4% |
| Muse Spark | 87.3% | 42.5% |