Calibration
ARC-AGI-2 → Vals GPQA Diamond
Vals GPQA Diamond is estimated from ARC-AGI-2 with a Michaelis–Menten + offset curve fitted on 24 models measured on both: y = 0.7444 + 0.3174·x / (0.54941 + x), R² = 0.79, cross-validated error 2.2 pp. It is used for 10 estimates.
| Estimated model | ARC-AGI-2 | Vals GPQA Diamond | Source |
|---|---|---|---|
| Claude Opus 4.5 Thinking | 37.6% | 87.3% | estimated ± 2.2 pp, high confidence |
| Claude Opus 4.6 (Adaptive) | 68.8% | 92.1% | estimated ± 2.2 pp, high confidence |
| Claude Sonnet 4.5 | 13.6% | 80.7% | estimated ± 2.2 pp, high confidence |
| Claude Sonnet 4.5 Thinking | 13.6% | 80.7% | estimated ± 2.2 pp, high confidence |
| dots3-note Preview | 81.4% | 93.4% | estimated ± 2.2 pp, high confidence |
| Gemini 3 Pro | 31.1% | 85.9% | estimated ± 2.2 pp, high confidence |
| Gemini 3 Pro Deep Think | 45.1% | 88.8% | estimated ± 2.2 pp, high confidence |
| GPT-5.2 | 52.9% | 90.0% | estimated ± 2.2 pp, high confidence |
| GPT-5.4 Pro | 83.3% | 93.6% | estimated ± 2.2 pp, high confidence |
| GPT-5.5 Pro | 84.2% | 93.7% | estimated ± 2.2 pp, high confidence |