Calibration
AIME 2025 → MATH-500
MATH-500 is estimated from AIME 2025 with a linear curve fitted on 17 models measured on both: y = 0.1203·x + 0.8742, R² = 0.71, cross-validated error 2.0 pp. It is used for 24 estimates.
| Estimated model | AIME 2025 | MATH-500 | Source |
|---|---|---|---|
| GPT-5.2 | 100.0% | 99.4% | estimated ± 2.0 pp, medium confidence |
| Step 3.5 Flash | 97.3% | 99.1% | estimated ± 2.0 pp, medium confidence |
| MAI-Thinking-1 | 97.0% | 99.1% | estimated ± 2.0 pp, medium confidence |
| Kimi K2.5 | 96.1% | 99.0% | estimated ± 2.0 pp, high confidence |
| GLM-4.7 | 95.7% | 98.9% | estimated ± 2.0 pp, high confidence |
| MiMo-V2-Flash | 94.1% | 98.7% | estimated ± 2.0 pp, high confidence |
| GLM-4.7-Flash | 91.6% | 98.4% | estimated ± 2.0 pp, high confidence |
| Nemotron 3 Super 100B | 90.2% | 98.3% | estimated ± 2.0 pp, high confidence |
| Granite 4.2 30B | 89.2% | 98.1% | estimated ± 2.0 pp, high confidence |
| Nemotron 3 Nano 30B | 89.1% | 98.1% | estimated ± 2.0 pp, high confidence |
| Claude Sonnet 4.5 | 87.0% | 97.9% | estimated ± 2.0 pp, high confidence |
| Granite 4.2 8B | 86.7% | 97.8% | estimated ± 2.0 pp, high confidence |
| Exaone 4.0 32B | 85.3% | 97.7% | estimated ± 2.0 pp, high confidence |
| Nemotron 3 Nano Omni 30B A3B | 82.1% | 97.3% | estimated ± 2.0 pp, high confidence |
| Granite 4.2 3B | 78.3% | 96.8% | estimated ± 2.0 pp, high confidence |
| Claude 4 Opus Thinking | 75.5% | 96.5% | estimated ± 2.0 pp, high confidence |
| Qwen3 235B 2507 | 70.3% | 95.9% | estimated ± 2.0 pp, high confidence |
| LFM2.5-2.6B | 51.9% | 93.7% | estimated ± 2.0 pp, high confidence |
| Gemini 2.5 Flash-Lite | 49.8% | 93.4% | estimated ± 2.0 pp, high confidence |
| Claude 4 Opus | 33.9% | 91.5% | estimated ± 2.0 pp, high confidence |
| Claude 4 Sonnet | 33.1% | 91.4% | estimated ± 2.0 pp, high confidence |
| Gemini 2.0 Flash | 29.7% | 91.0% | estimated ± 2.0 pp, high confidence |
| Gemini 2.0 Flash-Lite | 23.8% | 90.3% | estimated ± 2.0 pp, high confidence |
| Gemini 1.5 Flash | 14.7% | 89.2% | estimated ± 2.0 pp, medium confidence |
Fitted on
| Model | AIME 2025 | MATH-500 |
|---|---|---|
| Kimi K2 | 49.5% | 97.4% |
| LFM2.5-8B-A1B | 42.5% | 88.8% |
| Magistral Medium | 64.9% | 94.3% |
| Magistral Small | 62.8% | 95.9% |
| MiniCPM5-1B | 40.4% | 91.6% |
| MiniCPM5-2B | 86.5% | 94.6% |
| MiniMax M1 80k | 76.9% | 96.8% |
| Nemotron Ultra 253B | 72.5% | 97.0% |
| o4-mini | 92.7% | 97.5% |
| OrcaCyber-Nano-1.0 | 96.2% | 98.6% |
| Qwen3 235B-A22B | 24.7% | 91.2% |
| Qwen3 235B-A22B (Reasoning) | 81.5% | 98.0% |
| Qwen3 32B | 20.2% | 88.6% |
| Qwen3 32B (Reasoning) | 72.9% | 97.2% |
| QwQ-32B | 69.5% | 98.0% |
| Sarvam 105B | 88.3% | 98.6% |
| Ternary Bonsai 2 27B | 95.0% | 98.8% |