benchgap
Calibration

AIME 2025 → MATH-500

MATH-500 is estimated from AIME 2025 with a linear curve fitted on 17 models measured on both: y = 0.1203·x + 0.8742, R² = 0.71, cross-validated error 2.0 pp. It is used for 24 estimates.

Estimated modelAIME 2025MATH-500Source
GPT-5.2100.0%99.4%estimated ± 2.0 pp, medium confidence
Step 3.5 Flash97.3%99.1%estimated ± 2.0 pp, medium confidence
MAI-Thinking-197.0%99.1%estimated ± 2.0 pp, medium confidence
Kimi K2.596.1%99.0%estimated ± 2.0 pp, high confidence
GLM-4.795.7%98.9%estimated ± 2.0 pp, high confidence
MiMo-V2-Flash94.1%98.7%estimated ± 2.0 pp, high confidence
GLM-4.7-Flash91.6%98.4%estimated ± 2.0 pp, high confidence
Nemotron 3 Super 100B90.2%98.3%estimated ± 2.0 pp, high confidence
Granite 4.2 30B89.2%98.1%estimated ± 2.0 pp, high confidence
Nemotron 3 Nano 30B89.1%98.1%estimated ± 2.0 pp, high confidence
Claude Sonnet 4.587.0%97.9%estimated ± 2.0 pp, high confidence
Granite 4.2 8B86.7%97.8%estimated ± 2.0 pp, high confidence
Exaone 4.0 32B85.3%97.7%estimated ± 2.0 pp, high confidence
Nemotron 3 Nano Omni 30B A3B82.1%97.3%estimated ± 2.0 pp, high confidence
Granite 4.2 3B78.3%96.8%estimated ± 2.0 pp, high confidence
Claude 4 Opus Thinking75.5%96.5%estimated ± 2.0 pp, high confidence
Qwen3 235B 250770.3%95.9%estimated ± 2.0 pp, high confidence
LFM2.5-2.6B51.9%93.7%estimated ± 2.0 pp, high confidence
Gemini 2.5 Flash-Lite49.8%93.4%estimated ± 2.0 pp, high confidence
Claude 4 Opus33.9%91.5%estimated ± 2.0 pp, high confidence
Claude 4 Sonnet33.1%91.4%estimated ± 2.0 pp, high confidence
Gemini 2.0 Flash29.7%91.0%estimated ± 2.0 pp, high confidence
Gemini 2.0 Flash-Lite23.8%90.3%estimated ± 2.0 pp, high confidence
Gemini 1.5 Flash14.7%89.2%estimated ± 2.0 pp, medium confidence

Fitted on

ModelAIME 2025MATH-500
Kimi K249.5%97.4%
LFM2.5-8B-A1B42.5%88.8%
Magistral Medium64.9%94.3%
Magistral Small62.8%95.9%
MiniCPM5-1B40.4%91.6%
MiniCPM5-2B86.5%94.6%
MiniMax M1 80k76.9%96.8%
Nemotron Ultra 253B72.5%97.0%
o4-mini92.7%97.5%
OrcaCyber-Nano-1.096.2%98.6%
Qwen3 235B-A22B24.7%91.2%
Qwen3 235B-A22B (Reasoning)81.5%98.0%
Qwen3 32B20.2%88.6%
Qwen3 32B (Reasoning)72.9%97.2%
QwQ-32B69.5%98.0%
Sarvam 105B88.3%98.6%
Ternary Bonsai 2 27B95.0%98.8%