benchgap
Calibration

GPQA Diamond → ARC-AGI-2

ARC-AGI-2 is estimated from GPQA Diamond with a linear curve fitted on 19 models measured on both: y = 5.4701·x + -4.3526, R² = 0.71, cross-validated error 9.8 pp. It is used for 46 estimates.

Estimated modelGPQA DiamondARC-AGI-2Source
Sakana Fugu95.5%87.1%estimated ± 9.8 pp, medium confidence
Sakana Fugu-Ultra95.5%87.1%estimated ± 9.8 pp, medium confidence
Kimi K3.1 Pro94.2%80.0%estimated ± 9.8 pp, medium confidence
Step 5 Preview93.5%76.2%estimated ± 9.8 pp, medium confidence
Ornith-1.5-397B92.8%72.4%estimated ± 9.8 pp, medium confidence
Qwen3.8 Max92.6%71.3%estimated ± 9.8 pp, medium confidence
Pareto 26.10 Preview92.4%70.2%estimated ± 9.8 pp, medium confidence
Qwen3.7 Max92.4%70.2%estimated ± 9.8 pp, medium confidence
Hy4 preview92.3%69.6%estimated ± 9.8 pp, medium confidence
Qwen3.8-Flash-Next91.7%66.3%estimated ± 9.8 pp, medium confidence
GLM-5.291.2%63.6%estimated ± 9.8 pp, medium confidence
Qwen3.8-Omni-Flash91.0%62.5%estimated ± 9.8 pp, medium confidence
DeepSeek V4.1 Flash 101790.9%62.0%estimated ± 9.8 pp, medium confidence
Beam 1.0 501B-A23B90.6%60.3%estimated ± 9.8 pp, medium confidence
Kimi K2.690.5%59.8%estimated ± 9.8 pp, medium confidence
Particle-190.4%59.2%estimated ± 9.8 pp, medium confidence
Qwen3.7 Plus90.3%58.7%estimated ± 9.8 pp, medium confidence
Beam 1.0 501B-A23B FP890.3%58.7%estimated ± 9.8 pp, medium confidence
Interfaze Beta89.9%56.5%estimated ± 9.8 pp, medium confidence
Nemotron 3.5 Super VL 120B A12B NVFP489.9%56.5%estimated ± 9.8 pp, medium confidence
Nemotron 3.5 Super VL 120B A12B89.5%54.4%estimated ± 9.8 pp, medium confidence
Beam 1.0 501B-A23B NVFP489.3%53.2%estimated ± 9.8 pp, medium confidence
Mercury 389.2%52.8%estimated ± 9.8 pp, medium confidence
Claude Opus 4.689.2%52.7%estimated ± 9.8 pp, medium confidence
Ornith-1.5-35B-A3B89.2%52.7%estimated ± 9.8 pp, medium confidence
Qwen3.8-27B89.2%52.7%estimated ± 9.8 pp, medium confidence
Solar Pro 489.0%51.6%estimated ± 9.8 pp, medium confidence
Nemotron 3.5 Bolt 74B-A9B BF1688.8%50.6%estimated ± 9.8 pp, medium confidence
OrcaCyber-Nano-1.088.3%47.6%estimated ± 9.8 pp, medium confidence
Inkling87.9%45.6%estimated ± 9.8 pp, low confidence
Kimi K2.587.6%43.9%estimated ± 9.8 pp, low confidence
Hy3 Preview87.2%41.7%estimated ± 9.8 pp, low confidence
MiniMax M2.787.0%40.6%estimated ± 9.8 pp, low confidence
Nemotron 3 Ultra87.0%40.6%estimated ± 9.8 pp, low confidence
Nemotron 3.5 Bolt 74B-A9B86.4%37.5%estimated ± 9.8 pp, low confidence
Ornith-1.5-9B86.4%37.4%estimated ± 9.8 pp, low confidence
Solar Open 286.3%36.8%estimated ± 9.8 pp, low confidence
GLM-5.186.2%36.3%estimated ± 9.8 pp, low confidence
GLM-586.0%35.2%estimated ± 9.8 pp, low confidence
Ternary Bonsai 2 27B85.8%33.9%estimated ± 9.8 pp, low confidence
A.X K285.6%33.0%estimated ± 9.8 pp, low confidence
Ripple-65B Preview85.2%30.8%estimated ± 9.8 pp, low confidence
Ling 3.0 Flash85.0%29.5%estimated ± 9.8 pp, low confidence
MAI-Thinking-184.2%25.3%estimated ± 9.8 pp, low confidence
Ling 3.0 Flash FP884.0%24.2%estimated ± 9.8 pp, low confidence
K-EXAONE 2.082.2%14.4%estimated ± 9.8 pp, low confidence

Fitted on

ModelGPQA DiamondARC-AGI-2
Claude Opus 4.794.2%75.8%
Claude Opus 4.893.6%72.1%
DeepSeek V4 Flash 073188.1%61.4%
DeepSeek V4 Pro90.1%61.3%
Gemini 3.1 Pro94.3%77.1%
Gemini 3.5 Flash92.7%72.1%
GPT-5.492.8%74.0%
GPT-5.593.6%85.0%
GPT-5.6 Luna92.3%59.5%
GPT-5.6 Sol94.6%92.5%
GPT-5.6 Terra92.9%83.9%
GPT-6.2 Astra97.0%95.8%
GPT-6.2 Sol96.0%85.8%
GPT-6 Astra96.0%95.0%
Grok 4.2088.5%53.3%
Inkling 2 Small93.0%81.7%
Inkling-Small89.5%40.1%
Kimi K393.5%60.4%
Muse Spark89.5%42.5%