benchgap
Calibration

CritPt → ARC-AGI-1

ARC-AGI-1 is estimated from CritPt with a linear curve fitted on 37 models measured on both: y = 0.7491·x + 0.7474, R² = 0.46, cross-validated error 7.8 pp. It is used for 88 estimates.

Estimated modelCritPtARC-AGI-1Source
Muse Wonder Full32.9%99.4%estimated ± 7.8 pp, low confidence
Muse Wonder28.3%95.9%estimated ± 7.8 pp, low confidence
MiMo-V2.6-Pro26.6%94.7%estimated ± 7.8 pp, low confidence
Muse Spark 1.324.9%93.4%estimated ± 7.8 pp, low confidence
Sage-A1.520.0%89.7%estimated ± 7.8 pp, low confidence
Ling 3.1 Flash18.0%88.2%estimated ± 7.8 pp, low confidence
Muse Spark 1.217.7%88.0%estimated ± 7.8 pp, low confidence
Qwen3.8 Max Preview17.7%88.0%estimated ± 7.8 pp, low confidence
GPT-5.3 Codex16.9%87.4%estimated ± 7.8 pp, low confidence
Muse Spark Open14.9%85.9%estimated ± 7.8 pp, low confidence
MiMo-V2.6-Flash12.0%83.7%estimated ± 7.8 pp, low confidence
Kimi K2.7 Code10.0%82.2%estimated ± 7.8 pp, low confidence
Mistral Large 410.0%82.2%estimated ± 7.8 pp, low confidence
Quasar 438B9.4%81.8%estimated ± 7.8 pp, low confidence
GPT-5.2-Codex8.7%81.3%estimated ± 7.8 pp, low confidence
GPT-5.1-Codex5.7%79.0%estimated ± 7.8 pp, low confidence
GPT-5.1-Codex-Max5.7%79.0%estimated ± 7.8 pp, low confidence
Hy34.9%78.4%estimated ± 7.8 pp, low confidence
Apodex 1.14.6%78.2%estimated ± 7.8 pp, low confidence
Apodex 1.1 Mini4.6%78.2%estimated ± 7.8 pp, low confidence
Qwen 3.6 Max (preview)3.7%77.5%estimated ± 7.8 pp, low confidence
Muse Glimmer 30B2.6%76.7%estimated ± 7.8 pp, low confidence
Step 3.7 Flash2.3%76.5%estimated ± 7.8 pp, low confidence
Ling 3.0 Flash VL2.0%76.2%estimated ± 7.8 pp, low confidence
Gemini 2.5 Flash1.4%75.8%estimated ± 7.8 pp, low confidence
Claude 4 Sonnet1.1%75.6%estimated ± 7.8 pp, low confidence
K-Exaone1.1%75.6%estimated ± 7.8 pp, low confidence
MiMo-V2-Omni1.1%75.6%estimated ± 7.8 pp, low confidence
DeepSeek V3.20.9%75.4%estimated ± 7.8 pp, low confidence
Trinity-Large-Preview0.9%75.4%estimated ± 7.8 pp, low confidence
Trinity-Large-Thinking0.9%75.4%estimated ± 7.8 pp, low confidence
Gemma 4 E4B0.6%75.2%estimated ± 7.8 pp, low confidence
GLM-5V-Turbo0.6%75.2%estimated ± 7.8 pp, low confidence
Command A+0.3%75.0%estimated ± 7.8 pp, low confidence
GLM-5-Turbo0.3%75.0%estimated ± 7.8 pp, low confidence
Granite 4.2 30B0.3%75.0%estimated ± 7.8 pp, low confidence
Granite 4.2 8B0.3%75.0%estimated ± 7.8 pp, low confidence
MiMo-V2-Pro0.3%75.0%estimated ± 7.8 pp, low confidence
MiniCPM5-2B0.3%75.0%estimated ± 7.8 pp, low confidence
Mistral Small 40.3%75.0%estimated ± 7.8 pp, low confidence
North Mini Code0.3%75.0%estimated ± 7.8 pp, low confidence
Sarvam 30B0.3%75.0%estimated ± 7.8 pp, low confidence
Celeris-10.0%74.7%estimated ± 7.8 pp, low confidence
Claude 3.5 Haiku0.0%74.7%estimated ± 7.8 pp, low confidence
Claude 3 Haiku0.0%74.7%estimated ± 7.8 pp, low confidence
DeepSeek V30.0%74.7%estimated ± 7.8 pp, low confidence
DeepSeek V3 03240.0%74.7%estimated ± 7.8 pp, low confidence
Devstral Small0.0%74.7%estimated ± 7.8 pp, low confidence
Exaone 4.0 1.2B0.0%74.7%estimated ± 7.8 pp, low confidence
Exaone 4.0 32B0.0%74.7%estimated ± 7.8 pp, low confidence
Gemini 2.5 Flash-Lite0.0%74.7%estimated ± 7.8 pp, low confidence
Gemma 3 27B0.0%74.7%estimated ± 7.8 pp, low confidence
Gemma 4 E2B0.0%74.7%estimated ± 7.8 pp, low confidence
GLM-4.5-Air0.0%74.7%estimated ± 7.8 pp, low confidence
GPT-4.1 nano0.0%74.7%estimated ± 7.8 pp, low confidence
GPT-4o0.0%74.7%estimated ± 7.8 pp, low confidence
Granite-4.0-350M0.0%74.7%estimated ± 7.8 pp, low confidence
Granite-4.0-H-1B0.0%74.7%estimated ± 7.8 pp, low confidence
Granite-4.0-H-350M0.0%74.7%estimated ± 7.8 pp, low confidence
Granite 4.2 3B0.0%74.7%estimated ± 7.8 pp, low confidence
Grok 4.1 Fast0.0%74.7%estimated ± 7.8 pp, low confidence
Grok Code Fast 10.0%74.7%estimated ± 7.8 pp, low confidence
LFM2.5-2.6B0.0%74.7%estimated ± 7.8 pp, low confidence
LFM2.5-8B-A1B0.0%74.7%estimated ± 7.8 pp, low confidence
LFM2.5-VL-1.6B-Extract0.0%74.7%estimated ± 7.8 pp, low confidence
Ling 2.6 Flash0.0%74.7%estimated ± 7.8 pp, low confidence
Ling 3.0 Tiny0.0%74.7%estimated ± 7.8 pp, low confidence
Llama 3.1 405B0.0%74.7%estimated ± 7.8 pp, low confidence
Llama 3.1 70B0.0%74.7%estimated ± 7.8 pp, low confidence
Llama 3.3 70B0.0%74.7%estimated ± 7.8 pp, low confidence
Llama 4 Maverick0.0%74.7%estimated ± 7.8 pp, low confidence
Llama 4 Scout0.0%74.7%estimated ± 7.8 pp, low confidence
Magistral Small0.0%74.7%estimated ± 7.8 pp, low confidence
Mercury 2.50.0%74.7%estimated ± 7.8 pp, low confidence
Mistral Large 20.0%74.7%estimated ± 7.8 pp, low confidence
Mistral Medium 30.0%74.7%estimated ± 7.8 pp, low confidence
Mistral Medium 3.5 128B0.0%74.7%estimated ± 7.8 pp, low confidence
Mistral Small 3.10.0%74.7%estimated ± 7.8 pp, low confidence
Nova Pro0.0%74.7%estimated ± 7.8 pp, low confidence
Phi-40.0%74.7%estimated ± 7.8 pp, low confidence
Phi-4-reasoning0.0%74.7%estimated ± 7.8 pp, low confidence
Qwen3.5-4B0.0%74.7%estimated ± 7.8 pp, low confidence
Qwen3-Omni-30B-A3B-Instruct0.0%74.7%estimated ± 7.8 pp, low confidence
Qwen3-Omni-30B-A3B-Thinking0.0%74.7%estimated ± 7.8 pp, low confidence
Sarvam 105B0.0%74.7%estimated ± 7.8 pp, low confidence
Solar Pro 20.0%74.7%estimated ± 7.8 pp, low confidence
Solar Pro 30.0%74.7%estimated ± 7.8 pp, low confidence
Ultravox v0.6 Llama 3.3 70B0.0%74.7%estimated ± 7.8 pp, low confidence

Fitted on

ModelCritPtARC-AGI-1
Claude Fable 528.6%98.5%
Claude Fable 5.129.7%97.5%
Claude Haiku 5.518.9%89.0%
Claude Opus 4.5 Thinking4.6%80.0%
Claude Opus 4.6 (Adaptive)12.6%93.0%
Claude Opus 4.820.9%92.5%
Claude Opus 529.1%97.5%
Claude Opus 5.531.7%97.5%
Claude Sonnet 4.60.9%86.0%
DeepSeek V4 Flash 073116.6%89.0%
DeepSeek V4 Pro18.0%90.0%
Gemini 3.5 Flash13.1%92.5%
Gemini 3.6 Flash10.6%91.2%
Gemini 3.7 Flash14.3%95.5%
Gemini 3.8 Flash18.3%98.5%
Gemini 3 Pro9.1%75.0%
Gemini 4 Argon27.1%98.5%
GPT-5.14.9%72.8%
GPT-5.211.6%86.2%
GPT-5.423.4%93.7%
GPT-5.4 mini10.0%63.7%
GPT-5.4 nano9.3%51.5%
GPT-5.4 Pro30.0%94.5%
GPT-5.527.1%95.0%
GPT-5.5 Pro30.6%95.0%
GPT-5.6 Luna20.6%88.0%
GPT-5.6 Sol32.3%96.5%
GPT-5.6 Terra30.0%96.5%
GPT-6.1 Sol31.7%96.5%
GPT-6 Astra31.7%97.5%
GPT-6 Luna19.4%86.7%
GPT-6 Sol30.9%95.5%
Grok 4.515.4%85.7%
Grok 4.617.1%87.0%
Grok 4.717.7%89.5%
Inkling-Small8.3%84.0%
Kimi K323.4%94.5%