benchgap
Calibration

ARC-AGI-2 → HLE w/o tools

HLE w/o tools is estimated from ARC-AGI-2 with a linear curve fitted on 18 models measured on both: y = 0.3391·x + 0.2112, R² = 0.66, cross-validated error 6.6 pp. It is used for 0 estimates.

Fitted on

ModelARC-AGI-2HLE w/o tools
Claude Fable 5.190.0%60.9%
Claude Haiku 5.561.3%45.9%
Claude Opus 4.775.8%46.9%
Claude Opus 4.872.1%49.8%
Claude Opus 590.4%56.3%
Claude Opus 5.591.7%64.4%
Gemini 3.1 Pro77.1%45.4%
GPT-5.474.0%39.8%
GPT-5.4 mini18.9%28.2%
GPT-5.4 nano5.7%24.3%
GPT-5.4 Pro83.3%42.7%
GPT-5.585.0%41.4%
GPT-5.5 Pro84.2%43.1%
Grok 4.2053.3%31.6%
Inkling 2 Small81.7%44.1%
Inkling-Small40.1%31.6%
Kimi K360.4%43.5%
Muse Spark42.5%42.8%