benchgap
Calibration

AA-HLE → ARC-AGI-1

ARC-AGI-1 is estimated from AA-HLE with a linear curve fitted on 36 models measured on both: y = 0.7403·x + 0.5746, R² = 0.52, cross-validated error 8.1 pp. It is used for 2 estimates.

Estimated modelAA-HLEARC-AGI-1Source
Phi-4 Multimodal Instruct5.0%61.2%estimated ± 8.1 pp, low confidence
Llama 3.1 8B4.3%60.6%estimated ± 8.1 pp, low confidence

Fitted on

ModelAA-HLEARC-AGI-1
Claude Fable 555.5%98.5%
Claude Fable 5.159.1%97.5%
Claude Haiku 5.544.4%89.0%
Claude Opus 4.5 Thinking30.1%80.0%
Claude Opus 4.6 (Adaptive)39.9%93.0%
Claude Opus 4.848.7%92.5%
Claude Opus 554.9%97.5%
Claude Opus 5.561.4%97.5%
Claude Sonnet 4.613.3%86.0%
DeepSeek V4 Flash 073138.6%89.0%
DeepSeek V4 Pro41.0%90.0%
Gemini 3.5 Flash42.7%92.5%
Gemini 3.6 Flash40.8%91.2%
Gemini 3.7 Flash47.9%95.5%
Gemini 3.8 Flash47.8%98.5%
Gemini 3 Pro39.7%75.0%
Gemini 4 Argon57.1%98.5%
GPT-5.128.5%72.8%
GPT-5.237.7%86.2%
GPT-5.443.7%93.7%
GPT-5.4 mini28.1%63.7%
GPT-5.4 nano28.3%51.5%
GPT-5.545.8%95.0%
GPT-5.6 Luna39.5%88.0%
GPT-5.6 Sol49.5%96.5%
GPT-5.6 Terra42.9%96.5%
GPT-6.1 Sol52.9%96.5%
GPT-6 Astra54.7%97.5%
GPT-6 Luna38.5%86.7%
GPT-6 Sol47.9%95.5%
Grok 4.542.7%85.7%
Grok 4.642.9%87.0%
Grok 4.743.1%89.5%
Inkling 2 Small44.1%96.5%
Inkling-Small33.3%84.0%
Kimi K346.9%94.5%