benchgap
Calibration

ARC-AGI-1 → HLE

HLE is estimated from ARC-AGI-1 with a offset logistic curve fitted on 16 models measured on both: y = 0.4212 + (0.6476 − 0.4212) / (1 + exp(−60.69·(x − 0.9413))), R² = 0.67, cross-validated error 7.0 pp. It is used for 1 estimate.

Estimated modelARC-AGI-1HLESource
GPT-5.172.8%42.1%estimated ± 7.0 pp, medium confidence

Fitted on

ModelARC-AGI-1HLE
Claude Fable 5.197.5%65.0%
Claude Opus 4.892.5%57.9%
Claude Opus 597.5%64.7%
Claude Sonnet 4.686.0%49.0%
DeepSeek V4 Flash 073189.0%34.8%
DeepSeek V4 Pro90.0%42.7%
Gemini 3.5 Flash92.5%40.2%
GPT-5.493.7%52.1%
GPT-5.4 mini63.7%41.5%
GPT-5.4 nano51.5%37.7%
GPT-5.4 Pro94.5%58.7%
GPT-5.595.0%52.2%
GPT-5.5 Pro95.0%57.2%
Inkling 2 Small96.5%53.2%
Inkling-Small84.0%47.8%
Kimi K394.5%56.0%