Calibration

HLE w/o tools → ARC-AGI-2

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

ARC-AGI-2 is estimated from HLE w/o tools with a Hill curve fitted on 16 models measured on both: y = 0.0000 + (0.8775 − 0.0000)·x^6.00 / (0.32483^6.00 + x^6.00), R² = 0.80, cross-validated error 13.0 pp. It is used for 4 estimates.

Estimated model	HLE w/o tools	ARC-AGI-2	Source
Claude Haiku 5.5	45.9%	78.0%	estimated ± 13.0 pp, low confidence
Claude Sonnet 5.5	56.9%	84.8%	estimated ± 13.0 pp, low confidence
Gemma 4 26B A4B	8.7%	0.0%	estimated ± 13.0 pp, low confidence
Pareto 26.9	49.0%	80.9%	estimated ± 13.0 pp, low confidence