benchgap
Calibration

HLE w/ tools → MCP Atlas

MCP Atlas is estimated from HLE w/ tools with a Michaelis–Menten curve fitted on 8 models measured on both: y = 1.2688·x / (0.34234 + x), R² = 0.73, cross-validated error 7.3 pp. It is used for 4 estimates.

Estimated modelHLE w/ toolsMCP AtlasSource
Agents-A147.6%73.8%estimated ± 7.3 pp, medium confidence
Claude Sonnet 557.4%79.5%estimated ± 7.3 pp, medium confidence
Qwen3.8 Max56.2%78.9%estimated ± 7.3 pp, medium confidence
Step 3.7 Flash47.2%73.5%estimated ± 7.3 pp, medium confidence