benchgap
Calibration

AA Cyber Index → HLE w/ tools

HLE w/ tools is estimated from AA Cyber Index with a Michaelis–Menten curve fitted on 8 models measured on both: y = 0.8017·x / (0.12554 + x), R² = 0.60, cross-validated error 8.9 pp. It is used for 0 estimates.

Fitted on

ModelAA Cyber IndexHLE w/ tools
Claude Opus 5.528.7%67.7%
DeepSeek V4.1 Flash 101740.8%63.9%
GLM-5.336.4%62.5%
GLM-5.3-Flash49.9%55.3%
GPT-6 Astra33.4%57.2%
Kimi K3.1 Pro64.1%70.0%
Nemotron 3 Ultra13.1%37.4%
Beam 1.0 501B-A23B33.7%51.1%