benchgap
Calibration

DeepsecBench-AA → HLE w/ tools

HLE w/ tools is estimated from DeepsecBench-AA with a offset logistic curve fitted on 8 models measured on both: y = 0.5116 + (0.6461 − 0.5116) / (1 + exp(−74.98·(x − 0.2104))), R² = 0.67, cross-validated error 5.1 pp. It is used for 4 estimates.

Estimated modelDeepsecBench-AAHLE w/ toolsSource
GPT-6.2 Sol51.6%64.6%estimated ± 5.1 pp, low confidence
GPT-6.2 Astra46.8%64.6%estimated ± 5.1 pp, low confidence
Mistral Large 415.0%51.3%estimated ± 5.1 pp, medium confidence
Qwen3.8-27B11.5%51.2%estimated ± 5.1 pp, medium confidence

Fitted on

ModelDeepsecBench-AAHLE w/ tools
Claude Opus 525.9%64.7%
Claude Opus 5.526.9%67.7%
DeepSeek V4.1 Flash 101731.2%63.9%
GLM-5.324.3%62.5%
GLM-5.3-Flash19.9%55.3%
GPT-6 Astra36.9%57.2%
Kimi K3.1 Pro36.7%70.0%
Beam 1.0 501B-A23B6.9%51.1%