Calibration
DeepsecBench-AA → HLE w/ tools
HLE w/ tools is estimated from DeepsecBench-AA with a offset logistic curve fitted on 8 models measured on both: y = 0.5116 + (0.6461 − 0.5116) / (1 + exp(−74.98·(x − 0.2104))), R² = 0.67, cross-validated error 5.1 pp. It is used for 4 estimates.
| Estimated model | DeepsecBench-AA | HLE w/ tools | Source |
|---|---|---|---|
| GPT-6.2 Sol | 51.6% | 64.6% | estimated ± 5.1 pp, low confidence |
| GPT-6.2 Astra | 46.8% | 64.6% | estimated ± 5.1 pp, low confidence |
| Mistral Large 4 | 15.0% | 51.3% | estimated ± 5.1 pp, medium confidence |
| Qwen3.8-27B | 11.5% | 51.2% | estimated ± 5.1 pp, medium confidence |
Fitted on
| Model | DeepsecBench-AA | HLE w/ tools |
|---|---|---|
| Claude Opus 5 | 25.9% | 64.7% |
| Claude Opus 5.5 | 26.9% | 67.7% |
| DeepSeek V4.1 Flash 1017 | 31.2% | 63.9% |
| GLM-5.3 | 24.3% | 62.5% |
| GLM-5.3-Flash | 19.9% | 55.3% |
| GPT-6 Astra | 36.9% | 57.2% |
| Kimi K3.1 Pro | 36.7% | 70.0% |
| Beam 1.0 501B-A23B | 6.9% | 51.1% |