Calibration
ExploitGym → CWE-bench v1
CWE-bench v1 is estimated from ExploitGym with a linear curve fitted on 6 models measured on both: y = 0.5166·x + 0.4576, R² = 0.83, cross-validated error 4.5 pp. It is used for 17 estimates.
| Estimated model | ExploitGym | CWE-bench v1 | Source |
|---|---|---|---|
| GPT-6 Cyber | 48.5% | 70.8% | estimated ± 4.5 pp, medium confidence |
| Claude Mythos 5.2 | 38.1% | 65.4% | estimated ± 4.5 pp, medium confidence |
| GPT-6.1 Sol | 35.1% | 63.9% | estimated ± 4.5 pp, medium confidence |
| GPT-5.6 Sol | 33.7% | 63.2% | estimated ± 4.5 pp, medium confidence |
| GPT-5.6 Cyber | 32.1% | 62.3% | estimated ± 4.5 pp, medium confidence |
| Claude Mythos 5.1 | 23.4% | 57.8% | estimated ± 4.5 pp, medium confidence |
| GPT-5.6 Terra | 23.2% | 57.7% | estimated ± 4.5 pp, medium confidence |
| Kimi K3.1 Pro | 20.7% | 56.4% | estimated ± 4.5 pp, medium confidence |
| MiMo-V2.6-Pro | 17.8% | 55.0% | estimated ± 4.5 pp, medium confidence |
| Claude Mythos Preview | 17.5% | 54.8% | estimated ± 4.5 pp, medium confidence |
| GLM-5.3 | 15.0% | 53.5% | estimated ± 4.5 pp, low confidence |
| GPT-5.5 | 13.4% | 52.7% | estimated ± 4.5 pp, low confidence |
| GPT-5.6 Luna | 12.4% | 52.2% | estimated ± 4.5 pp, low confidence |
| GPT-6 Luna | 11.6% | 51.7% | estimated ± 4.5 pp, low confidence |
| GPT-5.4 | 6.0% | 48.9% | estimated ± 4.5 pp, low confidence |
| MiMo-V2.6-Flash | 6.0% | 48.9% | estimated ± 4.5 pp, low confidence |
| Muse Spark 1.1 | 0.8% | 46.2% | estimated ± 4.5 pp, low confidence |
Fitted on
| Model | ExploitGym | CWE-bench v1 |
|---|---|---|
| Claude Opus 5.5 | 33.3% | 67.0% |
| DeepSeek V4.1 Flash 1017 | 15.3% | 55.0% |
| Gemini 4 Argon | 39.5% | 68.0% |
| Gemini 4 Carbon | 53.5% | 71.0% |
| GPT-6 Astra | 42.4% | 68.0% |
| GPT-6 Sol | 22.1% | 52.0% |