Calibration
Gert Labs → ExploitGym
ExploitGym is estimated from Gert Labs with a linear curve fitted on 18 models measured on both: y = 0.9213·x + -0.4046, R² = 0.72, cross-validated error 8.3 pp. It is used for 63 estimates.
| Estimated model | Gert Labs | ExploitGym | Source |
|---|---|---|---|
| Qwen4 Max | 70.2% | 24.2% | estimated ± 8.3 pp, medium confidence |
| Ember-1 | 58.0% | 13.0% | estimated ± 8.3 pp, medium confidence |
| DeepSeek V4 Pro | 52.7% | 8.1% | estimated ± 8.3 pp, low confidence |
| GPT-5.3 Codex | 52.0% | 7.4% | estimated ± 8.3 pp, low confidence |
| Mercury 3 | 50.6% | 6.1% | estimated ± 8.3 pp, low confidence |
| Inkling 2 Small | 48.6% | 4.3% | estimated ± 8.3 pp, low confidence |
| Claude Opus 4.6 | 48.1% | 3.9% | estimated ± 8.3 pp, low confidence |
| Claude Opus 4.7 | 47.8% | 3.6% | estimated ± 8.3 pp, low confidence |
| GPT-5.2 | 47.7% | 3.4% | estimated ± 8.3 pp, low confidence |
| Qwen 3.6 Max (preview) | 47.5% | 3.3% | estimated ± 8.3 pp, low confidence |
| Gemini 3 Pro | 47.4% | 3.2% | estimated ± 8.3 pp, low confidence |
| Grok 4.5 | 47.2% | 3.1% | estimated ± 8.3 pp, low confidence |
| Kimi K2.6 | 47.0% | 2.8% | estimated ± 8.3 pp, low confidence |
| Claude Opus 4.5 | 46.0% | 1.9% | estimated ± 8.3 pp, low confidence |
| Qwen3.7 Max | 44.8% | 0.8% | estimated ± 8.3 pp, low confidence |
| Claude 4 Sonnet | 27.7% | 0.0% | estimated ± 8.3 pp, low confidence |
| Claude Sonnet 4.5 | 34.9% | 0.0% | estimated ± 8.3 pp, low confidence |
| DeepSeek V3.2 | 21.1% | 0.0% | estimated ± 8.3 pp, low confidence |
| Gemini 2.5 Pro | 32.6% | 0.0% | estimated ± 8.3 pp, low confidence |
| Gemini 3.1 Flash-Lite | 25.7% | 0.0% | estimated ± 8.3 pp, low confidence |
| Gemini 3.5 Flash-Lite | 23.8% | 0.0% | estimated ± 8.3 pp, low confidence |
| Gemini 3 Flash | 34.2% | 0.0% | estimated ± 8.3 pp, low confidence |
| Gemma 4 26B A4B | 19.6% | 0.0% | estimated ± 8.3 pp, low confidence |
| Gemma 4 31B | 27.7% | 0.0% | estimated ± 8.3 pp, low confidence |
| GLM-4.7 | 23.8% | 0.0% | estimated ± 8.3 pp, low confidence |
| GLM-5 | 27.7% | 0.0% | estimated ± 8.3 pp, low confidence |
| GLM-5.1 | 36.4% | 0.0% | estimated ± 8.3 pp, low confidence |
| GLM-5V-Turbo | 23.5% | 0.0% | estimated ± 8.3 pp, low confidence |
| GPT-4.1 | 18.9% | 0.0% | estimated ± 8.3 pp, low confidence |
| GPT-5.1 | 39.9% | 0.0% | estimated ± 8.3 pp, low confidence |
| GPT-5.1-Codex | 28.1% | 0.0% | estimated ± 8.3 pp, low confidence |
| GPT-5.2-Codex | 27.3% | 0.0% | estimated ± 8.3 pp, low confidence |
| GPT-OSS 120B | 21.1% | 0.0% | estimated ± 8.3 pp, low confidence |
| Grok 4 | 31.6% | 0.0% | estimated ± 8.3 pp, low confidence |
| Grok 4.1 Fast | 29.3% | 0.0% | estimated ± 8.3 pp, low confidence |
| Grok 4.20 Multi-agent | 30.2% | 0.0% | estimated ± 8.3 pp, low confidence |
| Grok 4.3 | 21.0% | 0.0% | estimated ± 8.3 pp, low confidence |
| Grok Build 0.1 | 27.2% | 0.0% | estimated ± 8.3 pp, low confidence |
| Hy3 | 26.3% | 0.0% | estimated ± 8.3 pp, low confidence |
| Hy3 Preview | 39.3% | 0.0% | estimated ± 8.3 pp, low confidence |
| Kimi K2.5 | 35.5% | 0.0% | estimated ± 8.3 pp, low confidence |
| Kimi K2.7 Code | 40.7% | 0.0% | estimated ± 8.3 pp, low confidence |
| Laguna M.1 | 20.3% | 0.0% | estimated ± 8.3 pp, low confidence |
| Laguna S 2.1 | 17.5% | 0.0% | estimated ± 8.3 pp, low confidence |
| Laguna XS 2.1 | 17.0% | 0.0% | estimated ± 8.3 pp, low confidence |
| Mercury 2.5 | 20.0% | 0.0% | estimated ± 8.3 pp, low confidence |
| MiMo-V2.5 | 29.9% | 0.0% | estimated ± 8.3 pp, low confidence |
| MiMo-V2-Pro | 24.7% | 0.0% | estimated ± 8.3 pp, low confidence |
| MiniMax M2.7 | 28.8% | 0.0% | estimated ± 8.3 pp, low confidence |
| Mistral Medium 3.5 128B | 20.7% | 0.0% | estimated ± 8.3 pp, low confidence |
| Muse Glimmer 30B | 32.7% | 0.0% | estimated ± 8.3 pp, low confidence |
| Nemotron 3.5 Super VL 120B A12B | 30.9% | 0.0% | estimated ± 8.3 pp, low confidence |
| Qwen3.5-27B | 28.9% | 0.0% | estimated ± 8.3 pp, low confidence |
| Qwen3.5-35B-A3B | 20.2% | 0.0% | estimated ± 8.3 pp, low confidence |
| Qwen3.5 397B | 29.2% | 0.0% | estimated ± 8.3 pp, low confidence |
| Qwen3.6-27B | 32.4% | 0.0% | estimated ± 8.3 pp, low confidence |
| Qwen3.6-35B-A3B | 21.5% | 0.0% | estimated ± 8.3 pp, low confidence |
| Qwen3.6 Plus | 31.4% | 0.0% | estimated ± 8.3 pp, low confidence |
| Qwen3.7 Flash | 24.6% | 0.0% | estimated ± 8.3 pp, low confidence |
| Qwen3.7 Plus | 35.3% | 0.0% | estimated ± 8.3 pp, low confidence |
| Beam 1.0 501B-A23B | 36.9% | 0.0% | estimated ± 8.3 pp, low confidence |
| Step 3.7 Flash | 20.3% | 0.0% | estimated ± 8.3 pp, low confidence |
| Trinity-Large-Thinking | 25.4% | 0.0% | estimated ± 8.3 pp, low confidence |
Fitted on
| Model | Gert Labs | ExploitGym |
|---|---|---|
| Claude Opus 5.5 | 83.3% | 33.3% |
| DeepSeek V4.1 Flash 1017 | 60.1% | 15.3% |
| Gemini 4 Argon | 80.6% | 39.5% |
| Gemini 4 Carbon | 83.7% | 53.5% |
| GLM-5.3 | 55.0% | 15.0% |
| GPT-5.4 | 59.2% | 6.0% |
| GPT-5.5 | 66.4% | 13.4% |
| GPT-5.6 Luna | 54.3% | 12.4% |
| GPT-5.6 Sol | 71.4% | 33.7% |
| GPT-5.6 Terra | 57.3% | 23.2% |
| GPT-6.1 Sol | 90.6% | 35.1% |
| GPT-6 Astra | 92.0% | 42.4% |
| GPT-6 Luna | 58.8% | 11.6% |
| GPT-6 Sol | 79.0% | 22.1% |
| Kimi K3.1 Pro | 65.8% | 20.7% |
| MiMo-V2.6-Flash | 53.5% | 6.0% |
| MiMo-V2.6-Pro | 59.5% | 17.8% |
| Muse Spark 1.1 | 56.1% | 0.8% |