benchgap
Calibration

Gert Labs → ExploitGym

ExploitGym is estimated from Gert Labs with a linear curve fitted on 18 models measured on both: y = 0.9213·x + -0.4046, R² = 0.72, cross-validated error 8.3 pp. It is used for 63 estimates.

Estimated modelGert LabsExploitGymSource
Qwen4 Max70.2%24.2%estimated ± 8.3 pp, medium confidence
Ember-158.0%13.0%estimated ± 8.3 pp, medium confidence
DeepSeek V4 Pro52.7%8.1%estimated ± 8.3 pp, low confidence
GPT-5.3 Codex52.0%7.4%estimated ± 8.3 pp, low confidence
Mercury 350.6%6.1%estimated ± 8.3 pp, low confidence
Inkling 2 Small48.6%4.3%estimated ± 8.3 pp, low confidence
Claude Opus 4.648.1%3.9%estimated ± 8.3 pp, low confidence
Claude Opus 4.747.8%3.6%estimated ± 8.3 pp, low confidence
GPT-5.247.7%3.4%estimated ± 8.3 pp, low confidence
Qwen 3.6 Max (preview)47.5%3.3%estimated ± 8.3 pp, low confidence
Gemini 3 Pro47.4%3.2%estimated ± 8.3 pp, low confidence
Grok 4.547.2%3.1%estimated ± 8.3 pp, low confidence
Kimi K2.647.0%2.8%estimated ± 8.3 pp, low confidence
Claude Opus 4.546.0%1.9%estimated ± 8.3 pp, low confidence
Qwen3.7 Max44.8%0.8%estimated ± 8.3 pp, low confidence
Claude 4 Sonnet27.7%0.0%estimated ± 8.3 pp, low confidence
Claude Sonnet 4.534.9%0.0%estimated ± 8.3 pp, low confidence
DeepSeek V3.221.1%0.0%estimated ± 8.3 pp, low confidence
Gemini 2.5 Pro32.6%0.0%estimated ± 8.3 pp, low confidence
Gemini 3.1 Flash-Lite25.7%0.0%estimated ± 8.3 pp, low confidence
Gemini 3.5 Flash-Lite23.8%0.0%estimated ± 8.3 pp, low confidence
Gemini 3 Flash34.2%0.0%estimated ± 8.3 pp, low confidence
Gemma 4 26B A4B19.6%0.0%estimated ± 8.3 pp, low confidence
Gemma 4 31B27.7%0.0%estimated ± 8.3 pp, low confidence
GLM-4.723.8%0.0%estimated ± 8.3 pp, low confidence
GLM-527.7%0.0%estimated ± 8.3 pp, low confidence
GLM-5.136.4%0.0%estimated ± 8.3 pp, low confidence
GLM-5V-Turbo23.5%0.0%estimated ± 8.3 pp, low confidence
GPT-4.118.9%0.0%estimated ± 8.3 pp, low confidence
GPT-5.139.9%0.0%estimated ± 8.3 pp, low confidence
GPT-5.1-Codex28.1%0.0%estimated ± 8.3 pp, low confidence
GPT-5.2-Codex27.3%0.0%estimated ± 8.3 pp, low confidence
GPT-OSS 120B21.1%0.0%estimated ± 8.3 pp, low confidence
Grok 431.6%0.0%estimated ± 8.3 pp, low confidence
Grok 4.1 Fast29.3%0.0%estimated ± 8.3 pp, low confidence
Grok 4.20 Multi-agent30.2%0.0%estimated ± 8.3 pp, low confidence
Grok 4.321.0%0.0%estimated ± 8.3 pp, low confidence
Grok Build 0.127.2%0.0%estimated ± 8.3 pp, low confidence
Hy326.3%0.0%estimated ± 8.3 pp, low confidence
Hy3 Preview39.3%0.0%estimated ± 8.3 pp, low confidence
Kimi K2.535.5%0.0%estimated ± 8.3 pp, low confidence
Kimi K2.7 Code40.7%0.0%estimated ± 8.3 pp, low confidence
Laguna M.120.3%0.0%estimated ± 8.3 pp, low confidence
Laguna S 2.117.5%0.0%estimated ± 8.3 pp, low confidence
Laguna XS 2.117.0%0.0%estimated ± 8.3 pp, low confidence
Mercury 2.520.0%0.0%estimated ± 8.3 pp, low confidence
MiMo-V2.529.9%0.0%estimated ± 8.3 pp, low confidence
MiMo-V2-Pro24.7%0.0%estimated ± 8.3 pp, low confidence
MiniMax M2.728.8%0.0%estimated ± 8.3 pp, low confidence
Mistral Medium 3.5 128B20.7%0.0%estimated ± 8.3 pp, low confidence
Muse Glimmer 30B32.7%0.0%estimated ± 8.3 pp, low confidence
Nemotron 3.5 Super VL 120B A12B30.9%0.0%estimated ± 8.3 pp, low confidence
Qwen3.5-27B28.9%0.0%estimated ± 8.3 pp, low confidence
Qwen3.5-35B-A3B20.2%0.0%estimated ± 8.3 pp, low confidence
Qwen3.5 397B29.2%0.0%estimated ± 8.3 pp, low confidence
Qwen3.6-27B32.4%0.0%estimated ± 8.3 pp, low confidence
Qwen3.6-35B-A3B21.5%0.0%estimated ± 8.3 pp, low confidence
Qwen3.6 Plus31.4%0.0%estimated ± 8.3 pp, low confidence
Qwen3.7 Flash24.6%0.0%estimated ± 8.3 pp, low confidence
Qwen3.7 Plus35.3%0.0%estimated ± 8.3 pp, low confidence
Beam 1.0 501B-A23B36.9%0.0%estimated ± 8.3 pp, low confidence
Step 3.7 Flash20.3%0.0%estimated ± 8.3 pp, low confidence
Trinity-Large-Thinking25.4%0.0%estimated ± 8.3 pp, low confidence

Fitted on

ModelGert LabsExploitGym
Claude Opus 5.583.3%33.3%
DeepSeek V4.1 Flash 101760.1%15.3%
Gemini 4 Argon80.6%39.5%
Gemini 4 Carbon83.7%53.5%
GLM-5.355.0%15.0%
GPT-5.459.2%6.0%
GPT-5.566.4%13.4%
GPT-5.6 Luna54.3%12.4%
GPT-5.6 Sol71.4%33.7%
GPT-5.6 Terra57.3%23.2%
GPT-6.1 Sol90.6%35.1%
GPT-6 Astra92.0%42.4%
GPT-6 Luna58.8%11.6%
GPT-6 Sol79.0%22.1%
Kimi K3.1 Pro65.8%20.7%
MiMo-V2.6-Flash53.5%6.0%
MiMo-V2.6-Pro59.5%17.8%
Muse Spark 1.156.1%0.8%