benchgap
Calibration

AA AutomationBench → ExploitGym

ExploitGym is estimated from AA AutomationBench with a Hill curve fitted on 7 models measured on both: y = 0.0561 + (1.2000 − 0.0561)·x^6.00 / (0.87948^6.00 + x^6.00), R² = 0.74, cross-validated error 9.8 pp. It is used for 3 estimates.

Estimated modelAA AutomationBenchExploitGymSource
GPT-6.2 Astra77.2%41.5%estimated ± 9.8 pp, low confidence
Qwen4 Max72.6%33.1%estimated ± 9.8 pp, low confidence
GPT-6.2 Sol71.3%30.9%estimated ± 9.8 pp, low confidence

Fitted on

ModelAA AutomationBenchExploitGym
Claude Opus 5.569.5%33.3%
Gemini 4 Argon77.5%39.5%
Gemini 4 Carbon77.8%53.5%
GLM-5.362.2%15.0%
GPT-6 Luna53.2%11.6%
Kimi K3.1 Pro73.3%20.7%
MiMo-V2.6-Pro58.6%17.8%