benchgap
Calibration

OSWorld 2.0 → ExploitGym

ExploitGym is estimated from OSWorld 2.0 with a Michaelis–Menten curve fitted on 10 models measured on both: y = 1.1202·x / (1.78710 + x), R² = 0.55, cross-validated error 10.4 pp. It is used for 2 estimates.

Estimated modelOSWorld 2.0ExploitGymSource
Qwen3.8-Flash-Next19.4%11.0%estimated ± 10.4 pp, low confidence
Qwen3.7 Plus2.8%1.7%estimated ± 10.4 pp, low confidence

Fitted on

ModelOSWorld 2.0ExploitGym
Claude Opus 5.548.7%33.3%
Gemini 4 Argon69.2%39.5%
GPT-5.513.0%13.4%
GPT-5.6 Luna45.6%12.4%
GPT-5.6 Sol62.6%33.7%
GPT-5.6 Terra50.2%23.2%
GPT-6 Astra72.6%42.4%
GPT-6 Sol60.5%22.1%
Kimi K3.1 Pro79.6%20.7%
Muse Spark 1.114.2%0.8%