Calibration
OSWorld 2.0 → ExploitGym
ExploitGym is estimated from OSWorld 2.0 with a inverse Michaelis–Menten curve fitted on 7 models measured on both: y = 0.26180·(x − 0.0000) / (0.0000 + 1.1770 − x), R² = 0.84, cross-validated error 6.9 pp. It is used for 6 estimates.
| Estimated model | OSWorld 2.0 | ExploitGym | Source |
|---|---|---|---|
| Claude Opus 4.7 | 13.9% | 3.5% | estimated ± 6.9 pp, low confidence |
| Claude Sonnet 4.6 | 8.3% | 2.0% | estimated ± 6.9 pp, low confidence |
| Gemini 3.7 Flash | 47.9% | 18.0% | estimated ± 6.9 pp, low confidence |
| Qwen3.7 Plus | 2.8% | 0.6% | estimated ± 6.9 pp, low confidence |
| Qwen3.8-Flash-Next | 19.4% | 5.2% | estimated ± 6.9 pp, low confidence |
| Qwen3.8 Max | 19.4% | 5.2% | estimated ± 6.9 pp, low confidence |