Calibration
OSWorld-Verified → AA EnterpriseOps-Gym
AA EnterpriseOps-Gym is estimated from OSWorld-Verified with a inverse Michaelis–Menten curve fitted on 5 models measured on both: y = 0.67472·(x − 0.0000) / (0.0000 + 2.0000 − x), R² = 0.70, cross-validated error 5.9 pp. It is used for 10 estimates.
| Estimated model | OSWorld-Verified | AA EnterpriseOps-Gym | Source |
|---|---|---|---|
| Claude Mythos 5 | 85.0% | 49.9% | estimated ± 5.9 pp, low confidence |
| Claude Sonnet 4.5 | 61.4% | 29.9% | estimated ± 5.9 pp, low confidence |
| Claude Sonnet 5 | 81.2% | 46.1% | estimated ± 5.9 pp, low confidence |
| Gemini 3.5 Flash-Lite | 74.0% | 39.6% | estimated ± 5.9 pp, low confidence |
| Gemini 3.6 Flash | 83.0% | 47.9% | estimated ± 5.9 pp, low confidence |
| Holo3-122B-A10B | 78.9% | 43.9% | estimated ± 5.9 pp, low confidence |
| Holo3-35B-A3B | 82.6% | 47.4% | estimated ± 5.9 pp, low confidence |
| Muse Spark 1.1 | 80.8% | 45.7% | estimated ± 5.9 pp, low confidence |
| UI-Mate-27B | 77.0% | 42.2% | estimated ± 5.9 pp, low confidence |
| UI-Mate-9B | 66.2% | 33.4% | estimated ± 5.9 pp, low confidence |