Calibration
OSWorld-Verified → OSWorld 2.0
OSWorld 2.0 is estimated from OSWorld-Verified with a linear curve fitted on 9 models measured on both: y = 1.1206·x + -0.7486, R² = 0.80, cross-validated error 3.6 pp. It is used for 25 estimates.
| Estimated model | OSWorld-Verified | OSWorld 2.0 | Source |
|---|---|---|---|
| Claude Fable 5 | 85.0% | 20.4% | estimated ± 3.6 pp, high confidence |
| Claude Mythos 5 | 85.0% | 20.4% | estimated ± 3.6 pp, high confidence |
| Claude Opus 4.5 | 66.3% | 0.0% | estimated ± 3.6 pp, medium confidence |
| Claude Opus 4.6 | 72.7% | 6.6% | estimated ± 3.6 pp, high confidence |
| Claude Sonnet 4.5 | 61.4% | 0.0% | estimated ± 3.6 pp, medium confidence |
| Claude Sonnet 5 | 81.2% | 16.1% | estimated ± 3.6 pp, high confidence |
| Gemini 3.5 Flash | 78.4% | 13.0% | estimated ± 3.6 pp, high confidence |
| Gemini 3.5 Flash-Lite | 74.0% | 8.1% | estimated ± 3.6 pp, high confidence |
| Gemini 3.6 Flash | 83.0% | 18.1% | estimated ± 3.6 pp, high confidence |
| GPT-5.2 | 47.3% | 0.0% | estimated ± 3.6 pp, medium confidence |
| GPT-5.3 Codex | 64.7% | 0.0% | estimated ± 3.6 pp, medium confidence |
| GPT-5.4 | 75.0% | 9.2% | estimated ± 3.6 pp, high confidence |
| GPT-5.4 mini | 72.1% | 5.9% | estimated ± 3.6 pp, high confidence |
| GPT-5.4 nano | 39.0% | 0.0% | estimated ± 3.6 pp, medium confidence |
| Holo3-122B-A10B | 78.9% | 13.5% | estimated ± 3.6 pp, high confidence |
| Holo3-35B-A3B | 82.6% | 17.7% | estimated ± 3.6 pp, high confidence |
| MiMo-V2.6-Flash | 80.8% | 15.7% | estimated ± 3.6 pp, high confidence |
| MiMo-V2.6-Pro | 82.0% | 17.0% | estimated ± 3.6 pp, high confidence |
| Muse Glimmer 30B | 65.9% | 0.0% | estimated ± 3.6 pp, medium confidence |
| Qwen3.5-122B-A10B | 58.0% | 0.0% | estimated ± 3.6 pp, medium confidence |
| Qwen3.5-27B | 56.2% | 0.0% | estimated ± 3.6 pp, medium confidence |
| Qwen3.5-35B-A3B | 54.5% | 0.0% | estimated ± 3.6 pp, medium confidence |
| Qwen3.8-27B | 84.3% | 19.6% | estimated ± 3.6 pp, high confidence |
| UI-Mate-27B | 77.0% | 11.4% | estimated ± 3.6 pp, high confidence |
| UI-Mate-9B | 66.2% | 0.0% | estimated ± 3.6 pp, medium confidence |