Calibration
Toolathlon-Verified → OSWorld 2.0
OSWorld 2.0 is estimated from Toolathlon-Verified with a offset logistic curve fitted on 5 models measured on both: y = 0.1939 + (0.7079 − 0.1939) / (1 + exp(−199.98·(x − 0.7780))), R² = 0.99, cross-validated error 5.5 pp. It is used for 14 estimates.
| Estimated model | Toolathlon-Verified | OSWorld 2.0 | Source |
|---|---|---|---|
| Claude Sonnet 5.5 | 77.8% | 45.2% | estimated ± 5.5 pp, low confidence |
| DeepSeek V4 Flash 0731 | 70.3% | 19.4% | estimated ± 5.5 pp, low confidence |
| DeepSeek V4 Pro 0813 | 74.1% | 19.4% | estimated ± 5.5 pp, low confidence |
| dots3-note Preview | 55.6% | 19.4% | estimated ± 5.5 pp, low confidence |
| GLM-5.3 | 73.0% | 19.4% | estimated ± 5.5 pp, low confidence |
| GLM-5.3-Flash | 78.4% | 59.0% | estimated ± 5.5 pp, low confidence |
| Hy4 preview | 74.1% | 19.4% | estimated ± 5.5 pp, low confidence |
| Inkling-Small | 54.4% | 19.4% | estimated ± 5.5 pp, low confidence |
| Kimi K3 | 73.2% | 19.4% | estimated ± 5.5 pp, low confidence |
| Laguna S 2.1 | 49.7% | 19.4% | estimated ± 5.5 pp, low confidence |
| Ornith-1.5-35B-A3B | 48.7% | 19.4% | estimated ± 5.5 pp, low confidence |
| Ornith-1.5-397B | 71.2% | 19.4% | estimated ± 5.5 pp, low confidence |
| Ornith-1.5-9B | 41.2% | 19.4% | estimated ± 5.5 pp, low confidence |
| Step 5 Preview | 74.1% | 19.4% | estimated ± 5.5 pp, low confidence |