Calibration
OSWorld-Verified → Toolathlon
Toolathlon is estimated from OSWorld-Verified with a Hill curve fitted on 9 models measured on both: y = 0.3254 + (1.2000 − 0.3254)·x^6.00 / (0.91011^6.00 + x^6.00), R² = 0.73, cross-validated error 7.6 pp. It is used for 4 estimates.
| Estimated model | OSWorld-Verified | Toolathlon | Source |
|---|---|---|---|
| Holo3-122B-A10B | 78.9% | 58.5% | estimated ± 7.6 pp, medium confidence |
| Holo3-35B-A3B | 82.6% | 63.8% | estimated ± 7.6 pp, medium confidence |
| UI-Mate-27B | 77.0% | 56.0% | estimated ± 7.6 pp, medium confidence |
| UI-Mate-9B | 66.2% | 43.8% | estimated ± 7.6 pp, medium confidence |