Calibration
OSWorld-Verified → JobBench
JobBench is estimated from OSWorld-Verified with a offset logistic curve fitted on 14 models measured on both: y = 0.3276 + (0.5215 − 0.3276) / (1 + exp(−60.16·(x − 0.7611))), R² = 0.60, cross-validated error 9.0 pp. It is used for 6 estimates.
| Estimated model | OSWorld-Verified | JobBench | Source |
|---|---|---|---|
| Holo3-122B-A10B | 78.9% | 49.0% | estimated ± 9.0 pp, medium confidence |
| Holo3-35B-A3B | 82.6% | 51.8% | estimated ± 9.0 pp, medium confidence |
| Qwen3.5-27B | 56.2% | 32.8% | estimated ± 9.0 pp, medium confidence |
| Qwen3.5-35B-A3B | 54.5% | 32.8% | estimated ± 9.0 pp, medium confidence |
| UI-Mate-27B | 77.0% | 45.0% | estimated ± 9.0 pp, medium confidence |
| UI-Mate-9B | 66.2% | 32.8% | estimated ± 9.0 pp, medium confidence |