Calibration
OSWorld-Verified → CyberGym
CyberGym is estimated from OSWorld-Verified with a Hill curve fitted on 10 models measured on both: y = 0.0000 + (1.0822 − 0.0000)·x^6.00 / (0.66913^6.00 + x^6.00), R² = 0.53, cross-validated error 11.3 pp. It is used for 4 estimates.
| Estimated model | OSWorld-Verified | CyberGym | Source |
|---|---|---|---|
| Holo3-122B-A10B | 78.9% | 78.8% | estimated ± 11.3 pp, low confidence |
| Holo3-35B-A3B | 82.6% | 84.3% | estimated ± 11.3 pp, low confidence |
| UI-Mate-27B | 77.0% | 75.6% | estimated ± 11.3 pp, low confidence |
| UI-Mate-9B | 66.2% | 52.4% | estimated ± 11.3 pp, low confidence |