Calibration
OSWorld-Verified → GDPval-AA
GDPval-AA is estimated from OSWorld-Verified with a offset logistic curve fitted on 20 models measured on both: y = 0.2073 + (0.4982 − 0.2073) / (1 + exp(−51.99·(x − 0.7623))), R² = 0.73, cross-validated error 8.8 pp. It is used for 4 estimates.
| Estimated model | OSWorld-Verified | GDPval-AA | Source |
|---|---|---|---|
| Holo3-122B-A10B | 78.9% | 43.9% | estimated ± 8.8 pp, medium confidence |
| Holo3-35B-A3B | 82.6% | 48.8% | estimated ± 8.8 pp, medium confidence |
| UI-Mate-27B | 77.0% | 38.1% | estimated ± 8.8 pp, medium confidence |
| UI-Mate-9B | 66.2% | 20.9% | estimated ± 8.8 pp, medium confidence |