Calibration
OSWorld-Verified → Gert Labs
Gert Labs is estimated from OSWorld-Verified with a inverse Michaelis–Menten curve fitted on 13 models measured on both: y = 1.07733·(x − 0.0000) / (0.0000 + 2.0000 − x), R² = 0.71, cross-validated error 7.2 pp. It is used for 9 estimates.
| Estimated model | OSWorld-Verified | Gert Labs | Source |
|---|---|---|---|
| Gemini 3.5 Flash-Lite | 74.0% | 63.3% | estimated ± 7.2 pp, medium confidence |
| Gemini 3.6 Flash | 83.0% | 76.4% | estimated ± 7.2 pp, medium confidence |
| Holo3-122B-A10B | 78.9% | 70.1% | estimated ± 7.2 pp, medium confidence |
| Holo3-35B-A3B | 82.6% | 75.7% | estimated ± 7.2 pp, medium confidence |
| Muse Glimmer 30B | 65.9% | 52.9% | estimated ± 7.2 pp, medium confidence |
| Qwen3.5-122B-A10B | 58.0% | 44.0% | estimated ± 7.2 pp, medium confidence |
| Qwen3.8-27B | 84.3% | 78.5% | estimated ± 7.2 pp, low confidence |
| UI-Mate-27B | 77.0% | 67.4% | estimated ± 7.2 pp, medium confidence |
| UI-Mate-9B | 66.2% | 53.3% | estimated ± 7.2 pp, medium confidence |