Calibration
OSWorld-Verified → ApprenticeBench
ApprenticeBench is estimated from OSWorld-Verified with a linear curve fitted on 6 models measured on both: y = 2.1592·x + -1.5257, R² = 0.90, cross-validated error 5.2 pp. It is used for 11 estimates.
| Estimated model | OSWorld-Verified | ApprenticeBench | Source |
|---|---|---|---|
| Gemini 3.5 Flash-Lite | 74.0% | 7.2% | estimated ± 5.2 pp, low confidence |
| Gemini 3.6 Flash | 83.0% | 26.7% | estimated ± 5.2 pp, low confidence |
| Holo3-122B-A10B | 78.9% | 17.7% | estimated ± 5.2 pp, low confidence |
| Holo3-35B-A3B | 82.6% | 25.7% | estimated ± 5.2 pp, low confidence |
| MiniMax M3 | 70.1% | 0.0% | estimated ± 5.2 pp, low confidence |
| Muse Glimmer 30B | 65.9% | 0.0% | estimated ± 5.2 pp, low confidence |
| Qwen3.5-122B-A10B | 58.0% | 0.0% | estimated ± 5.2 pp, low confidence |
| Qwen3.8-27B | 84.3% | 29.5% | estimated ± 5.2 pp, low confidence |
| Qwen3.8 Max | 86.1% | 33.3% | estimated ± 5.2 pp, low confidence |
| UI-Mate-27B | 77.0% | 13.7% | estimated ± 5.2 pp, low confidence |
| UI-Mate-9B | 66.2% | 0.0% | estimated ± 5.2 pp, low confidence |