Calibration
OSWorld-Verified → DeepSearchQA
DeepSearchQA is estimated from OSWorld-Verified with a Hill curve fitted on 7 models measured on both: y = 0.6926 + (1.2000 − 0.6926)·x^6.00 / (0.89987^6.00 + x^6.00), R² = 0.38, cross-validated error 7.9 pp. It is used for 10 estimates.
| Estimated model | OSWorld-Verified | DeepSearchQA | Source |
|---|---|---|---|
| Qwen3.8 Max | 86.1% | 91.3% | estimated ± 7.9 pp, low confidence |
| Holo3-35B-A3B | 82.6% | 88.2% | estimated ± 7.9 pp, low confidence |
| MiMo-V2.6-Pro | 82.0% | 87.7% | estimated ± 7.9 pp, low confidence |
| MiMo-V2.6-Flash | 80.8% | 86.7% | estimated ± 7.9 pp, low confidence |
| Holo3-122B-A10B | 78.9% | 85.1% | estimated ± 7.9 pp, low confidence |
| UI-Mate-27B | 77.0% | 83.6% | estimated ± 7.9 pp, low confidence |
| Claude Opus 4.5 | 66.3% | 76.3% | estimated ± 7.9 pp, low confidence |
| UI-Mate-9B | 66.2% | 76.2% | estimated ± 7.9 pp, low confidence |
| GPT-5.3 Codex | 64.7% | 75.4% | estimated ± 7.9 pp, low confidence |
| Claude Sonnet 4.5 | 61.4% | 73.9% | estimated ± 7.9 pp, low confidence |
Fitted on
| Model | OSWorld-Verified | DeepSearchQA |
|---|---|---|
| Claude Opus 4.6 | 72.7% | 73.7% |
| Claude Opus 4.8 | 83.4% | 93.1% |
| GPT-5.4 | 75.0% | 73.6% |
| Kimi K2.6 | 73.1% | 92.5% |
| Kimi K2.5 | 63.3% | 77.1% |
| Muse Glimmer 30B | 65.9% | 74.6% |
| Muse Spark 1.1 | 80.8% | 84.9% |