Calibration
BrowseComp → OSWorld 2.0
OSWorld 2.0 is estimated from BrowseComp with a offset logistic curve fitted on 10 models measured on both: y = 0.1770 + (0.6862 − 0.1770) / (1 + exp(−200.00·(x − 0.8722))), R² = 0.82, cross-validated error 13.8 pp. It is used for 10 estimates.
| Estimated model | BrowseComp | OSWorld 2.0 | Source |
|---|---|---|---|
| Agents-A1 | 75.5% | 17.7% | estimated ± 13.8 pp, low confidence |
| Agents-A1-4B | 66.8% | 17.7% | estimated ± 13.8 pp, low confidence |
| Atria Dawn Preview | 92.5% | 68.6% | estimated ± 13.8 pp, low confidence |
| GPT-5.4 Pro | 89.3% | 67.8% | estimated ± 13.8 pp, low confidence |
| GPT-5.5 Pro | 90.1% | 68.5% | estimated ± 13.8 pp, low confidence |
| Ling 3.0 Flash | 72.2% | 17.7% | estimated ± 13.8 pp, low confidence |
| LongCat-Flash-Lite-Sparse | 48.6% | 17.7% | estimated ± 13.8 pp, low confidence |
| Nemotron 3.5 Lightning 30B A3B NVFP4 | 36.8% | 17.7% | estimated ± 13.8 pp, low confidence |
| Beam | 77.4% | 17.7% | estimated ± 13.8 pp, low confidence |
| Solar Pro 4 | 49.2% | 17.7% | estimated ± 13.8 pp, low confidence |