Calibration
OSWorld 2.0 → AA-AnalystAgent
AA-AnalystAgent is estimated from OSWorld 2.0 with a Hill curve fitted on 8 models measured on both: y = 0.0919 + (0.5438 − 0.0919)·x^6.00 / (0.08961^6.00 + x^6.00), R² = 0.94, cross-validated error 5.5 pp. It is used for 14 estimates.
| Estimated model | OSWorld 2.0 | AA-AnalystAgent | Source |
|---|---|---|---|
| Claude Opus 4.7 | 13.9% | 51.4% | estimated ± 5.5 pp, medium confidence |
| Claude Opus 4.7 (Adaptive) | 18.2% | 53.7% | estimated ± 5.5 pp, medium confidence |
| Claude Opus 4.8 | 20.6% | 54.1% | estimated ± 5.5 pp, medium confidence |
| Claude Sonnet 4.6 | 8.3% | 26.7% | estimated ± 5.5 pp, medium confidence |
| Gemini 3.8 Flash | 59.0% | 54.4% | estimated ± 5.5 pp, medium confidence |
| Gemini 4 Argon | 69.2% | 54.4% | estimated ± 5.5 pp, medium confidence |
| GPT-5.6 Luna | 45.6% | 54.4% | estimated ± 5.5 pp, medium confidence |
| GPT-5.6 Terra | 50.2% | 54.4% | estimated ± 5.5 pp, medium confidence |
| GPT-6 Sol | 60.5% | 54.4% | estimated ± 5.5 pp, medium confidence |
| Kimi K2.6 | 4.6% | 10.0% | estimated ± 5.5 pp, medium confidence |
| Muse Spark 1.1 | 14.2% | 51.7% | estimated ± 5.5 pp, medium confidence |
| Muse Spark 1.3 | 66.9% | 54.4% | estimated ± 5.5 pp, medium confidence |
| Qwen3.7 Plus | 2.8% | 9.2% | estimated ± 5.5 pp, low confidence |
| Qwen3.8-Flash-Next | 19.4% | 53.9% | estimated ± 5.5 pp, medium confidence |