Calibration
ScreenSpot Pro → AA-MMMU-Pro
AA-MMMU-Pro is estimated from ScreenSpot Pro with a linear curve fitted on 12 models measured on both: y = 0.4105·x + 0.4570, R² = 0.42, cross-validated error 8.2 pp. It is used for 4 estimates.
| Estimated model | ScreenSpot Pro | AA-MMMU-Pro | Source |
|---|---|---|---|
| Holo2-235B-A22B | 70.6% | 74.7% | estimated ± 8.2 pp, low confidence |
| Holo2-30B-A3B | 66.1% | 72.8% | estimated ± 8.2 pp, low confidence |
| Holo2-8B | 58.9% | 69.9% | estimated ± 8.2 pp, low confidence |
| Holo2-4B | 57.2% | 69.2% | estimated ± 8.2 pp, low confidence |
Fitted on
| Model | ScreenSpot Pro | AA-MMMU-Pro |
|---|---|---|
| Claude Opus 4.5 | 45.7% | 71.2% |
| Claude Opus 4.6 | 83.1% | 72.5% |
| Gemini 3.1 Pro | 84.4% | 82.4% |
| Gemini 3 Pro | 72.7% | 80.2% |
| GPT-5.4 | 85.4% | 78.4% |
| GPT-6 Astra | 92.7% | 86.9% |
| Muse Glimmer 30B | 75.4% | 74.3% |
| Muse Spark | 84.1% | 80.5% |
| Nemotron 3 Nano Omni 30B A3B | 57.8% | 53.2% |
| Qwen3.5 397B | 65.6% | 77.3% |
| Qwen3.6 Plus | 68.2% | 78.0% |
| Qwen3.7 Plus | 79.0% | 80.5% |