Calibration
ScreenSpot Pro → SimpleVQA
SimpleVQA is estimated from ScreenSpot Pro with a Hill curve fitted on 5 models measured on both: y = 1.0000 + (0.0000 − 1.0000)·x^6.00 / (0.97887^6.00 + x^6.00), R² = 0.55, cross-validated error 5.7 pp. It is used for 13 estimates.
| Estimated model | ScreenSpot Pro | SimpleVQA | Source |
|---|---|---|---|
| Claude Opus 4.5 | 45.7% | 99.0% | estimated ± 5.7 pp, low confidence |
| Claude Opus 4.6 | 83.1% | 72.8% | estimated ± 5.7 pp, low confidence |
| Claude Opus 4.8 | 87.9% | 65.6% | estimated ± 5.7 pp, low confidence |
| Gemini 3 Pro | 72.7% | 85.6% | estimated ± 5.7 pp, low confidence |
| GPT-6 Astra | 92.7% | 58.1% | estimated ± 5.7 pp, low confidence |
| Holo2-235B-A22B | 70.6% | 87.7% | estimated ± 5.7 pp, low confidence |
| Holo2-30B-A3B | 66.1% | 91.3% | estimated ± 5.7 pp, low confidence |
| Holo2-4B | 57.2% | 96.2% | estimated ± 5.7 pp, low confidence |
| Holo2-8B | 58.9% | 95.5% | estimated ± 5.7 pp, low confidence |
| Muse Glimmer 30B | 75.4% | 82.7% | estimated ± 5.7 pp, low confidence |
| Nemotron 3 Nano Omni 30B A3B | 57.8% | 95.9% | estimated ± 5.7 pp, low confidence |
| Qwen3.5 397B | 65.6% | 91.7% | estimated ± 5.7 pp, low confidence |
| Qwen3.6 Plus | 68.2% | 89.7% | estimated ± 5.7 pp, low confidence |