Calibration
CharXiv → MMMU-Pro
MMMU-Pro is estimated from CharXiv with a Michaelis–Menten curve fitted on 23 models measured on both: y = 1.2710·x / (0.50191 + x), R² = 0.64, cross-validated error 3.2 pp. It is used for 6 estimates.
| Estimated model | CharXiv | MMMU-Pro | Source |
|---|---|---|---|
| Claude Sonnet 4.6 | 77.4% | 77.1% | estimated ± 3.2 pp, high confidence |
| Gemini 3.1 Flash-Lite | 73.2% | 75.4% | estimated ± 3.2 pp, high confidence |
| GLM-5.3-Flash | 89.4% | 81.4% | estimated ± 3.2 pp, high confidence |
| Muse Spark 1.1 | 88.4% | 81.1% | estimated ± 3.2 pp, high confidence |
| Sakana Fugu | 85.1% | 79.9% | estimated ± 3.2 pp, high confidence |
| Sakana Fugu-Ultra | 86.6% | 80.5% | estimated ± 3.2 pp, high confidence |