Calibration
CharXiv w/o tools → MMMU-Pro
MMMU-Pro is estimated from CharXiv w/o tools with a inverse Michaelis–Menten curve fitted on 9 models measured on both: y = 1.79877·(x − 0.2318) / (0.2318 + 2.0000 − x), R² = 0.80, cross-validated error 6.6 pp. It is used for 0 estimates.
Fitted on
| Model | CharXiv w/o tools | MMMU-Pro |
|---|---|---|
| dots3-note Preview | 83.1% | 79.1% |
| Inkling | 78.1% | 73.5% |
| Inkling 2 Mini | 80.6% | 74.9% |
| Inkling 2 Small | 85.6% | 81.6% |
| Inkling-Small | 77.4% | 74.0% |
| Kimi K3.1 Pro | 89.0% | 84.1% |
| Kimi K3 | 84.8% | 81.6% |
| LFM3-VL-2B | 70.2% | 45.7% |
| Qwen3.8 Max | 88.4% | 82.3% |