benchgap
Calibration

CharXiv w/o tools → MMMU-Pro

MMMU-Pro is estimated from CharXiv w/o tools with a inverse Michaelis–Menten curve fitted on 9 models measured on both: y = 1.79877·(x − 0.2318) / (0.2318 + 2.0000 − x), R² = 0.80, cross-validated error 6.6 pp. It is used for 0 estimates.

Fitted on

ModelCharXiv w/o toolsMMMU-Pro
dots3-note Preview83.1%79.1%
Inkling78.1%73.5%
Inkling 2 Mini80.6%74.9%
Inkling 2 Small85.6%81.6%
Inkling-Small77.4%74.0%
Kimi K3.1 Pro89.0%84.1%
Kimi K384.8%81.6%
LFM3-VL-2B70.2%45.7%
Qwen3.8 Max88.4%82.3%