benchgap
Calibration

MMMU-Pro → OCRBench V2

OCRBench V2 is estimated from MMMU-Pro with a Michaelis–Menten curve fitted on 5 models measured on both: y = 1.0251·x / (0.32293 + x), R² = 0.95, cross-validated error 4.4 pp. It is used for 42 estimates.

Estimated modelMMMU-ProOCRBench V2Source
GPT-6.1 Terra85.5%74.4%estimated ± 4.4 pp, low confidence
Kimi K3.1 Pro84.1%74.1%estimated ± 4.4 pp, low confidence
Gemini 3.1 Pro83.9%74.0%estimated ± 4.4 pp, low confidence
Gemini 3.5 Flash83.6%73.9%estimated ± 4.4 pp, low confidence
GPT-5.6 Sol83.0%73.8%estimated ± 4.4 pp, low confidence
Inkling 2 Small81.6%73.4%estimated ± 4.4 pp, medium confidence
Kimi K381.6%73.4%estimated ± 4.4 pp, medium confidence
Seed 2.1 Pro81.6%73.4%estimated ± 4.4 pp, medium confidence
GPT-5.481.2%73.3%estimated ± 4.4 pp, medium confidence
GPT-5.581.2%73.3%estimated ± 4.4 pp, medium confidence
GPT-5.6 Terra80.7%73.2%estimated ± 4.4 pp, medium confidence
Muse Spark80.4%73.1%estimated ± 4.4 pp, medium confidence
Seed 2.1 Turbo80.1%73.1%estimated ± 4.4 pp, medium confidence
Kimi K2.679.4%72.9%estimated ± 4.4 pp, medium confidence
dots3-note Preview79.1%72.8%estimated ± 4.4 pp, medium confidence
Qwen3.6 Plus78.8%72.7%estimated ± 4.4 pp, medium confidence
Hy478.6%72.7%estimated ± 4.4 pp, medium confidence
GPT-5.6 Luna78.4%72.6%estimated ± 4.4 pp, medium confidence
Grok 4.378.1%72.5%estimated ± 4.4 pp, medium confidence
MiniMax M378.1%72.5%estimated ± 4.4 pp, medium confidence
Pareto 26.978.0%72.5%estimated ± 4.4 pp, medium confidence
MiMo-V2.577.9%72.5%estimated ± 4.4 pp, medium confidence
Claude Opus 4.677.3%72.3%estimated ± 4.4 pp, medium confidence
Gemma 4 31B76.9%72.2%estimated ± 4.4 pp, medium confidence
Qwen3.5-122B-A10B76.9%72.2%estimated ± 4.4 pp, medium confidence
GPT-5.4 mini76.6%72.1%estimated ± 4.4 pp, medium confidence
Step 5 Preview76.0%71.9%estimated ± 4.4 pp, medium confidence
Hy4 flash75.8%71.9%estimated ± 4.4 pp, medium confidence
Grok 4.2075.2%71.7%estimated ± 4.4 pp, medium confidence
Qwen3.5-35B-A3B75.1%71.7%estimated ± 4.4 pp, medium confidence
Qwen3.5-27B75.0%71.7%estimated ± 4.4 pp, medium confidence
Inkling-Small74.0%71.4%estimated ± 4.4 pp, medium confidence
Muse Glimmer 30B74.0%71.4%estimated ± 4.4 pp, medium confidence
Gemma 4 26B A4B73.8%71.3%estimated ± 4.4 pp, medium confidence
Inkling73.5%71.2%estimated ± 4.4 pp, medium confidence
Gemma 4 12B69.1%69.9%estimated ± 4.4 pp, medium confidence
GPT-5.4 nano66.1%68.9%estimated ± 4.4 pp, medium confidence
Command A+63.0%67.8%estimated ± 4.4 pp, medium confidence
Gemma 4 E4B52.6%63.5%estimated ± 4.4 pp, medium confidence
Qwen2.5-VL-32B49.5%62.0%estimated ± 4.4 pp, medium confidence
Mistral Small 3.149.3%61.9%estimated ± 4.4 pp, medium confidence
Gemma 4 E2B44.2%59.2%estimated ± 4.4 pp, medium confidence

Fitted on

ModelMMMU-ProOCRBench V2
Interfaze Beta71.1%70.7%
LFM2.5-VL-3B30.5%47.5%
LFM3-VL-2B45.7%63.5%
Qwen3.7 Plus79.0%70.7%
Qwen3.8 Max82.3%74.2%