benchgap
Calibration

CharXiv → MMVU

MMVU is estimated from CharXiv with a Michaelis–Menten + offset curve fitted on 6 models measured on both: y = 0.2844 + 2.0000·x / (2.54828 + x), R² = 0.54, cross-validated error 3.6 pp. It is used for 14 estimates.

Estimated modelCharXivMMVUSource
Claude Mythos 593.5%82.1%estimated ± 3.6 pp, medium confidence
Glimpse92.9%81.9%estimated ± 3.6 pp, medium confidence
Claude Opus 4.791.0%81.1%estimated ± 3.6 pp, medium confidence
LongCat-2.590.2%80.7%estimated ± 3.6 pp, medium confidence
Claude Opus 4.889.9%80.6%estimated ± 3.6 pp, medium confidence
Gemini 3.7 Flash88.7%80.1%estimated ± 3.6 pp, medium confidence
Muse Spark 1.188.4%79.9%estimated ± 3.6 pp, medium confidence
Claude Sonnet 588.3%79.9%estimated ± 3.6 pp, medium confidence
Sakana Fugu-Ultra86.6%79.2%estimated ± 3.6 pp, medium confidence
Sakana Fugu85.1%78.5%estimated ± 3.6 pp, medium confidence
MiniMax M3.1 Flash82.5%77.4%estimated ± 3.6 pp, medium confidence
Claude Sonnet 4.677.4%75.0%estimated ± 3.6 pp, medium confidence
Nemotron 3 Nano Omni 30B A3B76.3%74.5%estimated ± 3.6 pp, low confidence
Gemini 3.1 Flash-Lite73.2%73.1%estimated ± 3.6 pp, low confidence

Fitted on

ModelCharXivMMVU
GLM-5.3-Flash89.4%80.5%
Kimi K2.577.5%80.4%
Qwen3.5-122B-A10B77.2%74.7%
Qwen3.5-27B79.5%73.3%
Qwen3.5-35B-A3B77.5%72.3%
Qwen3.8 Max93.5%82.4%