benchgap
Calibration

MMLU-ProX → SWE Multilingual

SWE Multilingual is estimated from MMLU-ProX with a inverse Michaelis–Menten curve fitted on 8 models measured on both: y = 0.84681·(x − 0.0000) / (0.0000 + 1.8176 − x), R² = 0.48, cross-validated error 3.2 pp. It is used for 4 estimates.

Estimated modelMMLU-ProXSWE MultilingualSource
Qwen3.5-122B-A10B82.2%69.9%estimated ± 3.2 pp, low confidence
Qwen3.5-27B82.2%69.9%estimated ± 3.2 pp, low confidence
Qwen3.5-35B-A3B81.0%68.1%estimated ± 3.2 pp, low confidence
Qwen3 235B 250779.4%65.7%estimated ± 3.2 pp, low confidence

Fitted on

ModelMMLU-ProXSWE Multilingual
Claude Opus 4.585.7%77.5%
GLM-583.1%73.3%
Kimi K2.582.3%73.0%
Nemotron 3 Ultra83.0%67.7%
Qwen3.5 397B84.7%69.3%
Qwen3.6 Plus84.7%73.8%
Qwen3.7 Max87.0%78.3%
Qwen3.7 Plus85.4%75.8%