benchgap
Calibration

MMLU-ProX → PolyMath

PolyMath is estimated from MMLU-ProX with a offset logistic curve fitted on 5 models measured on both: y = 0.3824 + (0.8666 − 0.3824) / (1 + exp(−133.95·(x − 0.8394))), R² = 0.98, cross-validated error 11.4 pp. It is used for 7 estimates.

Estimated modelMMLU-ProXPolyMathSource
Qwen3.6 Plus84.7%73.8%estimated ± 11.4 pp, low confidence
GLM-583.1%50.1%estimated ± 11.4 pp, low confidence
Nemotron 3 Ultra83.0%49.0%estimated ± 11.4 pp, low confidence
Qwen3.5-122B-A10B82.2%42.5%estimated ± 11.4 pp, low confidence
Qwen3.5-27B82.2%42.5%estimated ± 11.4 pp, low confidence
Qwen3.5-35B-A3B81.0%39.2%estimated ± 11.4 pp, low confidence
Qwen3 235B 250779.4%38.4%estimated ± 11.4 pp, low confidence

Fitted on

ModelMMLU-ProXPolyMath
Claude Opus 4.585.7%79.0%
Kimi K2.582.3%43.1%
Qwen3.5 397B84.7%73.3%
Qwen3.7 Max87.0%86.5%
Qwen3.7 Plus85.4%84.0%