benchgap
Calibration

MMMLU → MMLU-Redux

MMLU-Redux is estimated from MMMLU with a offset logistic curve fitted on 7 models measured on both: y = 0.0002 + (0.9667 − 0.0002) / (1 + exp(−14.08·(x − 0.6191))), R² = 0.99, cross-validated error 0.5 pp. It is used for 7 estimates.

Estimated modelMMMLUMMLU-ReduxSource
Interfaze Beta90.9%95.1%estimated ± 0.5 pp, low confidence
Claude 4 Opus Thinking88.8%94.5%estimated ± 0.5 pp, medium confidence
Claude 4 Opus87.4%94.1%estimated ± 0.5 pp, medium confidence
K-EXAONE 2.086.6%93.8%estimated ± 0.5 pp, medium confidence
Claude 3.7 Sonnet Thinking86.1%93.6%estimated ± 0.5 pp, medium confidence
Gemma 4 12B83.4%92.2%estimated ± 0.5 pp, medium confidence
Claude 3.7 Sonnet83.2%92.1%estimated ± 0.5 pp, medium confidence

Fitted on

ModelMMMLUMMLU-Redux
Qwen3 235B-A22B79.8%89.2%
Qwen3 235B-A22B (Reasoning)84.3%92.7%
Qwen3 32B76.5%85.7%
Qwen3 32B (Reasoning)80.6%90.9%
Qwen3.7 Max90.3%95.0%
Qwen3.7 Plus89.0%94.5%
QwQ-32B80.9%90.0%