Calibration
MMLU-Pro → MMLU-Redux
MMLU-Redux is estimated from MMLU-Pro with a Michaelis–Menten + offset curve fitted on 8 models measured on both: y = 0.2153 + 2.0000·x / (1.52755 + x), R² = 1.00, cross-validated error 0.7 pp. It is used for 38 estimates.
| Estimated model | MMLU-Pro | MMLU-Redux | Source |
|---|---|---|---|
| Celeris-1 | 75.9% | 87.9% | estimated ± 0.7 pp, high confidence |
| Claude Opus 4.6 | 82.0% | 91.4% | estimated ± 0.7 pp, high confidence |
| Claude Sonnet 4.6 | 79.2% | 89.8% | estimated ± 0.7 pp, high confidence |
| DeepSeek V3 | 75.9% | 87.9% | estimated ± 0.7 pp, high confidence |
| DeepSeek V4 Flash 0731 | 86.2% | 93.7% | estimated ± 0.7 pp, high confidence |
| DeepSeek V4 Pro 0813 | 87.5% | 94.4% | estimated ± 0.7 pp, high confidence |
| Exaone 4.0 32B | 81.8% | 91.3% | estimated ± 0.7 pp, high confidence |
| Gemma 4 12B | 77.2% | 88.7% | estimated ± 0.7 pp, high confidence |
| Gemma 4 26B A4B | 82.6% | 91.7% | estimated ± 0.7 pp, high confidence |
| Gemma 4 31B | 85.2% | 93.1% | estimated ± 0.7 pp, high confidence |
| Gemma 4 E2B | 60.0% | 77.9% | estimated ± 0.7 pp, high confidence |
| Gemma 4 E4B | 69.4% | 84.0% | estimated ± 0.7 pp, high confidence |
| GLM-4.7 | 84.3% | 92.7% | estimated ± 0.7 pp, high confidence |
| GLM-5 | 85.7% | 93.4% | estimated ± 0.7 pp, high confidence |
| Granite 4.2 30B | 77.6% | 88.9% | estimated ± 0.7 pp, high confidence |
| Granite 4.2 3B | 67.8% | 83.0% | estimated ± 0.7 pp, high confidence |
| Granite 4.2 8B | 74.0% | 86.8% | estimated ± 0.7 pp, high confidence |
| K-EXAONE 2.0 | 83.5% | 92.2% | estimated ± 0.7 pp, high confidence |
| Kimi K2.5 | 87.1% | 94.2% | estimated ± 0.7 pp, high confidence |
| Kimi K2.5 (Reasoning) | 87.1% | 94.2% | estimated ± 0.7 pp, high confidence |
| LFM2.5-230M | 20.3% | 44.9% | estimated ± 0.7 pp, medium confidence |
| LFM2.5-VL-450M | 19.3% | 44.0% | estimated ± 0.7 pp, medium confidence |
| LongCat-Flash-Lite-Sparse | 79.2% | 89.8% | estimated ± 0.7 pp, high confidence |
| MAI-Thinking-1 | 85.0% | 93.0% | estimated ± 0.7 pp, high confidence |
| MiMo-V2-Flash | 84.9% | 93.0% | estimated ± 0.7 pp, high confidence |
| Nemotron 3.5 Lightning 30B A3B NVFP4 | 81.6% | 91.2% | estimated ± 0.7 pp, high confidence |
| Nemotron 3 Nano Omni 30B A3B | 77.3% | 88.7% | estimated ± 0.7 pp, high confidence |
| Nemotron 3 Ultra | 86.8% | 94.0% | estimated ± 0.7 pp, high confidence |
| Qwen3 235B 2507 | 83.0% | 91.9% | estimated ± 0.7 pp, high confidence |
| Qwen3.5-122B-A10B | 86.7% | 93.9% | estimated ± 0.7 pp, high confidence |
| Qwen3.5-27B | 86.1% | 93.6% | estimated ± 0.7 pp, high confidence |
| Qwen3.5-35B-A3B | 85.3% | 93.2% | estimated ± 0.7 pp, high confidence |
| Qwen3.6-35B-A3B | 85.2% | 93.1% | estimated ± 0.7 pp, high confidence |
| Solar Open 2 | 86.2% | 93.7% | estimated ± 0.7 pp, high confidence |
| Solar Pro 4 | 86.3% | 93.7% | estimated ± 0.7 pp, high confidence |
| Soofi S 30B-A3B | 51.4% | 71.9% | estimated ± 0.7 pp, high confidence |
| ZAYA1-74B-Preview | 68.1% | 83.2% | estimated ± 0.7 pp, high confidence |
| ZAYA1-8B | 74.2% | 86.9% | estimated ± 0.7 pp, high confidence |