Calibration
MMLU-Redux → MMLU-Pro
MMLU-Pro is estimated from MMLU-Redux with a inverse Michaelis–Menten curve fitted on 8 models measured on both: y = 1.50020·(x − 0.2072) / (0.2072 + 2.0000 − x), R² = 0.99, cross-validated error 1.4 pp. It is used for 3 estimates.
| Estimated model | MMLU-Redux | MMLU-Pro | Source |
|---|---|---|---|
| Mellum2-12B-A2.5B-Instruct | 78.1% | 60.4% | estimated ± 1.4 pp, high confidence |
| Mellum2-12B-A2.5B-Thinking | 86.2% | 73.0% | estimated ± 1.4 pp, high confidence |
| Ternary Bonsai 2 27B | 89.1% | 77.9% | estimated ± 1.4 pp, high confidence |