Calibration
MMMLU → MMLU-Redux
MMLU-Redux is estimated from MMMLU with a offset logistic curve fitted on 7 models measured on both: y = 0.0002 + (0.9667 − 0.0002) / (1 + exp(−14.08·(x − 0.6191))), R² = 0.99, cross-validated error 0.5 pp. It is used for 7 estimates.
| Estimated model | MMMLU | MMLU-Redux | Source |
|---|---|---|---|
| Interfaze Beta | 90.9% | 95.1% | estimated ± 0.5 pp, low confidence |
| Claude 4 Opus Thinking | 88.8% | 94.5% | estimated ± 0.5 pp, medium confidence |
| Claude 4 Opus | 87.4% | 94.1% | estimated ± 0.5 pp, medium confidence |
| K-EXAONE 2.0 | 86.6% | 93.8% | estimated ± 0.5 pp, medium confidence |
| Claude 3.7 Sonnet Thinking | 86.1% | 93.6% | estimated ± 0.5 pp, medium confidence |
| Gemma 4 12B | 83.4% | 92.2% | estimated ± 0.5 pp, medium confidence |
| Claude 3.7 Sonnet | 83.2% | 92.1% | estimated ± 0.5 pp, medium confidence |
Fitted on
| Model | MMMLU | MMLU-Redux |
|---|---|---|
| Qwen3 235B-A22B | 79.8% | 89.2% |
| Qwen3 235B-A22B (Reasoning) | 84.3% | 92.7% |
| Qwen3 32B | 76.5% | 85.7% |
| Qwen3 32B (Reasoning) | 80.6% | 90.9% |
| Qwen3.7 Max | 90.3% | 95.0% |
| Qwen3.7 Plus | 89.0% | 94.5% |
| QwQ-32B | 80.9% | 90.0% |