Calibration
MMLU-Redux → SuperGPQA
SuperGPQA is estimated from MMLU-Redux with a Hill curve fitted on 9 models measured on both: y = 0.0000 + (1.2000 − 0.0000)·x^6.00 / (0.89960^6.00 + x^6.00), R² = 0.96, cross-validated error 4.1 pp. It is used for 6 estimates.
| Estimated model | MMLU-Redux | SuperGPQA | Source |
|---|---|---|---|
| DeepSeek-R1-0528 | 93.4% | 66.7% | estimated ± 4.1 pp, high confidence |
| Ternary Bonsai 2 27B | 89.1% | 58.3% | estimated ± 4.1 pp, high confidence |
| Mellum2.1-12B-A2.5B-Thinking | 87.8% | 55.6% | estimated ± 4.1 pp, high confidence |
| Mellum2-12B-A2.5B-Thinking | 86.2% | 52.4% | estimated ± 4.1 pp, high confidence |
| Qwen3 32B | 85.7% | 51.3% | estimated ± 4.1 pp, high confidence |
| Mellum2-12B-A2.5B-Instruct | 78.1% | 36.0% | estimated ± 4.1 pp, high confidence |
Fitted on
| Model | MMLU-Redux | SuperGPQA |
|---|---|---|
| Claude Opus 4.5 | 96.6% | 70.6% |
| Kimi K2.5 | 94.5% | 69.2% |
| MiniCPM5-1B | 70.1% | 23.1% |
| MiniCPM5-2B | 84.7% | 40.8% |
| Qwen3.5 397B | 94.9% | 70.4% |
| Qwen3.6-27B | 93.5% | 66.0% |
| Qwen3.6 Plus | 94.5% | 71.6% |
| Qwen3.7 Max | 95.0% | 73.6% |
| Qwen3.7 Plus | 94.5% | 71.4% |