Calibration
GPQA Diamond → Vals MMLU-Pro
Vals MMLU-Pro is estimated from GPQA Diamond with a Michaelis–Menten curve fitted on 23 models measured on both: y = 2.0000·x / (1.17813 + x), R² = 0.57, cross-validated error 1.6 pp. It is used for 14 estimates.
| Estimated model | GPQA Diamond | Vals MMLU-Pro | Source |
|---|---|---|---|
| A.X K2 | 85.6% | 84.2% | estimated ± 1.6 pp, high confidence |
| K-EXAONE 2.0 | 82.2% | 82.2% | estimated ± 1.6 pp, medium confidence |
| LLaDA2.2-mini | 44.4% | 54.8% | estimated ± 1.6 pp, medium confidence |
| LongCat-Flash-Lite-Sparse | 69.5% | 74.2% | estimated ± 1.6 pp, medium confidence |
| Mercury 2.5 | 79.0% | 80.3% | estimated ± 1.6 pp, medium confidence |
| MiniCPM5-1B | 26.3% | 36.5% | estimated ± 1.6 pp, medium confidence |
| MiniCPM5-2B | 70.2% | 74.7% | estimated ± 1.6 pp, medium confidence |
| Pareto 26.10 Preview | 92.4% | 87.9% | estimated ± 1.6 pp, high confidence |
| Beam | 90.5% | 86.9% | estimated ± 1.6 pp, high confidence |
| Solar Open 2 | 86.3% | 84.6% | estimated ± 1.6 pp, high confidence |
| Solar Pro 4 | 89.0% | 86.1% | estimated ± 1.6 pp, high confidence |
| Step 5 Preview | 93.5% | 88.5% | estimated ± 1.6 pp, high confidence |
| Trinity-Large-Preview | 63.3% | 69.9% | estimated ± 1.6 pp, medium confidence |
| Trinity-Large-Thinking | 76.3% | 78.6% | estimated ± 1.6 pp, medium confidence |