Calibration
SimpleQA → MMLU-Pro
MMLU-Pro is estimated from SimpleQA with a Michaelis–Menten + offset curve fitted on 7 models measured on both: y = 0.0000 + 0.8948·x / (0.03317 + x), R² = 0.75, cross-validated error 4.8 pp. It is used for 1 estimate.
| Estimated model | SimpleQA | MMLU-Pro | Source |
|---|---|---|---|
| Gemini 2.0 Flash-Lite | 16.5% | 74.5% | estimated ± 4.8 pp, medium confidence |
Fitted on
| Model | SimpleQA | MMLU-Pro |
|---|---|---|
| DeepSeek-R1 | 30.1% | 84.0% |
| DeepSeek-R1-0528 | 27.8% | 85.0% |
| DeepSeek V3 | 24.9% | 75.9% |
| Grok 3 [Beta] | 43.6% | 79.9% |
| Kimi K2 | 31.0% | 81.1% |
| Mistral Small 3.1 | 10.4% | 66.8% |
| Qwen3 235B 2507 | 54.3% | 83.0% |