Calibration
SimpleQA → MMLU
MMLU is estimated from SimpleQA with a Michaelis–Menten curve fitted on 6 models measured on both: y = 0.9250·x / (0.00914 + x), R² = 0.64, cross-validated error 4.1 pp. It is used for 1 estimate.
| Estimated model | SimpleQA | MMLU | Source |
|---|---|---|---|
| Gemini 2.0 Flash-Lite | 16.5% | 87.6% | estimated ± 4.1 pp, medium confidence |
Fitted on
| Model | SimpleQA | MMLU |
|---|---|---|
| DeepSeek-R1 | 30.1% | 90.8% |
| GPT-4.5 | 62.5% | 90.8% |
| Kimi K2 | 31.0% | 89.5% |
| Mistral Small 3.1 | 10.4% | 80.6% |
| o1-mini | 7.6% | 85.2% |
| o4-mini | 20.2% | 90.0% |