Calibration
SimpleQA → GPQA
GPQA is estimated from SimpleQA with a Michaelis–Menten curve fitted on 14 models measured on both: y = 0.8069·x / (0.04361 + x), R² = 0.48, cross-validated error 9.1 pp. It is used for 0 estimates.
Fitted on
| Model | SimpleQA | GPQA |
|---|---|---|
| DeepSeek-R1 | 30.1% | 71.5% |
| DeepSeek-R1-0528 | 27.8% | 81.0% |
| DeepSeek V3 | 24.9% | 59.1% |
| Gemini 1.5 Flash | 8.6% | 50.0% |
| Gemini 2.0 Flash | 29.9% | 65.2% |
| Gemini 2.0 Flash-Lite | 16.5% | 50.5% |
| Gemini 2.5 Flash-Lite | 10.7% | 64.6% |
| GPT-4.5 | 62.5% | 69.5% |
| Grok 3 [Beta] | 43.6% | 75.4% |
| Kimi K2 | 31.0% | 75.1% |
| Mistral Small 3.1 | 10.4% | 46.0% |
| o1-mini | 7.6% | 60.0% |
| o4-mini | 20.2% | 77.6% |
| Qwen3 235B 2507 | 54.3% | 77.5% |