Calibration
Vals GPQA Diamond → GPQA
GPQA is estimated from Vals GPQA Diamond with a Hill curve fitted on 25 models measured on both: y = 0.8051 + (1.2000 − 0.8051)·x^6.00 / (1.07066^6.00 + x^6.00), R² = 0.73, cross-validated error 2.6 pp. It is used for 0 estimates.
Fitted on
| Model | Vals GPQA Diamond | GPQA |
|---|---|---|
| Claude Opus 4.8 | 92.4% | 93.6% |
| Claude Sonnet 4.6 | 85.6% | 89.9% |
| DeepSeek V4 Flash 0731 | 89.9% | 88.1% |
| DeepSeek V4 Pro 0813 | 92.4% | 90.1% |
| Gemini 3.5 Flash | 92.7% | 92.2% |
| GLM-4.7 | 80.1% | 85.7% |
| GLM-5.2 | 85.6% | 91.2% |
| GPT-5.4 mini | 83.1% | 88.0% |
| GPT-5.4 nano | 77.5% | 82.8% |
| GPT-5.5 | 93.2% | 93.6% |
| GPT-5.6 Luna | 91.7% | 92.3% |
| GPT-5.6 Sol | 95.2% | 94.6% |
| GPT-5.6 Terra | 90.9% | 92.9% |
| Grok 4.3 | 91.4% | 90.1% |
| Inkling | 87.1% | 87.9% |
| Inkling-Small | 83.6% | 89.5% |
| Kimi K2.6 | 89.1% | 90.5% |
| Kimi K3 | 92.9% | 93.5% |
| Ling 3.0 Flash | 84.8% | 85.0% |
| MiMo-V2-Flash | 59.3% | 83.7% |
| Nemotron 3 Ultra | 86.1% | 87.0% |
| Qwen3.6 Plus | 87.4% | 90.4% |
| Qwen3.7 Max | 90.2% | 92.4% |
| Qwen3.8-27B | 88.9% | 89.2% |
| Qwen3.8 Max | 93.7% | 92.6% |