Calibration
GPQA → GPQA Diamond
GPQA Diamond is estimated from GPQA with a linear curve fitted on 48 models measured on both: y = 0.9990·x + 0.0005, R² = 1.00, cross-validated error 0.4 pp. It is used for 29 estimates.
| Estimated model | GPQA | GPQA Diamond | Source |
|---|---|---|---|
| Claude 3.5 Sonnet | 59.4% | 59.4% | estimated ± 0.4 pp, high confidence |
| Claude Mythos 5 | 94.1% | 94.1% | estimated ± 0.4 pp, high confidence |
| Claude Opus 4.5 | 87.0% | 87.0% | estimated ± 0.4 pp, high confidence |
| Claude Sonnet 4.5 | 83.4% | 83.4% | estimated ± 0.4 pp, high confidence |
| DeepSeek V3 | 59.1% | 59.1% | estimated ± 0.4 pp, high confidence |
| Gemini 2.5 Pro | 83.0% | 83.0% | estimated ± 0.4 pp, high confidence |
| Gemma 4 31B | 84.3% | 84.3% | estimated ± 0.4 pp, high confidence |
| Gemma 4 E2B | 43.4% | 43.4% | estimated ± 0.4 pp, high confidence |
| Gemma 4 E4B | 58.6% | 58.6% | estimated ± 0.4 pp, high confidence |
| GPT-4.1 | 66.3% | 66.3% | estimated ± 0.4 pp, high confidence |
| GPT-4.1 mini | 64.2% | 64.2% | estimated ± 0.4 pp, high confidence |
| GPT-4.1 nano | 50.3% | 50.3% | estimated ± 0.4 pp, high confidence |
| GPT-5.2 | 92.4% | 92.4% | estimated ± 0.4 pp, high confidence |
| Granite 4.2 30B | 66.4% | 66.4% | estimated ± 0.4 pp, high confidence |
| Granite 4.2 3B | 54.8% | 54.8% | estimated ± 0.4 pp, high confidence |
| Granite 4.2 8B | 64.1% | 64.1% | estimated ± 0.4 pp, high confidence |
| Kimi K2.5 (Reasoning) | 87.6% | 87.6% | estimated ± 0.4 pp, high confidence |
| LFM2.5-VL-450M | 25.7% | 25.7% | estimated ± 0.4 pp, high confidence |
| Ling 2.6 Flash | 59.0% | 59.0% | estimated ± 0.4 pp, high confidence |
| o1 | 75.7% | 75.7% | estimated ± 0.4 pp, high confidence |
| o1-pro | 79.0% | 79.0% | estimated ± 0.4 pp, high confidence |
| o3-mini | 77.2% | 77.2% | estimated ± 0.4 pp, high confidence |
| Qwen3 235B 2507 | 77.5% | 77.5% | estimated ± 0.4 pp, high confidence |
| Qwen3.5-122B-A10B | 86.6% | 86.6% | estimated ± 0.4 pp, high confidence |
| Qwen3.5-27B | 85.5% | 85.5% | estimated ± 0.4 pp, high confidence |
| Qwen3.5-35B-A3B | 84.2% | 84.2% | estimated ± 0.4 pp, high confidence |
| Qwen3.5 397B | 88.4% | 88.4% | estimated ± 0.4 pp, high confidence |
| Qwen3.6-27B | 87.8% | 87.8% | estimated ± 0.4 pp, high confidence |
| Qwen3.6-35B-A3B | 86.0% | 86.0% | estimated ± 0.4 pp, high confidence |