Calibration
HLE → SuperGPQA
SuperGPQA is estimated from HLE with a linear curve fitted on 13 models measured on both: y = 0.9393·x + 0.4229, R² = 0.85, cross-validated error 7.2 pp. It is used for 63 estimates.
| Estimated model | HLE | SuperGPQA | Source |
|---|---|---|---|
| Claude Fable 5.1 | 65.0% | 100.0% | estimated ± 7.2 pp, low confidence |
| Claude Fable 5.2 | 74.2% | 100.0% | estimated ± 7.2 pp, low confidence |
| Claude Mythos 5 | 64.5% | 100.0% | estimated ± 7.2 pp, low confidence |
| Claude Opus 5 | 64.7% | 100.0% | estimated ± 7.2 pp, low confidence |
| Kimi K3.1 Pro | 70.0% | 100.0% | estimated ± 7.2 pp, low confidence |
| Muse Spark 1.1 | 62.1% | 100.0% | estimated ± 7.2 pp, low confidence |
| Hy4 | 59.1% | 97.8% | estimated ± 7.2 pp, low confidence |
| GPT-5.4 Pro | 58.7% | 97.4% | estimated ± 7.2 pp, low confidence |
| Claude Opus 4.8 | 57.9% | 96.7% | estimated ± 7.2 pp, low confidence |
| Claude Sonnet 5 | 57.4% | 96.2% | estimated ± 7.2 pp, low confidence |
| GPT-5.5 Pro | 57.2% | 96.0% | estimated ± 7.2 pp, low confidence |
| Apodex 1.1 | 56.1% | 95.0% | estimated ± 7.2 pp, low confidence |
| Kimi K3 | 56.0% | 94.9% | estimated ± 7.2 pp, low confidence |
| Hy4 preview | 55.4% | 94.3% | estimated ± 7.2 pp, low confidence |
| Hy4 flash | 55.2% | 94.1% | estimated ± 7.2 pp, low confidence |
| Claude Opus 4.7 | 54.7% | 93.7% | estimated ± 7.2 pp, low confidence |
| Inkling 2 Small | 53.2% | 92.3% | estimated ± 7.2 pp, low confidence |
| dots3-note Preview | 52.6% | 91.7% | estimated ± 7.2 pp, medium confidence |
| GPT-5.5 | 52.2% | 91.3% | estimated ± 7.2 pp, medium confidence |
| GPT-5.4 | 52.1% | 91.2% | estimated ± 7.2 pp, medium confidence |
| Qwen4 Max | 51.7% | 90.8% | estimated ± 7.2 pp, medium confidence |
| Muse Spark | 50.4% | 89.6% | estimated ± 7.2 pp, medium confidence |
| MiMo-V2.5-Pro | 48.0% | 87.4% | estimated ± 7.2 pp, medium confidence |
| Inkling-Small | 47.8% | 87.2% | estimated ± 7.2 pp, medium confidence |
| Agents-A1 | 47.6% | 87.0% | estimated ± 7.2 pp, medium confidence |
| Step 5 Preview | 46.5% | 86.0% | estimated ± 7.2 pp, medium confidence |
| Inkling | 46.0% | 85.5% | estimated ± 7.2 pp, medium confidence |
| Ornith-1.5-397B | 44.6% | 84.2% | estimated ± 7.2 pp, medium confidence |
| Qwen3.8 Max | 43.6% | 83.2% | estimated ± 7.2 pp, medium confidence |
| DeepSeek V4 Pro | 42.7% | 82.4% | estimated ± 7.2 pp, medium confidence |
| DeepSeek V4.1 Flash 1017 | 42.6% | 82.3% | estimated ± 7.2 pp, medium confidence |
| GPT-5.4 mini | 41.5% | 81.3% | estimated ± 7.2 pp, medium confidence |
| GLM-5.2 | 40.5% | 80.3% | estimated ± 7.2 pp, medium confidence |
| Gemini 3.5 Flash | 40.2% | 80.0% | estimated ± 7.2 pp, medium confidence |
| GPT-5.4 nano | 37.7% | 77.7% | estimated ± 7.2 pp, medium confidence |
| Ling 3.1 Flash | 37.4% | 77.5% | estimated ± 7.2 pp, medium confidence |
| Qwen3.8-Omni-Flash | 36.5% | 76.6% | estimated ± 7.2 pp, medium confidence |
| Qwen3.8-Flash-Next | 35.9% | 76.0% | estimated ± 7.2 pp, medium confidence |
| Grok 4.3 | 35.0% | 75.2% | estimated ± 7.2 pp, medium confidence |
| DeepSeek V4 Flash 0731 | 34.8% | 75.0% | estimated ± 7.2 pp, medium confidence |
| Kimi K2.6 | 34.7% | 74.9% | estimated ± 7.2 pp, medium confidence |
| GLM-5.1 | 31.0% | 71.4% | estimated ± 7.2 pp, medium confidence |
| Qwen3.8-27B | 30.8% | 71.2% | estimated ± 7.2 pp, medium confidence |
| Nemotron 3.5 Super VL 120B A12B | 29.2% | 69.7% | estimated ± 7.2 pp, medium confidence |
| Nemotron 3.5 Super VL 120B A12B NVFP4 | 29.0% | 69.5% | estimated ± 7.2 pp, medium confidence |
| MiniMax M2.7 | 28.0% | 68.6% | estimated ± 7.2 pp, medium confidence |
| A.X K2 | 27.8% | 68.4% | estimated ± 7.2 pp, medium confidence |
| Nemotron 3 Ultra | 26.7% | 67.4% | estimated ± 7.2 pp, medium confidence |
| Gemma 4 31B | 26.5% | 67.2% | estimated ± 7.2 pp, medium confidence |
| Ornith-1.5-35B-A3B | 25.6% | 66.3% | estimated ± 7.2 pp, medium confidence |
| Hy3 Preview | 25.5% | 66.2% | estimated ± 7.2 pp, medium confidence |
| GLM-4.7 | 24.8% | 65.6% | estimated ± 7.2 pp, medium confidence |
| Ling 3.0 Flash | 22.7% | 63.6% | estimated ± 7.2 pp, medium confidence |
| Ornith-1.5-9B | 20.2% | 61.3% | estimated ± 7.2 pp, medium confidence |
| Gemini 2.5 Pro | 18.8% | 59.9% | estimated ± 7.2 pp, medium confidence |
| K-EXAONE 2.0 | 18.3% | 59.5% | estimated ± 7.2 pp, medium confidence |
| Nemotron 3 Super 100B | 18.3% | 59.4% | estimated ± 7.2 pp, medium confidence |
| Gemma 4 26B A4B | 17.2% | 58.4% | estimated ± 7.2 pp, medium confidence |
| GPT-OSS 120B | 14.9% | 56.3% | estimated ± 7.2 pp, medium confidence |
| GPT-OSS 20B | 10.9% | 52.5% | estimated ± 7.2 pp, medium confidence |
| Nemotron 3 Nano 30B | 10.6% | 52.2% | estimated ± 7.2 pp, medium confidence |
| LFM2.5-24B-A2B | 7.4% | 49.2% | estimated ± 7.2 pp, medium confidence |
| LFM3-1.7B | 7.2% | 49.0% | estimated ± 7.2 pp, medium confidence |
Fitted on
| Model | HLE | SuperGPQA |
|---|---|---|
| Claude Opus 4.5 | 30.8% | 70.6% |
| Claude Opus 4.6 | 53.0% | 95.0% |
| Claude Sonnet 4.6 | 49.0% | 95.0% |
| GLM-5 | 30.5% | 66.8% |
| Kimi K2 | 4.7% | 57.2% |
| Kimi K2.5 | 30.1% | 69.2% |
| MiniCPM5-2B | 8.9% | 40.8% |
| Qwen3.5 397B | 28.7% | 70.4% |
| Qwen3.6-27B | 24.0% | 66.0% |
| Qwen3.6-35B-A3B | 21.4% | 64.7% |
| Qwen3.6 Plus | 28.8% | 71.6% |
| Qwen3.7 Max | 41.4% | 73.6% |
| Qwen3.7 Plus | 34.7% | 71.4% |