Calibration
GPQA Diamond → HLE w/o tools
HLE w/o tools is estimated from GPQA Diamond with a offset logistic curve fitted on 26 models measured on both: y = 0.0574 + (0.5331 − 0.0574) / (1 + exp(−27.36·(x − 0.8799))), R² = 0.90, cross-validated error 4.4 pp. It is used for 29 estimates.
| Estimated model | GPQA Diamond | HLE w/o tools | Source |
|---|---|---|---|
| A.X K2 | 85.6% | 22.0% | estimated ± 4.4 pp, high confidence |
| GLM-5.1 | 86.2% | 23.8% | estimated ± 4.4 pp, high confidence |
| GPT-5.6 Luna | 92.3% | 42.1% | estimated ± 4.4 pp, high confidence |
| GPT-5.6 Sol | 94.6% | 46.6% | estimated ± 4.4 pp, high confidence |
| GPT-5.6 Terra | 92.9% | 43.5% | estimated ± 4.4 pp, high confidence |
| GPT-6 Astra | 96.0% | 48.5% | estimated ± 4.4 pp, medium confidence |
| Interfaze Beta | 89.9% | 35.6% | estimated ± 4.4 pp, high confidence |
| K-EXAONE 2.0 | 82.2% | 13.8% | estimated ± 4.4 pp, high confidence |
| LFM2.5-230M | 25.4% | 5.7% | estimated ± 4.4 pp, medium confidence |
| Ling 3.0 Flash FP8 | 84.0% | 17.7% | estimated ± 4.4 pp, high confidence |
| LLaDA2.2-mini | 44.4% | 5.7% | estimated ± 4.4 pp, medium confidence |
| LongCat-Flash-Lite-Sparse | 69.5% | 6.0% | estimated ± 4.4 pp, medium confidence |
| MAI-Thinking-1 | 84.2% | 18.2% | estimated ± 4.4 pp, high confidence |
| Mellum2-12B-A2.5B-Instruct | 40.9% | 5.7% | estimated ± 4.4 pp, medium confidence |
| Mellum2-12B-A2.5B-Thinking | 57.6% | 5.8% | estimated ± 4.4 pp, medium confidence |
| Mercury 2.5 | 79.0% | 9.5% | estimated ± 4.4 pp, high confidence |
| MiniCPM5-1B | 26.3% | 5.7% | estimated ± 4.4 pp, medium confidence |
| MiniCPM5-2B | 70.2% | 6.1% | estimated ± 4.4 pp, medium confidence |
| MiniMax M2.7 | 87.0% | 26.3% | estimated ± 4.4 pp, high confidence |
| Nemotron 3 Nano Omni 30B A3B | 72.2% | 6.4% | estimated ± 4.4 pp, medium confidence |
| Pareto 26.10 Preview | 92.4% | 42.4% | estimated ± 4.4 pp, high confidence |
| Solar Pro 4 | 89.0% | 32.8% | estimated ± 4.4 pp, high confidence |
| Soofi S 30B-A3B | 43.4% | 5.7% | estimated ± 4.4 pp, medium confidence |
| Step 5 Preview | 93.5% | 44.7% | estimated ± 4.4 pp, high confidence |
| Ternary Bonsai 2 27B | 85.8% | 22.5% | estimated ± 4.4 pp, high confidence |
| Trinity-Large-Preview | 63.3% | 5.8% | estimated ± 4.4 pp, medium confidence |
| Trinity-Large-Thinking | 76.3% | 7.6% | estimated ± 4.4 pp, high confidence |
| ZAYA1-74B-Preview | 57.3% | 5.7% | estimated ± 4.4 pp, medium confidence |
| ZAYA1-8B | 71.0% | 6.2% | estimated ± 4.4 pp, medium confidence |