Calibration
Vals GPQA Diamond → HLE-Verified
HLE-Verified is estimated from Vals GPQA Diamond with a offset logistic curve fitted on 6 models measured on both: y = 0.0000 + (0.5443 − 0.0000) / (1 + exp(−122.25·(x − 0.8867))), R² = 1.00, cross-validated error 4.6 pp. It is used for 10 estimates.
| Estimated model | Vals GPQA Diamond | HLE-Verified | Source |
|---|---|---|---|
| Qwen3.8 Max | 93.7% | 54.3% | estimated ± 4.6 pp, medium confidence |
| Grok 4.20 | 88.6% | 26.0% | estimated ± 4.6 pp, low confidence |
| Gemini 3 Flash | 87.9% | 15.3% | estimated ± 4.6 pp, low confidence |
| GLM-5.3-Flash | 86.4% | 3.2% | estimated ± 4.6 pp, low confidence |
| Nemotron 3 Ultra | 86.1% | 2.3% | estimated ± 4.6 pp, low confidence |
| Claude Sonnet 4.6 | 85.6% | 1.2% | estimated ± 4.6 pp, low confidence |
| Ling 3.0 Flash | 84.8% | 0.5% | estimated ± 4.6 pp, low confidence |
| GLM-5.1 | 84.5% | 0.3% | estimated ± 4.6 pp, low confidence |
| Gemini 3.5 Flash-Lite | 83.8% | 0.1% | estimated ± 4.6 pp, low confidence |
| Qwen3.5 Flash | 82.8% | 0.0% | estimated ± 4.6 pp, low confidence |
Fitted on
| Model | Vals GPQA Diamond | HLE-Verified |
|---|---|---|
| Claude Opus 5 | 93.4% | 54.4% |
| Claude Sonnet 5 | 88.9% | 31.0% |
| Gemini 3.7 Flash | 93.9% | 53.6% |
| Gemini 3.8 Flash | 94.4% | 54.9% |
| GPT-5.6 Sol | 95.2% | 54.5% |
| GPT-5.6 Terra | 90.9% | 51.1% |