Calibration
GPQA → HLE
HLE is estimated from GPQA with a inverse Michaelis–Menten curve fitted on 42 models measured on both: y = 0.05839·(x − 0.0000) / (0.0000 + 1.0375 − x), R² = 0.58, cross-validated error 8.2 pp. It is used for 25 estimates.
| Estimated model | GPQA | HLE | Source |
|---|---|---|---|
| Claude 3.5 Sonnet | 59.4% | 7.8% | estimated ± 8.2 pp, low confidence |
| DeepSeek V3 | 59.1% | 7.7% | estimated ± 8.2 pp, low confidence |
| Gemma 4 E2B | 43.4% | 4.2% | estimated ± 8.2 pp, low confidence |
| Gemma 4 E4B | 58.6% | 7.6% | estimated ± 8.2 pp, low confidence |
| GPT-4.1 | 66.3% | 10.3% | estimated ± 8.2 pp, low confidence |
| GPT-4.1 mini | 64.2% | 9.5% | estimated ± 8.2 pp, low confidence |
| GPT-4.1 nano | 50.3% | 5.5% | estimated ± 8.2 pp, low confidence |
| Granite 4.2 30B | 66.4% | 10.4% | estimated ± 8.2 pp, low confidence |
| Granite 4.2 3B | 54.8% | 6.5% | estimated ± 8.2 pp, low confidence |
| Granite 4.2 8B | 64.1% | 9.5% | estimated ± 8.2 pp, low confidence |
| Interfaze Beta | 89.9% | 37.9% | estimated ± 8.2 pp, medium confidence |
| Kimi K2.5 (Reasoning) | 87.6% | 31.7% | estimated ± 8.2 pp, medium confidence |
| LFM2.5-230M | 25.4% | 1.9% | estimated ± 8.2 pp, low confidence |
| LFM2.5-VL-450M | 25.7% | 1.9% | estimated ± 8.2 pp, low confidence |
| Ling 2.6 Flash | 59.0% | 7.7% | estimated ± 8.2 pp, low confidence |
| Ling 3.0 Flash FP8 | 84.0% | 24.8% | estimated ± 8.2 pp, medium confidence |
| MAI-Thinking-1 | 84.2% | 25.1% | estimated ± 8.2 pp, medium confidence |
| MiMo-V2-Flash | 83.7% | 24.4% | estimated ± 8.2 pp, medium confidence |
| Nemotron 3 Nano Omni 30B A3B | 72.2% | 13.4% | estimated ± 8.2 pp, low confidence |
| o1 | 75.7% | 15.8% | estimated ± 8.2 pp, low confidence |
| o1-pro | 79.0% | 18.6% | estimated ± 8.2 pp, low confidence |
| o3-mini | 77.2% | 17.0% | estimated ± 8.2 pp, low confidence |
| Soofi S 30B-A3B | 43.4% | 4.2% | estimated ± 8.2 pp, low confidence |
| ZAYA1-74B-Preview | 57.3% | 7.2% | estimated ± 8.2 pp, low confidence |
| ZAYA1-8B | 71.0% | 12.7% | estimated ± 8.2 pp, low confidence |