benchgap
Calibration

HLE → SuperGPQA

SuperGPQA is estimated from HLE with a linear curve fitted on 13 models measured on both: y = 0.9393·x + 0.4229, R² = 0.85, cross-validated error 7.2 pp. It is used for 63 estimates.

Estimated modelHLESuperGPQASource
Claude Fable 5.165.0%100.0%estimated ± 7.2 pp, low confidence
Claude Fable 5.274.2%100.0%estimated ± 7.2 pp, low confidence
Claude Mythos 564.5%100.0%estimated ± 7.2 pp, low confidence
Claude Opus 564.7%100.0%estimated ± 7.2 pp, low confidence
Kimi K3.1 Pro70.0%100.0%estimated ± 7.2 pp, low confidence
Muse Spark 1.162.1%100.0%estimated ± 7.2 pp, low confidence
Hy459.1%97.8%estimated ± 7.2 pp, low confidence
GPT-5.4 Pro58.7%97.4%estimated ± 7.2 pp, low confidence
Claude Opus 4.857.9%96.7%estimated ± 7.2 pp, low confidence
Claude Sonnet 557.4%96.2%estimated ± 7.2 pp, low confidence
GPT-5.5 Pro57.2%96.0%estimated ± 7.2 pp, low confidence
Apodex 1.156.1%95.0%estimated ± 7.2 pp, low confidence
Kimi K356.0%94.9%estimated ± 7.2 pp, low confidence
Hy4 preview55.4%94.3%estimated ± 7.2 pp, low confidence
Hy4 flash55.2%94.1%estimated ± 7.2 pp, low confidence
Claude Opus 4.754.7%93.7%estimated ± 7.2 pp, low confidence
Inkling 2 Small53.2%92.3%estimated ± 7.2 pp, low confidence
dots3-note Preview52.6%91.7%estimated ± 7.2 pp, medium confidence
GPT-5.552.2%91.3%estimated ± 7.2 pp, medium confidence
GPT-5.452.1%91.2%estimated ± 7.2 pp, medium confidence
Qwen4 Max51.7%90.8%estimated ± 7.2 pp, medium confidence
Muse Spark50.4%89.6%estimated ± 7.2 pp, medium confidence
MiMo-V2.5-Pro48.0%87.4%estimated ± 7.2 pp, medium confidence
Inkling-Small47.8%87.2%estimated ± 7.2 pp, medium confidence
Agents-A147.6%87.0%estimated ± 7.2 pp, medium confidence
Step 5 Preview46.5%86.0%estimated ± 7.2 pp, medium confidence
Inkling46.0%85.5%estimated ± 7.2 pp, medium confidence
Ornith-1.5-397B44.6%84.2%estimated ± 7.2 pp, medium confidence
Qwen3.8 Max43.6%83.2%estimated ± 7.2 pp, medium confidence
DeepSeek V4 Pro42.7%82.4%estimated ± 7.2 pp, medium confidence
DeepSeek V4.1 Flash 101742.6%82.3%estimated ± 7.2 pp, medium confidence
GPT-5.4 mini41.5%81.3%estimated ± 7.2 pp, medium confidence
GLM-5.240.5%80.3%estimated ± 7.2 pp, medium confidence
Gemini 3.5 Flash40.2%80.0%estimated ± 7.2 pp, medium confidence
GPT-5.4 nano37.7%77.7%estimated ± 7.2 pp, medium confidence
Ling 3.1 Flash37.4%77.5%estimated ± 7.2 pp, medium confidence
Qwen3.8-Omni-Flash36.5%76.6%estimated ± 7.2 pp, medium confidence
Qwen3.8-Flash-Next35.9%76.0%estimated ± 7.2 pp, medium confidence
Grok 4.335.0%75.2%estimated ± 7.2 pp, medium confidence
DeepSeek V4 Flash 073134.8%75.0%estimated ± 7.2 pp, medium confidence
Kimi K2.634.7%74.9%estimated ± 7.2 pp, medium confidence
GLM-5.131.0%71.4%estimated ± 7.2 pp, medium confidence
Qwen3.8-27B30.8%71.2%estimated ± 7.2 pp, medium confidence
Nemotron 3.5 Super VL 120B A12B29.2%69.7%estimated ± 7.2 pp, medium confidence
Nemotron 3.5 Super VL 120B A12B NVFP429.0%69.5%estimated ± 7.2 pp, medium confidence
MiniMax M2.728.0%68.6%estimated ± 7.2 pp, medium confidence
A.X K227.8%68.4%estimated ± 7.2 pp, medium confidence
Nemotron 3 Ultra26.7%67.4%estimated ± 7.2 pp, medium confidence
Gemma 4 31B26.5%67.2%estimated ± 7.2 pp, medium confidence
Ornith-1.5-35B-A3B25.6%66.3%estimated ± 7.2 pp, medium confidence
Hy3 Preview25.5%66.2%estimated ± 7.2 pp, medium confidence
GLM-4.724.8%65.6%estimated ± 7.2 pp, medium confidence
Ling 3.0 Flash22.7%63.6%estimated ± 7.2 pp, medium confidence
Ornith-1.5-9B20.2%61.3%estimated ± 7.2 pp, medium confidence
Gemini 2.5 Pro18.8%59.9%estimated ± 7.2 pp, medium confidence
K-EXAONE 2.018.3%59.5%estimated ± 7.2 pp, medium confidence
Nemotron 3 Super 100B18.3%59.4%estimated ± 7.2 pp, medium confidence
Gemma 4 26B A4B17.2%58.4%estimated ± 7.2 pp, medium confidence
GPT-OSS 120B14.9%56.3%estimated ± 7.2 pp, medium confidence
GPT-OSS 20B10.9%52.5%estimated ± 7.2 pp, medium confidence
Nemotron 3 Nano 30B10.6%52.2%estimated ± 7.2 pp, medium confidence
LFM2.5-24B-A2B7.4%49.2%estimated ± 7.2 pp, medium confidence
LFM3-1.7B7.2%49.0%estimated ± 7.2 pp, medium confidence

Fitted on

ModelHLESuperGPQA
Claude Opus 4.530.8%70.6%
Claude Opus 4.653.0%95.0%
Claude Sonnet 4.649.0%95.0%
GLM-530.5%66.8%
Kimi K24.7%57.2%
Kimi K2.530.1%69.2%
MiniCPM5-2B8.9%40.8%
Qwen3.5 397B28.7%70.4%
Qwen3.6-27B24.0%66.0%
Qwen3.6-35B-A3B21.4%64.7%
Qwen3.6 Plus28.8%71.6%
Qwen3.7 Max41.4%73.6%
Qwen3.7 Plus34.7%71.4%