benchgap
Calibration

Vals GPQA Diamond → AA-HLE

AA-HLE is estimated from Vals GPQA Diamond with a offset logistic curve fitted on 46 models measured on both: y = 0.1086 + (0.9376 − 0.1086) / (1 + exp(−11.91·(x − 0.9589))), R² = 0.70, cross-validated error 6.8 pp. It is used for 8 estimates.

Estimated modelVals GPQA DiamondAA-HLESource
GLM-5.3-Flash86.4%31.1%estimated ± 6.8 pp, medium confidence
Qwen3.5 Flash82.8%25.3%estimated ± 6.8 pp, medium confidence
MiMo-V2.581.6%23.6%estimated ± 6.8 pp, medium confidence
Gemini 3.1 Flash-Lite81.1%23.0%estimated ± 6.8 pp, medium confidence
Claude Haiku 4.572.2%15.5%estimated ± 6.8 pp, medium confidence
GLM-4.572.2%15.5%estimated ± 6.8 pp, medium confidence
Laguna XS.255.1%11.5%estimated ± 6.8 pp, medium confidence
Laguna M.127.0%10.9%estimated ± 6.8 pp, low confidence

Fitted on

ModelVals GPQA DiamondAA-HLE
Claude Fable 593.2%55.5%
Claude Fable 5.193.4%59.1%
Claude Opus 4.790.2%33.3%
Claude Opus 4.892.4%48.7%
Claude Opus 593.4%54.9%
Claude Sonnet 4.685.6%13.3%
Claude Sonnet 588.9%41.3%
DeepSeek V4 Flash 073189.9%38.6%
DeepSeek V4 Pro 081392.4%41.0%
Gemini 3.1 Pro95.5%47.0%
Gemini 3.5 Flash92.7%42.7%
Gemini 3.5 Flash-Lite83.8%18.8%
Gemini 3.6 Flash93.4%40.8%
Gemini 3.7 Flash93.9%47.9%
Gemini 3.8 Flash94.4%47.8%
Gemini 3 Flash87.9%15.0%
GLM-4.674.5%5.5%
GLM-4.780.1%27.4%
GLM-5.184.5%30.1%
GLM-5.285.6%41.1%
GLM-5.388.1%42.3%
GPT-5.4 mini83.1%28.1%
GPT-5.4 nano77.5%28.3%
GPT-5.593.2%45.8%
GPT-5.6 Luna91.7%39.5%
GPT-5.6 Sol95.2%49.5%
GPT-5.6 Terra90.9%42.9%
Grok 4.391.4%37.2%
Grok 4.592.9%42.7%
Grok 4.694.7%42.9%
Inkling87.1%31.9%
Inkling-Small83.6%33.3%
Kimi K2.689.1%37.5%
Kimi K392.9%46.9%
Ling 3.0 Flash84.8%23.7%
MiMo-V2.5-Pro82.6%35.7%
MiMo-V2-Flash59.3%8.6%
MiniMax M2.786.6%29.6%
MiniMax M392.7%39.0%
Mistral Medium 3.5 128B34.8%13.8%
Muse Spark89.6%40.7%
Muse Spark 1.191.2%46.2%
Nemotron 3 Ultra86.1%28.4%
Qwen3.6 Plus87.4%27.8%
Qwen3.7 Max90.2%40.5%
Qwen3.8-27B88.9%33.9%