benchgap
Calibration

Vals GPQA Diamond → Artificial Analysis Intelligence Index

Artificial Analysis Intelligence Index is estimated from Vals GPQA Diamond with a offset logistic curve fitted on 44 models measured on both: y = 0.2021 + (0.4362 − 0.2021) / (1 + exp(−112.00·(x − 0.8960))), R² = 0.62, cross-validated error 9.0 pp. It is used for 8 estimates.

Estimated modelVals GPQA DiamondArtificial Analysis Intelligence IndexSource
GLM-5.3-Flash86.4%20.8%estimated ± 9.0 pp, medium confidence
Qwen3.5 Flash82.8%20.2%estimated ± 9.0 pp, medium confidence
MiMo-V2.581.6%20.2%estimated ± 9.0 pp, medium confidence
Gemini 3.1 Flash-Lite81.1%20.2%estimated ± 9.0 pp, medium confidence
Claude Haiku 4.572.2%20.2%estimated ± 9.0 pp, medium confidence
GLM-4.572.2%20.2%estimated ± 9.0 pp, medium confidence
Laguna M.127.0%20.2%estimated ± 9.0 pp, low confidence
Laguna XS.255.1%20.2%estimated ± 9.0 pp, low confidence

Fitted on

ModelVals GPQA DiamondArtificial Analysis Intelligence Index
Claude Fable 593.2%49.6%
Claude Fable 5.193.4%49.0%
Claude Opus 4.790.2%30.9%
Claude Opus 4.892.4%41.8%
Claude Opus 593.4%50.8%
Claude Sonnet 4.685.6%24.7%
Claude Sonnet 588.9%38.2%
DeepSeek V4 Flash 073189.9%34.3%
DeepSeek V4 Pro 081392.4%53.2%
Gemini 3.5 Flash92.7%50.2%
Gemini 3.5 Flash-Lite83.8%11.7%
Gemini 3.6 Flash93.4%34.0%
Gemini 3.7 Flash93.9%39.1%
Gemini 3.8 Flash94.4%30.6%
Gemini 3 Flash87.9%17.9%
GLM-4.674.5%14.9%
GLM-4.780.1%22.2%
GLM-5.184.5%26.1%
GLM-5.285.6%33.7%
GLM-5.388.1%31.7%
GPT-5.4 mini83.1%24.1%
GPT-5.4 nano77.5%20.7%
GPT-5.593.2%38.4%
GPT-5.6 Luna91.7%51.2%
GPT-5.6 Sol95.2%58.9%
GPT-5.6 Terra90.9%54.9%
Grok 4.391.4%37.6%
Grok 4.592.9%38.8%
Grok 4.694.7%44.3%
Inkling87.1%12.5%
Inkling-Small83.6%12.9%
Kimi K2.689.1%27.0%
Kimi K392.9%43.6%
Ling 3.0 Flash84.8%20.1%
MiMo-V2.5-Pro82.6%26.0%
MiMo-V2-Flash59.3%16.1%
MiniMax M2.786.6%22.8%
MiniMax M392.7%18.7%
Muse Spark89.6%31.3%
Muse Spark 1.191.2%33.7%
Nemotron 3 Ultra86.1%11.8%
Qwen3.6 Plus87.4%27.0%
Qwen3.7 Max90.2%29.5%
Qwen3.8-27B88.9%20.1%