benchgap
Calibration

AA-HLE → Artificial Analysis Intelligence Index

Artificial Analysis Intelligence Index is estimated from AA-HLE with a offset logistic curve fitted on 166 models measured on both: y = 0.0702 + (0.7333 − 0.0702) / (1 + exp(−7.39·(x − 0.4589))), R² = 0.81, cross-validated error 7.5 pp. It is used for 23 estimates.

Estimated modelAA-HLEArtificial Analysis Intelligence IndexSource
GPT-5.3 Codex42.5%36.0%estimated ± 7.5 pp, medium confidence
Ling 3.1 Flash39.4%32.4%estimated ± 7.5 pp, medium confidence
Kimi K2.7 Code35.0%27.5%estimated ± 7.5 pp, medium confidence
Hy333.5%26.0%estimated ± 7.5 pp, medium confidence
Ling 3.0 Flash VL22.0%16.7%estimated ± 7.5 pp, medium confidence
Quasar 438B18.7%14.9%estimated ± 7.5 pp, medium confidence
Trinity-Large-Thinking15.8%13.5%estimated ± 7.5 pp, medium confidence
Mercury 2.511.8%12.0%estimated ± 7.5 pp, medium confidence
Qwen3.5-4B9.9%11.4%estimated ± 7.5 pp, medium confidence
Ling 3.0 Tiny9.3%11.2%estimated ± 7.5 pp, medium confidence
Qwen3-Omni-30B-A3B-Thinking7.5%10.7%estimated ± 7.5 pp, medium confidence
LFM2.5-8B-A1B6.9%10.5%estimated ± 7.5 pp, medium confidence
Celeris-16.8%10.5%estimated ± 7.5 pp, medium confidence
LFM2.5-2.6B6.2%10.4%estimated ± 7.5 pp, medium confidence
Exaone 4.0 1.2B5.7%10.3%estimated ± 7.5 pp, medium confidence
LFM2.5-VL-1.6B-Extract5.1%10.1%estimated ± 7.5 pp, medium confidence
Exaone 4.0 32B5.0%10.1%estimated ± 7.5 pp, medium confidence
Phi-4 Multimodal Instruct5.0%10.1%estimated ± 7.5 pp, medium confidence
Llama 4 Maverick4.9%10.1%estimated ± 7.5 pp, medium confidence
Qwen3-Omni-30B-A3B-Instruct4.6%10.0%estimated ± 7.5 pp, medium confidence
Llama 3.1 405B4.0%9.9%estimated ± 7.5 pp, medium confidence
Llama 4 Scout3.8%9.8%estimated ± 7.5 pp, medium confidence
Ultravox v0.6 Llama 3.3 70B3.6%9.8%estimated ± 7.5 pp, medium confidence

Fitted on

ModelAA-HLEArtificial Analysis Intelligence Index
A.X K229.6%9.0%
Apodex 1.134.1%13.7%
Apodex 1.1 Mini34.1%13.7%
Claude 3.5 Haiku3.6%8.9%
Claude 3.7 Sonnet4.2%15.3%
Claude 3.7 Sonnet Thinking9.7%17.7%
Claude 3 Haiku4.1%5.6%
Claude 3 Opus2.8%8.7%
Claude 4 Opus6.2%16.6%
Claude 4 Opus Thinking12.3%20.6%
Claude 4 Sonnet4.3%16.6%
Claude Fable 555.5%49.6%
Claude Fable 5.159.1%49.2%
Claude Fable 5.274.3%64.6%
Claude Haiku 5.544.4%36.5%
Claude Opus 4.513.2%23.7%
Claude Opus 4.5 Thinking30.1%29.1%
Claude Opus 4.619.1%26.4%
Claude Opus 4.6 (Adaptive)39.9%32.0%
Claude Opus 4.742.3%40.7%
Claude Opus 4.848.7%41.8%
Claude Opus 554.9%50.8%
Claude Opus 5.561.4%59.1%
Claude Sonnet 4.613.3%24.7%
Claude Sonnet 541.3%38.2%
Claude Sonnet 5.555.0%54.5%
Command A+12.0%22.5%
DeepSeek-R115.8%13.1%
DeepSeek R1 Distill Qwen 32B4.6%8.4%
DeepSeek V32.9%8.5%
DeepSeek V3 03244.7%9.7%
DeepSeek V3.16.7%13.7%
DeepSeek V3.211.2%16.0%
DeepSeek V4.1 Flash 101739.2%39.5%
DeepSeek V4 Flash 073138.6%34.3%
DeepSeek V4 Pro41.0%53.2%
Devstral Small3.8%7.6%
Gemini 1.0 Pro4.2%5.3%
Gemini 1.5 Flash3.2%7.1%
Gemini 1.5 Pro4.6%7.9%
Gemini 2.0 Flash4.3%8.9%
Gemini 2.5 Flash4.7%9.9%
Gemini 2.5 Flash-Lite3.7%6.7%
Gemini 2.5 Pro22.5%16.1%
Gemini 3.5 Flash42.7%50.2%
Gemini 3.5 Flash-Lite18.8%11.7%
Gemini 3.6 Flash40.8%34.0%
Gemini 3.7 Flash47.9%39.1%
Gemini 3.8 Flash47.8%30.9%
Gemini 3 Flash15.0%17.9%
Gemini 3 Pro39.7%28.0%
Gemini 4 Argon57.1%50.4%
Gemma 3 27B4.4%4.9%
Gemma 4 31B23.6%4.4%
Gemma 4 E4B3.8%4.8%
GLM-4.5-Air7.0%11.1%
GLM-4.65.5%14.9%
GLM-4.727.4%22.2%
GLM-529.3%27.9%
GLM-5.130.1%26.1%
GLM-5.241.1%33.7%
GLM-5.342.3%31.7%
GLM-5-Turbo27.8%26.6%
GLM-5V-Turbo17.1%23.5%
GPT-4.14.2%12.7%
GPT-4.1 mini5.0%10.2%
GPT-4.1 nano3.8%7.8%
GPT-4 Turbo3.1%7.0%
GPT-4o2.4%8.4%
GPT-4o mini4.2%6.7%
GPT-528.5%23.0%
GPT-5.128.5%24.7%
GPT-5.1-Codex25.7%23.7%
GPT-5.1-Codex-Max25.7%23.7%
GPT-5.237.7%30.5%
GPT-5.2-Codex35.7%28.5%
GPT-5.443.7%39.0%
GPT-5.4 mini28.1%24.1%
GPT-5.4 nano28.3%20.7%
GPT-5.545.8%38.4%
GPT-5.6 Luna39.5%51.2%
GPT-5.6 Sol49.5%58.9%
GPT-5.6 Terra42.9%54.9%
GPT-6.1 Sol52.9%54.6%
GPT-6.1 Terra46.2%51.3%
GPT-6 Astra54.7%56.9%
GPT-6 Luna38.5%27.9%
GPT-6 Sol47.9%47.6%
GPT-OSS 120B19.6%3.8%
Granite 4.2 3B6.6%4.7%
Granite 4.2 8B9.7%5.1%
Grok 426.7%22.5%
Grok 4.1 Fast5.1%11.3%
Grok 4.1 Fast (Reasoning)19.3%20.4%
Grok 4.337.2%37.6%
Grok 4.542.7%38.8%
Grok 4.642.9%44.3%
Grok 4.743.1%35.5%
Grok 4 Fast (Reasoning)19.1%17.9%
Grok Code Fast 18.0%14.1%
Hy3 Preview33.5%41.2%
Inkling31.9%12.5%
Inkling 2 Small44.1%30.6%
Inkling-Small33.3%12.9%
K-Exaone13.9%14.4%
K-EXAONE 2.018.6%7.6%
Kimi K2.637.5%27.0%
Kimi K3.1 Pro52.5%46.5%
Kimi K27.4%12.7%
Kimi K2.530.7%23.5%
Kimi K346.9%43.6%
Ling 2.6 Flash6.3%14.1%
Ling 3.0 Flash23.7%20.1%
Magistral Medium9.8%9.0%
Magistral Small7.6%8.2%
MiMo-V2.5-Pro35.7%26.0%
MiMo-V2.6-Flash35.1%26.7%
MiMo-V2.6-Pro49.4%46.3%
MiMo-V2-Flash8.6%16.1%
MiMo-V2-Omni22.1%23.9%
MiMo-V2-Pro30.4%28.6%
MiniMax M2.729.6%22.8%
MiniMax M339.0%18.9%
MiniMax M3.1 Flash48.0%29.1%
Mistral Large 23.3%7.6%
Mistral Large 34.2%9.3%
Mistral Medium 34.1%9.0%
Mistral Medium 3.5 128B13.8%4.7%
Mistral Small 3.14.3%7.1%
Mistral Small 49.9%4.3%
Muse Glimmer 30B22.0%6.3%
Muse Spark40.7%31.3%
Muse Spark 1.146.2%33.7%
Muse Spark 1.245.5%39.6%
Muse Spark 1.348.7%48.1%
Muse Spark Open43.9%24.9%
Muse Wonder50.1%43.6%
Muse Wonder Full53.6%50.1%
Nemotron 3.5 Lightning 30B A3B NVFP410.6%5.5%
Nemotron 3.5 Super VL 120B A12B30.7%13.6%
Nemotron 3.5 Super VL 120B A12B NVFP430.7%13.6%
Nemotron 3 Ultra28.4%11.8%
North Mini Code11.1%3.4%
Nova Pro3.2%7.0%
o17.0%15.2%
o1-mini3.6%9.8%
o3-mini7.9%12.5%
o4-mini16.5%16.7%
Qwen2.5 Coder 32B Instruct3.5%6.7%
Qwen3.5-27B23.9%22.9%
Qwen3.5-35B-A3B21.0%19.3%
Qwen3.6-27B23.1%21.4%
Qwen 3.6 Max (preview)30.8%28.4%
Qwen3.6 Plus27.8%27.0%
Qwen3.7 Max40.5%29.5%
Qwen3.8-27B33.9%20.1%
Qwen3.8-Flash-Next38.0%28.6%
Qwen3.8 Max Preview43.1%33.7%
Qwen3 Max11.9%15.6%
QwQ-32B7.4%9.5%
Beam 1.0 501B-A23B34.7%17.7%
Solar Pro 23.7%7.0%
Solar Pro 310.3%7.8%
Solar Pro 429.8%14.2%
Step 3.7 Flash21.4%19.5%
Step 5 Preview46.5%28.5%