benchgap
Calibration

Artificial Analysis Intelligence Index → HealthBench Hard

HealthBench Hard is estimated from Artificial Analysis Intelligence Index with a Hill curve fitted on 11 models measured on both: y = 0.0000 + (0.3539 − 0.0000)·x^6.00 / (0.24880^6.00 + x^6.00), R² = 0.43, cross-validated error 7.7 pp. It is used for 132 estimates.

Estimated modelArtificial Analysis Intelligence IndexHealthBench HardSource
A.X K221.5%10.3%estimated ± 7.7 pp, low confidence
Apodex 1.126.4%20.8%estimated ± 7.7 pp, low confidence
Apodex 1.1 Mini26.4%20.8%estimated ± 7.7 pp, low confidence
Celeris-16.4%0.0%estimated ± 7.7 pp, low confidence
Claude 3 Haiku5.6%0.0%estimated ± 7.7 pp, low confidence
Claude 3 Opus8.7%0.1%estimated ± 7.7 pp, low confidence
Claude 4.1 Opus18.6%5.2%estimated ± 7.7 pp, low confidence
Claude 4.1 Opus Thinking22.9%13.3%estimated ± 7.7 pp, low confidence
Claude 4 Sonnet16.6%2.9%estimated ± 7.7 pp, low confidence
Claude Fable 549.6%34.8%estimated ± 7.7 pp, low confidence
Claude Fable 5.153.3%35.0%estimated ± 7.7 pp, low confidence
Claude Haiku 5.543.4%34.2%estimated ± 7.7 pp, low confidence
Claude Opus 4.5 Thinking29.1%25.5%estimated ± 7.7 pp, low confidence
Claude Opus 4.6 (Adaptive)32.0%28.9%estimated ± 7.7 pp, low confidence
Claude Opus 4.730.9%27.8%estimated ± 7.7 pp, low confidence
Claude Opus 550.8%34.9%estimated ± 7.7 pp, low confidence
Claude Opus 5.557.6%35.2%estimated ± 7.7 pp, low confidence
Claude Sonnet 538.2%32.9%estimated ± 7.7 pp, low confidence
Claude Sonnet 5.556.0%35.1%estimated ± 7.7 pp, low confidence
Command A+22.5%12.5%estimated ± 7.7 pp, low confidence
DeepSeek-R113.1%0.7%estimated ± 7.7 pp, low confidence
DeepSeek R1 Distill Qwen 32B8.4%0.1%estimated ± 7.7 pp, low confidence
DeepSeek V3 03249.7%0.1%estimated ± 7.7 pp, low confidence
DeepSeek V3.113.7%1.0%estimated ± 7.7 pp, low confidence
DeepSeek V3.1 (Reasoning)13.5%0.9%estimated ± 7.7 pp, low confidence
DeepSeek V3.216.0%2.4%estimated ± 7.7 pp, low confidence
Exaone 4.0 1.2B5.2%0.0%estimated ± 7.7 pp, low confidence
Exaone 4.0 32B6.3%0.0%estimated ± 7.7 pp, low confidence
Gemini 1.0 Pro5.3%0.0%estimated ± 7.7 pp, low confidence
Gemini 1.5 Pro7.9%0.0%estimated ± 7.7 pp, low confidence
Gemini 2.5 Flash9.9%0.1%estimated ± 7.7 pp, low confidence
Gemini 3.5 Flash-Lite22.2%11.8%estimated ± 7.7 pp, low confidence
Gemini 3.6 Flash34.0%30.7%estimated ± 7.7 pp, low confidence
Gemini 3.7 Flash39.1%33.2%estimated ± 7.7 pp, low confidence
Gemini 3.8 Flash40.9%33.7%estimated ± 7.7 pp, low confidence
Gemini 3 Flash17.9%4.3%estimated ± 7.7 pp, low confidence
Gemini 3 Pro28.0%23.7%estimated ± 7.7 pp, low confidence
Gemini 4 Argon52.6%35.0%estimated ± 7.7 pp, low confidence
Gemma 3 27B4.9%0.0%estimated ± 7.7 pp, low confidence
Gemma 4 26B A4B16.7%2.9%estimated ± 7.7 pp, low confidence
GLM-4.5-Air11.1%0.3%estimated ± 7.7 pp, low confidence
GLM-4.614.9%1.6%estimated ± 7.7 pp, low confidence
GLM-5.126.1%20.1%estimated ± 7.7 pp, low confidence
GLM-5.344.8%34.4%estimated ± 7.7 pp, low confidence
GLM-5.3-Flash41.8%33.9%estimated ± 7.7 pp, low confidence
GLM-5-Turbo26.6%21.2%estimated ± 7.7 pp, low confidence
GLM-5V-Turbo23.5%14.7%estimated ± 7.7 pp, low confidence
GPT-4 Turbo7.0%0.0%estimated ± 7.7 pp, low confidence
GPT-4o8.4%0.1%estimated ± 7.7 pp, low confidence
GPT-4o mini6.7%0.0%estimated ± 7.7 pp, low confidence
GPT-5.124.7%17.4%estimated ± 7.7 pp, low confidence
GPT-5.1-Codex23.7%15.1%estimated ± 7.7 pp, low confidence
GPT-5.1-Codex-Max23.7%15.1%estimated ± 7.7 pp, low confidence
GPT-5.2-Codex28.5%24.5%estimated ± 7.7 pp, low confidence
GPT-5.3 Codex32.5%29.5%estimated ± 7.7 pp, low confidence
GPT-5 (high)23.0%13.6%estimated ± 7.7 pp, low confidence
GPT-5 (medium)22.9%13.3%estimated ± 7.7 pp, low confidence
GPT-OSS 120B11.6%0.4%estimated ± 7.7 pp, low confidence
GPT-OSS 20B9.0%0.1%estimated ± 7.7 pp, low confidence
Granite-4.0-350M4.8%0.0%estimated ± 7.7 pp, low confidence
Granite-4.0-H-1B5.2%0.0%estimated ± 7.7 pp, low confidence
Granite-4.0-H-350M4.8%0.0%estimated ± 7.7 pp, low confidence
Grok 422.5%12.5%estimated ± 7.7 pp, low confidence
Grok 4.1 Fast11.3%0.3%estimated ± 7.7 pp, low confidence
Grok 4.1 Fast (Reasoning)20.4%8.2%estimated ± 7.7 pp, low confidence
Grok 4.538.8%33.1%estimated ± 7.7 pp, low confidence
Grok 4.644.3%34.3%estimated ± 7.7 pp, low confidence
Grok 4.746.5%34.6%estimated ± 7.7 pp, low confidence
Grok 4 Fast (Reasoning)17.9%4.4%estimated ± 7.7 pp, low confidence
Grok Code Fast 114.1%1.1%estimated ± 7.7 pp, low confidence
Hy325.3%18.6%estimated ± 7.7 pp, low confidence
K-Exaone14.4%1.3%estimated ± 7.7 pp, low confidence
K-EXAONE 2.019.7%7.0%estimated ± 7.7 pp, low confidence
Kimi K212.7%0.6%estimated ± 7.7 pp, low confidence
Kimi K2.7 Code25.8%19.6%estimated ± 7.7 pp, low confidence
LFM2.5-2.6B8.4%0.1%estimated ± 7.7 pp, low confidence
LFM2.5-8B-A1B7.2%0.0%estimated ± 7.7 pp, low confidence
LFM2.5-VL-1.6B-Extract4.8%0.0%estimated ± 7.7 pp, low confidence
Ling 3.0 Flash VL24.6%17.0%estimated ± 7.7 pp, low confidence
Ling 3.0 Tiny11.1%0.3%estimated ± 7.7 pp, low confidence
Ling 3.1 Flash41.1%33.7%estimated ± 7.7 pp, low confidence
Llama 3.1 405B7.3%0.0%estimated ± 7.7 pp, low confidence
Llama 4 Maverick10.0%0.1%estimated ± 7.7 pp, low confidence
Llama 4 Scout8.1%0.0%estimated ± 7.7 pp, low confidence
Mercury 2.512.3%0.5%estimated ± 7.7 pp, low confidence
MiMo-V2.5-Pro26.0%20.0%estimated ± 7.7 pp, low confidence
MiMo-V2.6-Flash37.9%32.8%estimated ± 7.7 pp, low confidence
MiMo-V2.6-Pro46.3%34.6%estimated ± 7.7 pp, low confidence
MiMo-V2-Omni23.9%15.6%estimated ± 7.7 pp, low confidence
MiMo-V2-Pro28.6%24.8%estimated ± 7.7 pp, low confidence
MiniCPM5-2B12.5%0.5%estimated ± 7.7 pp, low confidence
MiniMax M2.722.8%13.1%estimated ± 7.7 pp, low confidence
MiniMax M329.2%25.6%estimated ± 7.7 pp, low confidence
Mistral Large 27.6%0.0%estimated ± 7.7 pp, low confidence
Mistral Large 39.3%0.1%estimated ± 7.7 pp, low confidence
Mistral Large 438.4%32.9%estimated ± 7.7 pp, low confidence
Mistral Medium 39.0%0.1%estimated ± 7.7 pp, low confidence
Mistral Medium 3.5 128B14.2%1.2%estimated ± 7.7 pp, low confidence
Mistral Small 411.3%0.3%estimated ± 7.7 pp, low confidence
Mistral Small 4 (Reasoning)11.3%0.3%estimated ± 7.7 pp, low confidence
Muse Glimmer 30B17.5%3.8%estimated ± 7.7 pp, low confidence
Muse Spark 1.133.7%30.5%estimated ± 7.7 pp, low confidence
Muse Spark 1.239.6%33.3%estimated ± 7.7 pp, low confidence
Muse Spark 1.348.1%34.7%estimated ± 7.7 pp, low confidence
Nemotron 3 Nano 30B8.9%0.1%estimated ± 7.7 pp, low confidence
Nemotron 3 Super 100B12.8%0.7%estimated ± 7.7 pp, low confidence
Nemotron Ultra 253B7.5%0.0%estimated ± 7.7 pp, low confidence
North Mini Code9.9%0.1%estimated ± 7.7 pp, low confidence
Nova Pro7.0%0.0%estimated ± 7.7 pp, low confidence
o1-preview11.4%0.3%estimated ± 7.7 pp, low confidence
o320.2%7.9%estimated ± 7.7 pp, low confidence
o3-pro21.9%11.2%estimated ± 7.7 pp, low confidence
Phi-45.9%0.0%estimated ± 7.7 pp, low confidence
Phi-4 Multimodal Instruct5.8%0.0%estimated ± 7.7 pp, low confidence
Quasar 438B26.7%21.5%estimated ± 7.7 pp, low confidence
Qwen2.5 Coder 32B Instruct6.7%0.0%estimated ± 7.7 pp, low confidence
Qwen3.5 397B (Reasoning)21.4%10.3%estimated ± 7.7 pp, low confidence
Qwen 3.6 Max (preview)28.4%24.3%estimated ± 7.7 pp, low confidence
Qwen3.8 Max Preview45.4%34.5%estimated ± 7.7 pp, low confidence
Qwen3 Max15.6%2.0%estimated ± 7.7 pp, low confidence
Qwen3-Omni-30B-A3B-Instruct6.0%0.0%estimated ± 7.7 pp, low confidence
Qwen3-Omni-30B-A3B-Thinking7.8%0.0%estimated ± 7.7 pp, low confidence
Sarvam 105B8.8%0.1%estimated ± 7.7 pp, low confidence
Sarvam 30B6.6%0.0%estimated ± 7.7 pp, low confidence
Solar Pro 27.0%0.0%estimated ± 7.7 pp, low confidence
Solar Pro 37.8%0.0%estimated ± 7.7 pp, low confidence
Solar Pro 428.2%24.0%estimated ± 7.7 pp, low confidence
Step 3.7 Flash19.5%6.6%estimated ± 7.7 pp, low confidence
Step 5 Preview43.7%34.2%estimated ± 7.7 pp, low confidence
Trinity-Large-Preview10.8%0.2%estimated ± 7.7 pp, low confidence
Trinity-Large-Thinking10.8%0.2%estimated ± 7.7 pp, low confidence
Ultravox v0.6 Llama 3.3 70B7.7%0.0%estimated ± 7.7 pp, low confidence