benchgap
Calibration

Artificial Analysis Intelligence Index → C-Eval

C-Eval is estimated from Artificial Analysis Intelligence Index with a linear curve fitted on 5 models measured on both: y = 0.2988·x + 0.8558, R² = 0.86, cross-validated error 1.0 pp. It is used for 130 estimates.

Estimated modelArtificial Analysis Intelligence IndexC-EvalSource
Grok 4.644.3%98.8%estimated ± 1.0 pp, low confidence
Kimi K343.6%98.6%estimated ± 1.0 pp, low confidence
Muse Wonder43.6%98.6%estimated ± 1.0 pp, low confidence
Qwen4 Max42.9%98.4%estimated ± 1.0 pp, low confidence
Claude Opus 4.841.8%98.1%estimated ± 1.0 pp, low confidence
Claude Opus 4.740.7%97.7%estimated ± 1.0 pp, low confidence
Muse Spark 1.239.6%97.4%estimated ± 1.0 pp, low confidence
DeepSeek V4.1 Flash 101739.5%97.4%estimated ± 1.0 pp, low confidence
Gemini 3.7 Flash39.1%97.3%estimated ± 1.0 pp, low confidence
GPT-5.439.0%97.2%estimated ± 1.0 pp, low confidence
Grok 4.538.8%97.2%estimated ± 1.0 pp, low confidence
GPT-5.538.4%97.0%estimated ± 1.0 pp, low confidence
Claude Sonnet 538.2%97.0%estimated ± 1.0 pp, low confidence
Claude Haiku 5.536.5%96.5%estimated ± 1.0 pp, low confidence
Muse Ember36.0%96.3%estimated ± 1.0 pp, low confidence
Grok 4.735.5%96.2%estimated ± 1.0 pp, low confidence
Gemini 3.6 Flash34.0%95.7%estimated ± 1.0 pp, low confidence
Muse Spark 1.133.7%95.7%estimated ± 1.0 pp, low confidence
GLM-5.233.7%95.7%estimated ± 1.0 pp, low confidence
Qwen3.8 Max Preview33.7%95.6%estimated ± 1.0 pp, low confidence
Claude Opus 4.6 (Adaptive)32.0%95.1%estimated ± 1.0 pp, low confidence
GLM-5.331.7%95.0%estimated ± 1.0 pp, low confidence
Muse Spark31.3%94.9%estimated ± 1.0 pp, low confidence
Gemini 3.8 Flash30.9%94.8%estimated ± 1.0 pp, low confidence
Inkling 2 Small30.6%94.7%estimated ± 1.0 pp, low confidence
GPT-5.230.5%94.7%estimated ± 1.0 pp, low confidence
Qwen3.7 Max29.5%94.4%estimated ± 1.0 pp, low confidence
Claude Opus 4.5 Thinking29.1%94.3%estimated ± 1.0 pp, low confidence
MiniMax M3.1 Flash29.1%94.3%estimated ± 1.0 pp, low confidence
MiMo-V2-Pro28.6%94.1%estimated ± 1.0 pp, low confidence
Step 5 Preview28.5%94.1%estimated ± 1.0 pp, low confidence
GPT-5.2-Codex28.5%94.1%estimated ± 1.0 pp, low confidence
Qwen 3.6 Max (preview)28.4%94.1%estimated ± 1.0 pp, low confidence
Gemini 3 Pro28.0%93.9%estimated ± 1.0 pp, low confidence
GPT-6 Luna27.9%93.9%estimated ± 1.0 pp, low confidence
MiMo-V2.6-Flash26.7%93.6%estimated ± 1.0 pp, medium confidence
GLM-5-Turbo26.6%93.5%estimated ± 1.0 pp, medium confidence
Claude Opus 4.626.4%93.5%estimated ± 1.0 pp, medium confidence
Mistral Large 426.1%93.4%estimated ± 1.0 pp, medium confidence
MiMo-V2.5-Pro26.0%93.3%estimated ± 1.0 pp, medium confidence
Muse Spark Open24.9%93.0%estimated ± 1.0 pp, medium confidence
GPT-5.124.7%93.0%estimated ± 1.0 pp, medium confidence
Claude Sonnet 4.624.7%93.0%estimated ± 1.0 pp, medium confidence
GPT-5.4 mini24.1%92.8%estimated ± 1.0 pp, medium confidence
MiMo-V2-Omni23.9%92.7%estimated ± 1.0 pp, medium confidence
GPT-5.1-Codex23.7%92.7%estimated ± 1.0 pp, medium confidence
GPT-5.1-Codex-Max23.7%92.7%estimated ± 1.0 pp, medium confidence
GLM-5V-Turbo23.5%92.6%estimated ± 1.0 pp, medium confidence
GPT-523.0%92.4%estimated ± 1.0 pp, medium confidence
Qwen3.5-27B22.9%92.4%estimated ± 1.0 pp, medium confidence
Command A+22.5%92.3%estimated ± 1.0 pp, medium confidence
Grok 422.5%92.3%estimated ± 1.0 pp, medium confidence
o3-pro21.9%92.1%estimated ± 1.0 pp, medium confidence
Claude 4 Opus Thinking20.6%91.7%estimated ± 1.0 pp, medium confidence
Grok 4.1 Fast (Reasoning)20.4%91.7%estimated ± 1.0 pp, medium confidence
Step 3.7 Flash19.5%91.4%estimated ± 1.0 pp, medium confidence
Qwen3.5-35B-A3B19.3%91.4%estimated ± 1.0 pp, medium confidence
MiniMax M318.9%91.2%estimated ± 1.0 pp, medium confidence
Claude 4.1 Opus18.6%91.1%estimated ± 1.0 pp, medium confidence
Grok 4 Fast (Reasoning)17.9%90.9%estimated ± 1.0 pp, medium confidence
Gemini 3 Flash17.9%90.9%estimated ± 1.0 pp, medium confidence
Beam 1.0 501B-A23B17.7%90.9%estimated ± 1.0 pp, medium confidence
Claude 3.7 Sonnet Thinking17.7%90.9%estimated ± 1.0 pp, medium confidence
o4-mini16.7%90.6%estimated ± 1.0 pp, medium confidence
Claude 4 Opus16.6%90.5%estimated ± 1.0 pp, medium confidence
Claude 4 Sonnet16.6%90.5%estimated ± 1.0 pp, medium confidence
MiMo-V2-Flash16.1%90.4%estimated ± 1.0 pp, medium confidence
DeepSeek V3.216.0%90.4%estimated ± 1.0 pp, medium confidence
Qwen3 Max15.6%90.2%estimated ± 1.0 pp, medium confidence
Claude 3.7 Sonnet15.3%90.1%estimated ± 1.0 pp, medium confidence
o115.2%90.1%estimated ± 1.0 pp, medium confidence
GLM-4.614.9%90.0%estimated ± 1.0 pp, medium confidence
K-Exaone14.4%89.9%estimated ± 1.0 pp, medium confidence
Solar Pro 414.2%89.8%estimated ± 1.0 pp, medium confidence
Grok Code Fast 114.1%89.8%estimated ± 1.0 pp, medium confidence
Ling 2.6 Flash14.1%89.8%estimated ± 1.0 pp, medium confidence
DeepSeek V3.113.7%89.7%estimated ± 1.0 pp, medium confidence
Apodex 1.113.7%89.7%estimated ± 1.0 pp, medium confidence
Apodex 1.1 Mini13.7%89.7%estimated ± 1.0 pp, medium confidence
DeepSeek-R113.1%89.5%estimated ± 1.0 pp, medium confidence
Inkling-Small12.9%89.4%estimated ± 1.0 pp, medium confidence
Kimi K212.7%89.4%estimated ± 1.0 pp, medium confidence
GPT-4.112.7%89.4%estimated ± 1.0 pp, medium confidence
Inkling12.5%89.3%estimated ± 1.0 pp, medium confidence
o3-mini12.5%89.3%estimated ± 1.0 pp, medium confidence
o1-pro12.4%89.3%estimated ± 1.0 pp, medium confidence
Gemini 3.5 Flash-Lite11.7%89.1%estimated ± 1.0 pp, medium confidence
o1-preview11.4%89.0%estimated ± 1.0 pp, medium confidence
Grok 4.1 Fast11.3%89.0%estimated ± 1.0 pp, medium confidence
GLM-4.5-Air11.1%88.9%estimated ± 1.0 pp, medium confidence
GPT-4.1 mini10.2%88.6%estimated ± 1.0 pp, medium confidence
Gemini 2.5 Flash9.9%88.5%estimated ± 1.0 pp, medium confidence
o1-mini9.8%88.5%estimated ± 1.0 pp, medium confidence
DeepSeek V3 03249.7%88.5%estimated ± 1.0 pp, medium confidence
GPT-4.59.6%88.4%estimated ± 1.0 pp, medium confidence
Mistral Large 39.3%88.4%estimated ± 1.0 pp, low confidence
Magistral Medium9.0%88.3%estimated ± 1.0 pp, low confidence
Mistral Medium 39.0%88.3%estimated ± 1.0 pp, low confidence
Claude 3.5 Haiku8.9%88.3%estimated ± 1.0 pp, low confidence
Gemini 2.0 Flash8.9%88.3%estimated ± 1.0 pp, low confidence
Claude 3 Opus8.7%88.2%estimated ± 1.0 pp, low confidence
DeepSeek V38.5%88.1%estimated ± 1.0 pp, low confidence
GPT-4o8.4%88.1%estimated ± 1.0 pp, low confidence
DeepSeek R1 Distill Qwen 32B8.4%88.1%estimated ± 1.0 pp, low confidence
Magistral Small8.2%88.0%estimated ± 1.0 pp, low confidence
Gemini 1.5 Pro7.9%87.9%estimated ± 1.0 pp, low confidence
GPT-4.1 nano7.8%87.9%estimated ± 1.0 pp, low confidence
Solar Pro 37.8%87.9%estimated ± 1.0 pp, low confidence
Devstral Small7.6%87.9%estimated ± 1.0 pp, low confidence
Mistral Large 27.6%87.8%estimated ± 1.0 pp, low confidence
Mistral Small 3.17.1%87.7%estimated ± 1.0 pp, low confidence
Gemini 1.5 Flash7.1%87.7%estimated ± 1.0 pp, low confidence
GPT-4 Turbo7.0%87.7%estimated ± 1.0 pp, low confidence
Solar Pro 27.0%87.7%estimated ± 1.0 pp, low confidence
Nova Pro7.0%87.7%estimated ± 1.0 pp, low confidence
Qwen2.5 Coder 32B Instruct6.7%87.6%estimated ± 1.0 pp, low confidence
Gemini 2.5 Flash-Lite6.7%87.6%estimated ± 1.0 pp, low confidence
GPT-4o mini6.7%87.6%estimated ± 1.0 pp, low confidence
DeepSeek-V2.56.6%87.6%estimated ± 1.0 pp, low confidence
Muse Glimmer 30B6.3%87.5%estimated ± 1.0 pp, low confidence
Claude 3 Haiku5.6%87.2%estimated ± 1.0 pp, low confidence
Nemotron 3.5 Lightning 30B A3B NVFP45.5%87.2%estimated ± 1.0 pp, low confidence
Gemini 1.0 Pro5.3%87.2%estimated ± 1.0 pp, low confidence
Granite 4.2 8B5.1%87.1%estimated ± 1.0 pp, low confidence
Gemma 3 27B4.9%87.0%estimated ± 1.0 pp, low confidence
Gemma 4 E4B4.8%87.0%estimated ± 1.0 pp, low confidence
Granite 4.2 3B4.7%87.0%estimated ± 1.0 pp, low confidence
Mistral Medium 3.5 128B4.7%87.0%estimated ± 1.0 pp, low confidence
Mistral Small 44.3%86.9%estimated ± 1.0 pp, low confidence
North Mini Code3.4%86.6%estimated ± 1.0 pp, low confidence

Fitted on

ModelArtificial Analysis Intelligence IndexC-Eval
Claude Opus 4.523.7%92.2%
Kimi K2.523.5%94.0%
Qwen3.6-27B21.4%91.4%
Qwen3.6 Plus27.0%93.3%
QwQ-32B9.5%88.4%