benchgap
Calibration

AA-Omniscience Accuracy → MMLU-Pro

MMLU-Pro is estimated from AA-Omniscience Accuracy with a Hill curve fitted on 30 models measured on both: y = 0.6806 + (0.8554 − 0.6806)·x^6.00 / (0.14042^6.00 + x^6.00), R² = 0.56, cross-validated error 5.1 pp. It is used for 63 estimates.

Estimated modelAA-Omniscience AccuracyMMLU-ProSource
GPT-6.2 Astra68.6%85.5%estimated ± 5.1 pp, low confidence
GPT-6.1 Sol61.8%85.5%estimated ± 5.1 pp, low confidence
GPT-6.2 Sol61.6%85.5%estimated ± 5.1 pp, low confidence
GPT-6 Astra61.6%85.5%estimated ± 5.1 pp, low confidence
Gemini 4 Carbon58.3%85.5%estimated ± 5.1 pp, low confidence
GPT-6 Sol54.5%85.5%estimated ± 5.1 pp, low confidence
Gemini 4 Hydrogen50.5%85.5%estimated ± 5.1 pp, low confidence
Gemini 4 Argon48.3%85.5%estimated ± 5.1 pp, medium confidence
GPT-6.1 Terra47.6%85.5%estimated ± 5.1 pp, medium confidence
DeepSeek V4.1 Flash 101746.4%85.5%estimated ± 5.1 pp, medium confidence
Muse Spark 1.343.6%85.5%estimated ± 5.1 pp, medium confidence
Grok 4.742.7%85.5%estimated ± 5.1 pp, medium confidence
Muse Spark Open42.0%85.5%estimated ± 5.1 pp, medium confidence
GPT-5.2-Codex41.1%85.5%estimated ± 5.1 pp, medium confidence
Muse Wonder Full41.0%85.5%estimated ± 5.1 pp, medium confidence
GPT-5.1-Codex39.9%85.5%estimated ± 5.1 pp, medium confidence
GPT-5.1-Codex-Max39.9%85.5%estimated ± 5.1 pp, medium confidence
GPT-6 Luna39.6%85.5%estimated ± 5.1 pp, medium confidence
Gemini 2.5 Pro39.1%85.5%estimated ± 5.1 pp, medium confidence
Muse Wonder38.0%85.5%estimated ± 5.1 pp, medium confidence
Ling 3.0 Tiny37.9%85.5%estimated ± 5.1 pp, medium confidence
Qwen 3.6 Max (preview)37.9%85.5%estimated ± 5.1 pp, medium confidence
Step 5 Preview36.4%85.5%estimated ± 5.1 pp, medium confidence
MiMo-V2.6-Pro34.8%85.5%estimated ± 5.1 pp, medium confidence
Hy3 Preview31.5%85.4%estimated ± 5.1 pp, medium confidence
Qwen3.8 Max Preview29.5%85.3%estimated ± 5.1 pp, medium confidence
GLM-5V-Turbo29.3%85.3%estimated ± 5.1 pp, medium confidence
GLM-5-Turbo28.4%85.3%estimated ± 5.1 pp, medium confidence
Apodex 1.128.2%85.3%estimated ± 5.1 pp, medium confidence
Apodex 1.1 Mini28.2%85.3%estimated ± 5.1 pp, medium confidence
Muse Glimmer 30B28.0%85.3%estimated ± 5.1 pp, medium confidence
MiMo-V2.6-Flash27.2%85.2%estimated ± 5.1 pp, medium confidence
MiMo-V2-Pro26.6%85.2%estimated ± 5.1 pp, medium confidence
Gemini 2.5 Flash26.1%85.1%estimated ± 5.1 pp, medium confidence
Mistral Large 426.0%85.1%estimated ± 5.1 pp, medium confidence
Step 3.7 Flash25.8%85.1%estimated ± 5.1 pp, medium confidence
DeepSeek V3 032424.3%84.9%estimated ± 5.1 pp, medium confidence
DeepSeek V3.224.0%84.9%estimated ± 5.1 pp, medium confidence
Grok Code Fast 123.5%84.8%estimated ± 5.1 pp, medium confidence
Claude 4 Sonnet22.7%84.6%estimated ± 5.1 pp, medium confidence
Mercury 2.522.0%84.4%estimated ± 5.1 pp, medium confidence
Mistral Small 421.4%84.2%estimated ± 5.1 pp, medium confidence
Muse Ember21.1%84.1%estimated ± 5.1 pp, medium confidence
Magistral Medium20.3%83.8%estimated ± 5.1 pp, medium confidence
Mistral Large 219.9%83.6%estimated ± 5.1 pp, medium confidence
MiMo-V2-Omni19.3%83.3%estimated ± 5.1 pp, medium confidence
Solar Pro 318.5%82.7%estimated ± 5.1 pp, medium confidence
North Mini Code18.4%82.7%estimated ± 5.1 pp, medium confidence
Mistral Medium 318.3%82.6%estimated ± 5.1 pp, medium confidence
Magistral Small18.0%82.3%estimated ± 5.1 pp, medium confidence
Claude 3 Haiku17.6%82.0%estimated ± 5.1 pp, medium confidence
A.X K217.5%81.9%estimated ± 5.1 pp, medium confidence
Grok 4.1 Fast17.2%81.5%estimated ± 5.1 pp, medium confidence
Nova Pro16.9%81.2%estimated ± 5.1 pp, medium confidence
K-Exaone16.4%80.6%estimated ± 5.1 pp, medium confidence
GLM-4.5-Air16.3%80.5%estimated ± 5.1 pp, medium confidence
Solar Pro 216.1%80.2%estimated ± 5.1 pp, medium confidence
Ling 2.6 Flash15.6%79.5%estimated ± 5.1 pp, medium confidence
Devstral Small14.7%78.0%estimated ± 5.1 pp, medium confidence
Gemma 3 27B13.0%74.8%estimated ± 5.1 pp, medium confidence
Command A+9.8%69.9%estimated ± 5.1 pp, medium confidence
LFM2.5-8B-A1B9.4%69.5%estimated ± 5.1 pp, medium confidence
LFM3-1.7B7.1%68.3%estimated ± 5.1 pp, low confidence

Fitted on

ModelAA-Omniscience AccuracyMMLU-Pro
Claude 3.5 Haiku13.2%65.0%
Claude Opus 4.540.9%89.5%
Claude Opus 4.645.8%82.0%
Claude Sonnet 4.638.6%79.2%
DeepSeek-R130.5%84.0%
DeepSeek V325.5%75.9%
DeepSeek V3.123.1%83.7%
DeepSeek V4 Flash 073140.4%86.2%
DeepSeek V4 Pro49.1%87.5%
Gemma 4 12B15.7%77.2%
Gemma 4 31B19.9%85.2%
Gemma 4 E4B9.7%69.4%
GLM-4.729.3%84.3%
GLM-526.3%85.7%
Granite 4.2 30B11.2%77.6%
Granite 4.2 3B9.2%67.8%
Granite 4.2 8B11.8%74.0%
K-EXAONE 2.014.5%83.5%
Kimi K227.4%81.1%
Kimi K2.535.2%87.1%
MiMo-V2-Flash15.6%84.9%
Mistral Small 3.115.1%66.8%
Nemotron 3.5 Lightning 30B A3B NVFP414.7%81.6%
Nemotron 3 Ultra21.6%86.8%
Qwen3.5-27B20.7%86.1%
Qwen3.5-35B-A3B20.1%85.3%
Qwen3.6-27B19.6%86.2%
Qwen3.6 Plus26.4%88.5%
Qwen3.7 Max31.1%89.6%
Solar Pro 418.4%86.3%