Calibration

Vals MMLU-Pro → AA-GPQA Diamond

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap
License: https://creativecommons.org/licenses/by-nc/4.0/

AA-GPQA Diamond is estimated from Vals MMLU-Pro with a linear curve fitted on 46 models measured on both: y = 1.0782·x + -0.0451, R² = 0.41, cross-validated error 4.5 pp. It is used for 0 estimates.

Fitted on

Model	Vals MMLU-Pro	AA-GPQA Diamond
Claude Fable 5	91.5%	92.6%
Claude Fable 5.1	92.4%	93.7%
Claude Opus 4.7	89.9%	88.5%
Claude Opus 4.8	89.6%	92.0%
Claude Opus 5	91.6%	93.2%
Claude Sonnet 4.6	87.3%	79.9%
Claude Sonnet 5	87.5%	91.1%
DeepSeek V4 Flash 0731	86.2%	90.8%
DeepSeek V4 Pro 0813	87.0%	92.8%
Gemini 3.1 Pro	91.0%	94.1%
Gemini 3.5 Flash	89.5%	92.2%
Gemini 3.5 Flash-Lite	85.8%	83.8%
Gemini 3.6 Flash	89.3%	92.8%
Gemini 3.7 Flash	90.1%	94.5%
Gemini 3.8 Flash	90.2%	95.3%
Gemini 3 Flash	88.6%	81.2%
GLM-4.6	82.2%	63.2%
GLM-4.7	82.7%	85.9%
GLM-5.1	86.9%	86.8%
GLM-5.2	86.7%	89.5%
GLM-5.3	86.8%	91.7%
GPT-5.4 mini	84.6%	87.5%
GPT-5.4 nano	77.2%	81.7%
GPT-5.5	88.1%	93.5%
GPT-5.6 Luna	86.0%	91.1%
GPT-5.6 Sol	89.1%	94.1%
GPT-5.6 Terra	86.7%	92.5%
Grok 4.3	85.8%	90.1%
Grok 4.5	89.2%	93.1%
Grok 4.6	89.4%	94.9%
Inkling	86.3%	87.2%
Inkling-Small	85.6%	89.5%
Kimi K2.6	87.6%	91.1%
Kimi K3	88.0%	93.5%
Ling 3.0 Flash	82.0%	85.5%
MiMo-V2.5-Pro	84.6%	86.6%
MiniMax M2.7	80.4%	87.4%
MiniMax M3	84.2%	92.9%
Mistral Medium 3.5 128B	75.3%	74.8%
Muse Spark	87.3%	88.4%
Muse Spark 1.1	88.7%	89.8%
Muse Spark 1.2	88.3%	90.4%
Nemotron 3 Ultra	85.8%	86.7%
Qwen3.6 Plus	87.7%	88.2%
Qwen3.7 Max	89.3%	92.3%
Qwen3.8-27B	84.3%	90.5%