benchgap
Calibration

Vals MMLU-Pro → Vals GPQA Diamond

Vals GPQA Diamond is estimated from Vals MMLU-Pro with a Hill curve fitted on 55 models measured on both: y = 0.0000 + (1.2000 − 0.0000)·x^6.00 / (0.73519^6.00 + x^6.00), R² = 0.77, cross-validated error 3.8 pp. It is used for 1 estimate.

Estimated modelVals MMLU-ProVals GPQA DiamondSource
Muse Spark 1.288.3%90.0%estimated ± 3.8 pp, high confidence

Fitted on

ModelVals MMLU-ProVals GPQA Diamond
Claude Fable 591.5%93.2%
Claude Fable 5.192.4%93.4%
Claude Haiku 4.578.7%72.2%
Claude Opus 4.789.9%90.2%
Claude Opus 4.889.6%92.4%
Claude Opus 591.6%93.4%
Claude Sonnet 4.687.3%85.6%
Claude Sonnet 587.5%88.9%
DeepSeek V4 Flash 073186.2%89.9%
DeepSeek V4 Pro 081387.0%92.4%
Gemini 3.1 Flash-Lite86.2%81.1%
Gemini 3.1 Pro91.0%95.5%
Gemini 3.5 Flash89.5%92.7%
Gemini 3.5 Flash-Lite85.8%83.8%
Gemini 3.6 Flash89.3%93.4%
Gemini 3.7 Flash90.1%93.9%
Gemini 3.8 Flash90.2%94.4%
Gemini 3 Flash88.6%87.9%
GLM-4.581.2%72.2%
GLM-4.682.2%74.5%
GLM-4.782.7%80.1%
GLM-5.186.9%84.5%
GLM-5.286.7%85.6%
GLM-5.386.8%88.1%
GLM-5.3-Flash86.1%86.4%
GPT-5.4 mini84.6%83.1%
GPT-5.4 nano77.2%77.5%
GPT-5.588.1%93.2%
GPT-5.6 Luna86.0%91.7%
GPT-5.6 Sol89.1%95.2%
GPT-5.6 Terra86.7%90.9%
Grok 4.2086.3%88.6%
Grok 4.385.8%91.4%
Grok 4.589.2%92.9%
Grok 4.689.4%94.7%
Inkling86.3%87.1%
Inkling-Small85.6%83.6%
Kimi K2.687.6%89.1%
Kimi K388.0%92.9%
Laguna M.168.8%27.0%
Laguna XS.269.1%55.1%
Ling 3.0 Flash82.0%84.8%
MiMo-V2.582.9%81.6%
MiMo-V2.5-Pro84.6%82.6%
MiniMax M2.780.4%86.6%
MiniMax M384.2%92.7%
Mistral Medium 3.5 128B75.3%34.8%
Muse Spark87.3%89.6%
Muse Spark 1.188.7%91.2%
Nemotron 3 Ultra85.8%86.1%
Qwen3.5 Flash84.1%82.8%
Qwen3.6 Plus87.7%87.4%
Qwen3.7 Max89.3%90.2%
Qwen3.8-27B84.3%88.9%
Qwen3.8 Max88.6%93.7%