benchgap
Calibration

Vals MMLU-Pro → AA-HLE

AA-HLE is estimated from Vals MMLU-Pro with a inverse Michaelis–Menten curve fitted on 46 models measured on both: y = 0.06551·(x − 0.0000) / (0.0000 + 1.0294 − x), R² = 0.51, cross-validated error 7.5 pp. It is used for 0 estimates.

Fitted on

ModelVals MMLU-ProAA-HLE
Claude Fable 591.5%55.5%
Claude Fable 5.192.4%59.1%
Claude Opus 4.789.9%33.3%
Claude Opus 4.889.6%48.7%
Claude Opus 591.6%54.9%
Claude Sonnet 4.687.3%13.3%
Claude Sonnet 587.5%41.3%
DeepSeek V4 Flash 073186.2%38.6%
DeepSeek V4 Pro 081387.0%41.0%
Gemini 3.1 Pro91.0%47.0%
Gemini 3.5 Flash89.5%42.7%
Gemini 3.5 Flash-Lite85.8%18.8%
Gemini 3.6 Flash89.3%40.8%
Gemini 3.7 Flash90.1%47.9%
Gemini 3.8 Flash90.2%47.8%
Gemini 3 Flash88.6%15.0%
GLM-4.682.2%5.5%
GLM-4.782.7%27.4%
GLM-5.186.9%30.1%
GLM-5.286.7%41.1%
GLM-5.386.8%42.3%
GPT-5.4 mini84.6%28.1%
GPT-5.4 nano77.2%28.3%
GPT-5.588.1%45.8%
GPT-5.6 Luna86.0%39.5%
GPT-5.6 Sol89.1%49.5%
GPT-5.6 Terra86.7%42.9%
Grok 4.385.8%37.2%
Grok 4.589.2%42.7%
Grok 4.689.4%42.9%
Inkling86.3%31.9%
Inkling-Small85.6%33.3%
Kimi K2.687.6%37.5%
Kimi K388.0%46.9%
Ling 3.0 Flash82.0%23.7%
MiMo-V2.5-Pro84.6%35.7%
MiniMax M2.780.4%29.6%
MiniMax M384.2%39.0%
Mistral Medium 3.5 128B75.3%13.8%
Muse Spark87.3%40.7%
Muse Spark 1.188.7%46.2%
Muse Spark 1.288.3%45.5%
Nemotron 3 Ultra85.8%28.4%
Qwen3.6 Plus87.7%27.8%
Qwen3.7 Max89.3%40.5%
Qwen3.8-27B84.3%33.9%