benchgap
Calibration

Vals GPQA Diamond → AA-Omniscience Accuracy

AA-Omniscience Accuracy is estimated from Vals GPQA Diamond with a offset logistic curve fitted on 45 models measured on both: y = 0.2417 + (0.6464 − 0.2417) / (1 + exp(−42.21·(x − 0.9103))), R² = 0.73, cross-validated error 7.7 pp. It is used for 0 estimates.

Fitted on

ModelVals GPQA DiamondAA-Omniscience Accuracy
Claude Fable 593.2%65.4%
Claude Fable 5.193.4%68.4%
Claude Opus 4.790.2%44.7%
Claude Opus 4.892.4%48.8%
Claude Opus 593.4%60.9%
Claude Sonnet 4.685.6%38.6%
Claude Sonnet 588.9%40.1%
DeepSeek V4 Flash 073189.9%40.4%
DeepSeek V4 Pro 081392.4%49.1%
Gemini 3.5 Flash92.7%51.9%
Gemini 3.5 Flash-Lite83.8%26.5%
Gemini 3.6 Flash93.4%50.0%
Gemini 3.7 Flash93.9%55.3%
Gemini 3.8 Flash94.4%50.5%
Gemini 3 Flash87.9%45.8%
GLM-4.674.5%21.4%
GLM-4.780.1%29.3%
GLM-5.184.5%23.7%
GLM-5.285.6%24.3%
GLM-5.388.1%32.3%
GPT-5.4 mini83.1%37.5%
GPT-5.4 nano77.5%25.7%
GPT-5.593.2%58.0%
GPT-5.6 Luna91.7%42.7%
GPT-5.6 Sol95.2%59.4%
GPT-5.6 Terra90.9%46.8%
Grok 4.391.4%34.6%
Grok 4.592.9%51.6%
Grok 4.694.7%48.2%
Inkling87.1%39.3%
Inkling-Small83.6%31.8%
Kimi K2.689.1%32.6%
Kimi K392.9%47.6%
Ling 3.0 Flash84.8%18.2%
MiMo-V2.5-Pro82.6%22.4%
MiMo-V2-Flash59.3%15.6%
MiniMax M2.786.6%26.8%
MiniMax M392.7%44.9%
Mistral Medium 3.5 128B34.8%23.0%
Muse Spark89.6%49.6%
Muse Spark 1.191.2%52.1%
Nemotron 3 Ultra86.1%21.6%
Qwen3.6 Plus87.4%26.4%
Qwen3.7 Max90.2%31.1%
Qwen3.8-27B88.9%17.3%