benchgap
Calibration

GDPval-AA → OSWorld-Verified

OSWorld-Verified is estimated from GDPval-AA with a linear curve fitted on 21 models measured on both: y = 0.5293·x + 0.5541, R² = 0.49, cross-validated error 8.4 pp. It is used for 58 estimates.

Estimated modelGDPval-AAOSWorld-VerifiedSource
Claude Opus 5.568.3%91.6%estimated ± 8.4 pp, low confidence
Muse Wonder Full67.9%91.3%estimated ± 8.4 pp, low confidence
Claude Sonnet 5.566.9%90.8%estimated ± 8.4 pp, low confidence
Muse Wonder63.5%89.0%estimated ± 8.4 pp, low confidence
Claude Fable 5.162.7%88.6%estimated ± 8.4 pp, low confidence
Grok 4.760.7%87.5%estimated ± 8.4 pp, low confidence
Qwen3.8 Max Preview58.3%86.3%estimated ± 8.4 pp, low confidence
Gemini 4 Argon56.2%85.2%estimated ± 8.4 pp, low confidence
Claude Haiku 5.556.0%85.1%estimated ± 8.4 pp, low confidence
GPT-6.1 Sol54.8%84.4%estimated ± 8.4 pp, low confidence
GPT-6 Sol50.4%82.1%estimated ± 8.4 pp, low confidence
Muse Spark 1.248.6%81.1%estimated ± 8.4 pp, low confidence
GPT-6 Luna46.6%80.1%estimated ± 8.4 pp, low confidence
Muse Spark Open45.1%79.3%estimated ± 8.4 pp, low confidence
Grok 4.544.2%78.8%estimated ± 8.4 pp, low confidence
Apodex 1.1 Mini34.4%73.6%estimated ± 8.4 pp, low confidence
Quasar 438B34.1%73.5%estimated ± 8.4 pp, low confidence
Ling 3.0 Flash VL33.0%72.9%estimated ± 8.4 pp, low confidence
Hy328.2%70.3%estimated ± 8.4 pp, low confidence
Kimi K2.7 Code26.9%69.6%estimated ± 8.4 pp, low confidence
A.X K223.0%67.6%estimated ± 8.4 pp, low confidence
GPT-520.9%66.5%estimated ± 8.4 pp, low confidence
MiniCPM5-2B10.4%60.9%estimated ± 8.4 pp, low confidence
MiMo-V2-Flash6.4%58.8%estimated ± 8.4 pp, low confidence
Granite 4.2 30B3.7%57.4%estimated ± 8.4 pp, low confidence
Gemma 4 26B A4B3.2%57.1%estimated ± 8.4 pp, low confidence
Ling 3.0 Tiny3.0%57.0%estimated ± 8.4 pp, low confidence
Command A+0.7%55.8%estimated ± 8.4 pp, low confidence
Celeris-10.0%55.4%estimated ± 8.4 pp, low confidence
Claude 3.5 Haiku0.0%55.4%estimated ± 8.4 pp, low confidence
DeepSeek V30.0%55.4%estimated ± 8.4 pp, low confidence
DeepSeek V3 03240.0%55.4%estimated ± 8.4 pp, low confidence
Gemma 3 27B0.0%55.4%estimated ± 8.4 pp, low confidence
Gemma 4 12B0.0%55.4%estimated ± 8.4 pp, low confidence
Gemma 4 E2B0.0%55.4%estimated ± 8.4 pp, low confidence
Gemma 4 E4B0.0%55.4%estimated ± 8.4 pp, low confidence
GPT-4.1 mini0.0%55.4%estimated ± 8.4 pp, low confidence
GPT-4.1 nano0.0%55.4%estimated ± 8.4 pp, low confidence
GPT-4o0.0%55.4%estimated ± 8.4 pp, low confidence
GPT-4o mini0.0%55.4%estimated ± 8.4 pp, low confidence
GPT-OSS 20B0.0%55.4%estimated ± 8.4 pp, low confidence
Granite 4.2 3B0.0%55.4%estimated ± 8.4 pp, low confidence
Granite 4.2 8B0.0%55.4%estimated ± 8.4 pp, low confidence
K-Exaone0.0%55.4%estimated ± 8.4 pp, low confidence
Ling 2.6 Flash0.0%55.4%estimated ± 8.4 pp, low confidence
Llama 3.3 70B0.0%55.4%estimated ± 8.4 pp, low confidence
Llama 4 Maverick0.0%55.4%estimated ± 8.4 pp, low confidence
Llama 4 Scout0.0%55.4%estimated ± 8.4 pp, low confidence
Mercury 2.50.0%55.4%estimated ± 8.4 pp, low confidence
Mistral Large 30.0%55.4%estimated ± 8.4 pp, low confidence
Mistral Small 3.10.0%55.4%estimated ± 8.4 pp, low confidence
Mistral Small 40.0%55.4%estimated ± 8.4 pp, low confidence
Nemotron 3 Nano 30B0.0%55.4%estimated ± 8.4 pp, low confidence
Nemotron 3 Nano Omni 30B A3B0.0%55.4%estimated ± 8.4 pp, low confidence
North Mini Code0.0%55.4%estimated ± 8.4 pp, low confidence
Solar Pro 30.0%55.4%estimated ± 8.4 pp, low confidence
Trinity-Large-Preview0.0%55.4%estimated ± 8.4 pp, low confidence
Ultravox v0.6 Llama 3.3 70B0.0%55.4%estimated ± 8.4 pp, low confidence

Fitted on

ModelGDPval-AAOSWorld-Verified
Claude Fable 555.5%85.0%
Claude Opus 4.742.6%78.0%
Claude Opus 4.847.7%83.4%
Claude Sonnet 548.2%81.2%
Gemini 3.5 Flash42.2%78.4%
Gemini 3.5 Flash-Lite24.2%74.0%
Gemini 3.6 Flash39.2%83.0%
GPT-5.437.3%75.0%
GPT-5.4 mini25.6%72.1%
GPT-5.4 nano22.4%39.0%
GPT-5.542.5%78.7%
Kimi K2.626.9%73.1%
Kimi K2.517.0%63.3%
MiMo-V2.6-Flash55.4%80.8%
MiMo-V2.6-Pro59.2%82.0%
MiniMax M337.1%70.1%
Muse Glimmer 30B14.2%65.9%
Muse Spark 1.135.6%80.8%
Qwen3.5-122B-A10B15.7%58.0%
Qwen3.7 Plus13.3%73.3%
Qwen3.8-27B45.9%84.3%