benchgap
Calibration

Claw-Eval → OSWorld-Verified

OSWorld-Verified is estimated from Claw-Eval with a Hill curve fitted on 9 models measured on both: y = 0.0000 + (0.7519 − 0.0000)·x^6.00 / (0.39826^6.00 + x^6.00), R² = 0.49, cross-validated error 3.8 pp. It is used for 32 estimates.

Estimated modelClaw-EvalOSWorld-VerifiedSource
Ornith-1.5-397B81.4%74.2%estimated ± 3.8 pp, low confidence
LongCat-2.578.3%73.9%estimated ± 3.8 pp, low confidence
K-EXAONE 2.077.7%73.9%estimated ± 3.8 pp, low confidence
Ornith-1.0-397B77.1%73.8%estimated ± 3.8 pp, low confidence
dots3-note Preview73.4%73.3%estimated ± 3.8 pp, medium confidence
Ornith-1.5-35B-A3B72.5%73.2%estimated ± 3.8 pp, medium confidence
Qwen3.6-27B72.4%73.2%estimated ± 3.8 pp, medium confidence
Spark-X2.6-4B69.9%72.7%estimated ± 3.8 pp, medium confidence
Ornith-1.0-35B69.8%72.7%estimated ± 3.8 pp, medium confidence
Qwen3.6-35B-A3B68.7%72.4%estimated ± 3.8 pp, medium confidence
LFM3-1.7B68.6%72.4%estimated ± 3.8 pp, medium confidence
Step 3.7 Flash67.1%72.0%estimated ± 3.8 pp, medium confidence
Ornith-1.5-9B66.5%71.9%estimated ± 3.8 pp, medium confidence
Qwen3.7 Max65.2%71.5%estimated ± 3.8 pp, medium confidence
LLaDA2.2-flash64.2%71.1%estimated ± 3.8 pp, medium confidence
MiMo-V2.5-Pro63.8%71.0%estimated ± 3.8 pp, medium confidence
Muse Spark63.8%71.0%estimated ± 3.8 pp, medium confidence
Ornith-1.0-9B63.1%70.7%estimated ± 3.8 pp, medium confidence
LFM2.5-2.6B62.8%70.6%estimated ± 3.8 pp, medium confidence
GLM-5.162.3%70.4%estimated ± 3.8 pp, medium confidence
MiMo-V2.562.3%70.4%estimated ± 3.8 pp, medium confidence
Qwen3.6 Plus58.8%68.6%estimated ± 3.8 pp, medium confidence
Gemini 3.1 Pro57.8%67.9%estimated ± 3.8 pp, medium confidence
MiMo-V2-Pro57.8%67.9%estimated ± 3.8 pp, medium confidence
GLM-557.7%67.9%estimated ± 3.8 pp, medium confidence
LLaDA2.2-mini57.2%67.5%estimated ± 3.8 pp, medium confidence
GLM-5-Turbo55.8%66.4%estimated ± 3.8 pp, medium confidence
GLM-5V-Turbo53.8%64.6%estimated ± 3.8 pp, medium confidence
Gemini 3 Flash49.2%58.7%estimated ± 3.8 pp, low confidence
MiniMax M2.748.7%57.9%estimated ± 3.8 pp, low confidence
MiMo-V2-Omni45.2%51.2%estimated ± 3.8 pp, low confidence
DeepSeek V3.240.2%38.7%estimated ± 3.8 pp, low confidence

Fitted on

ModelClaw-EvalOSWorld-Verified
Claude Opus 4.559.6%66.3%
Claude Opus 4.670.4%72.7%
Claude Sonnet 4.667.8%72.1%
GPT-5.460.3%75.0%
Kimi K2.662.3%73.1%
Kimi K2.552.3%63.3%
MiniMax M374.5%70.1%
Qwen3.5 397B56.8%62.2%
Qwen3.7 Plus62.7%73.3%