Calibration
Claw-Eval → OSWorld-Verified
OSWorld-Verified is estimated from Claw-Eval with a Hill curve fitted on 9 models measured on both: y = 0.0000 + (0.7519 − 0.0000)·x^6.00 / (0.39826^6.00 + x^6.00), R² = 0.49, cross-validated error 3.8 pp. It is used for 32 estimates.
| Estimated model | Claw-Eval | OSWorld-Verified | Source |
|---|---|---|---|
| Ornith-1.5-397B | 81.4% | 74.2% | estimated ± 3.8 pp, low confidence |
| LongCat-2.5 | 78.3% | 73.9% | estimated ± 3.8 pp, low confidence |
| K-EXAONE 2.0 | 77.7% | 73.9% | estimated ± 3.8 pp, low confidence |
| Ornith-1.0-397B | 77.1% | 73.8% | estimated ± 3.8 pp, low confidence |
| dots3-note Preview | 73.4% | 73.3% | estimated ± 3.8 pp, medium confidence |
| Ornith-1.5-35B-A3B | 72.5% | 73.2% | estimated ± 3.8 pp, medium confidence |
| Qwen3.6-27B | 72.4% | 73.2% | estimated ± 3.8 pp, medium confidence |
| Spark-X2.6-4B | 69.9% | 72.7% | estimated ± 3.8 pp, medium confidence |
| Ornith-1.0-35B | 69.8% | 72.7% | estimated ± 3.8 pp, medium confidence |
| Qwen3.6-35B-A3B | 68.7% | 72.4% | estimated ± 3.8 pp, medium confidence |
| LFM3-1.7B | 68.6% | 72.4% | estimated ± 3.8 pp, medium confidence |
| Step 3.7 Flash | 67.1% | 72.0% | estimated ± 3.8 pp, medium confidence |
| Ornith-1.5-9B | 66.5% | 71.9% | estimated ± 3.8 pp, medium confidence |
| Qwen3.7 Max | 65.2% | 71.5% | estimated ± 3.8 pp, medium confidence |
| LLaDA2.2-flash | 64.2% | 71.1% | estimated ± 3.8 pp, medium confidence |
| MiMo-V2.5-Pro | 63.8% | 71.0% | estimated ± 3.8 pp, medium confidence |
| Muse Spark | 63.8% | 71.0% | estimated ± 3.8 pp, medium confidence |
| Ornith-1.0-9B | 63.1% | 70.7% | estimated ± 3.8 pp, medium confidence |
| LFM2.5-2.6B | 62.8% | 70.6% | estimated ± 3.8 pp, medium confidence |
| GLM-5.1 | 62.3% | 70.4% | estimated ± 3.8 pp, medium confidence |
| MiMo-V2.5 | 62.3% | 70.4% | estimated ± 3.8 pp, medium confidence |
| Qwen3.6 Plus | 58.8% | 68.6% | estimated ± 3.8 pp, medium confidence |
| Gemini 3.1 Pro | 57.8% | 67.9% | estimated ± 3.8 pp, medium confidence |
| MiMo-V2-Pro | 57.8% | 67.9% | estimated ± 3.8 pp, medium confidence |
| GLM-5 | 57.7% | 67.9% | estimated ± 3.8 pp, medium confidence |
| LLaDA2.2-mini | 57.2% | 67.5% | estimated ± 3.8 pp, medium confidence |
| GLM-5-Turbo | 55.8% | 66.4% | estimated ± 3.8 pp, medium confidence |
| GLM-5V-Turbo | 53.8% | 64.6% | estimated ± 3.8 pp, medium confidence |
| Gemini 3 Flash | 49.2% | 58.7% | estimated ± 3.8 pp, low confidence |
| MiniMax M2.7 | 48.7% | 57.9% | estimated ± 3.8 pp, low confidence |
| MiMo-V2-Omni | 45.2% | 51.2% | estimated ± 3.8 pp, low confidence |
| DeepSeek V3.2 | 40.2% | 38.7% | estimated ± 3.8 pp, low confidence |
Fitted on
| Model | Claw-Eval | OSWorld-Verified |
|---|---|---|
| Claude Opus 4.5 | 59.6% | 66.3% |
| Claude Opus 4.6 | 70.4% | 72.7% |
| Claude Sonnet 4.6 | 67.8% | 72.1% |
| GPT-5.4 | 60.3% | 75.0% |
| Kimi K2.6 | 62.3% | 73.1% |
| Kimi K2.5 | 52.3% | 63.3% |
| MiniMax M3 | 74.5% | 70.1% |
| Qwen3.5 397B | 56.8% | 62.2% |
| Qwen3.7 Plus | 62.7% | 73.3% |