Calibration
Vibe Code Bench → SWE-bench Pro
SWE-bench Pro is estimated from Vibe Code Bench with a linear curve fitted on 14 models measured on both: y = 0.0873·x + 0.5259, R² = 0.34, cross-validated error 2.9 pp. It is used for 0 estimates.
Fitted on
| Model | Vibe Code Bench | SWE-bench Pro |
|---|---|---|
| Claude Opus 4.6 | 57.6% | 53.4% |
| Claude Opus 4.7 | 71.0% | 64.3% |
| DeepSeek V4 Pro | 49.9% | 55.4% |
| Gemini 3.5 Flash | 48.7% | 55.1% |
| GLM-5.1 | 31.5% | 58.4% |
| GPT-5.2 | 53.5% | 55.6% |
| GPT-5.3 Codex | 61.8% | 56.8% |
| GPT-5.4 | 67.4% | 57.7% |
| GPT-5.5 | 69.8% | 58.6% |
| Grok 4.20 | 4.1% | 51.8% |
| Kimi K2.6 | 37.9% | 58.6% |
| MiniMax M2.7 | 27.0% | 56.2% |
| Muse Spark | 19.7% | 52.4% |
| Qwen3.6 Plus | 25.6% | 56.6% |