Calibration
Vals SWE-bench → CursorBench 4.0
CursorBench 4.0 is estimated from Vals SWE-bench with a inverse Michaelis–Menten curve fitted on 8 models measured on both: y = 0.46732·(x − 0.0000) / (0.0000 + 2.0000 − x), R² = 0.46, cross-validated error 4.4 pp. It is used for 9 estimates.
| Estimated model | Vals SWE-bench | CursorBench 4.0 | Source |
|---|---|---|---|
| Claude Opus 4.7 | 82.0% | 32.5% | estimated ± 4.4 pp, medium confidence |
| Claude Sonnet 4.6 | 77.4% | 29.5% | estimated ± 4.4 pp, low confidence |
| Gemini 3 Flash | 75.0% | 28.0% | estimated ± 4.4 pp, low confidence |
| GPT-5.2-Codex | 72.4% | 26.5% | estimated ± 4.4 pp, low confidence |
| GLM-4.7 | 69.4% | 24.8% | estimated ± 4.4 pp, low confidence |
| Claude Haiku 4.5 | 66.6% | 23.3% | estimated ± 4.4 pp, low confidence |
| Mistral Medium 3.5 128B | 66.4% | 23.2% | estimated ± 4.4 pp, low confidence |
| Qwen3.5 Flash | 64.4% | 22.2% | estimated ± 4.4 pp, low confidence |
| Gemini 3.1 Flash-Lite | 62.8% | 21.4% | estimated ± 4.4 pp, low confidence |
Fitted on
| Model | Vals SWE-bench | CursorBench 4.0 |
|---|---|---|
| Claude Opus 5 | 97.0% | 46.6% |
| Claude Sonnet 5 | 79.6% | 34.1% |
| Composer 2.5 | 79.6% | 27.7% |
| Gemini 3.8 Flash | 80.0% | 39.6% |
| GPT-5.6 Luna | 93.0% | 35.9% |
| GPT-5.6 Sol | 96.2% | 41.7% |
| GPT-5.6 Terra | 95.4% | 41.3% |
| Grok 4.6 | 95.6% | 41.4% |