Calibration
Vals SWE-bench → CursorBench 4.0
CursorBench 4.0 is estimated from Vals SWE-bench with a inverse Michaelis–Menten curve fitted on 8 models measured on both: y = 0.46732·(x − 0.0000) / (0.0000 + 2.0000 − x), R² = 0.46, cross-validated error 4.4 pp. It is used for 6 estimates.
| Estimated model | Vals SWE-bench | CursorBench 4.0 | Source |
|---|---|---|---|
| Claude Opus 4.7 | 82.0% | 32.5% | estimated ± 4.4 pp, medium confidence |
| Claude Sonnet 4.6 | 77.4% | 29.5% | estimated ± 4.4 pp, low confidence |
| Gemini 3.1 Flash-Lite | 62.8% | 21.4% | estimated ± 4.4 pp, low confidence |
| Gemini 3 Flash | 75.0% | 28.0% | estimated ± 4.4 pp, low confidence |
| GPT-5.2-Codex | 72.4% | 26.5% | estimated ± 4.4 pp, low confidence |
| Qwen3.5 Flash | 64.4% | 22.2% | estimated ± 4.4 pp, low confidence |