Calibration
Vals SWE-bench → CursorBench 3.1
CursorBench 3.1 is estimated from Vals SWE-bench with a linear curve fitted on 7 models measured on both: y = 1.0263·x + -0.2797, R² = 0.68, cross-validated error 5.8 pp. It is used for 6 estimates.
| Estimated model | Vals SWE-bench | CursorBench 3.1 | Source |
|---|---|---|---|
| Claude Haiku 4.5 | 66.6% | 40.4% | estimated ± 5.8 pp, low confidence |
| Claude Opus 4.7 | 82.0% | 56.2% | estimated ± 5.8 pp, low confidence |
| Gemini 3.1 Flash-Lite | 62.8% | 36.5% | estimated ± 5.8 pp, low confidence |
| Gemini 3 Flash | 75.0% | 49.0% | estimated ± 5.8 pp, low confidence |
| GPT-5.2-Codex | 72.4% | 46.3% | estimated ± 5.8 pp, low confidence |
| Qwen3.5 Flash | 64.4% | 38.1% | estimated ± 5.8 pp, low confidence |