Calibration
Vals SWE-bench → CursorBench 3.2
CursorBench 3.2 is estimated from Vals SWE-bench with a Michaelis–Menten curve fitted on 17 models measured on both: y = 2.0000·x / (1.95432 + x), R² = 0.47, cross-validated error 5.4 pp. It is used for 4 estimates.
| Estimated model | Vals SWE-bench | CursorBench 3.2 | Source |
|---|---|---|---|
| Gemini 3 Flash | 75.0% | 55.5% | estimated ± 5.4 pp, low confidence |
| GPT-5.2-Codex | 72.4% | 54.1% | estimated ± 5.4 pp, low confidence |
| Qwen3.5 Flash | 64.4% | 49.6% | estimated ± 5.4 pp, low confidence |
| Gemini 3.1 Flash-Lite | 62.8% | 48.6% | estimated ± 5.4 pp, low confidence |
Fitted on
| Model | Vals SWE-bench | CursorBench 3.2 |
|---|---|---|
| Claude Fable 5 | 95.0% | 70.5% |
| Claude Opus 4.8 | 88.6% | 62.3% |
| Claude Opus 5 | 97.0% | 70.0% |
| Claude Sonnet 5 | 79.6% | 61.5% |
| Composer 2.5 | 79.6% | 56.1% |
| Gemini 3.5 Flash | 78.8% | 48.8% |
| Gemini 3.6 Flash | 79.6% | 53.5% |
| Gemini 3.8 Flash | 80.0% | 69.2% |
| GLM-5.2 | 82.8% | 55.0% |
| GPT-5.5 | 82.6% | 58.4% |
| GPT-5.6 Luna | 93.0% | 61.1% |
| GPT-5.6 Sol | 96.2% | 67.2% |
| GPT-5.6 Terra | 95.4% | 64.9% |
| Grok 4.5 | 86.6% | 66.7% |
| Grok 4.6 | 95.6% | 70.8% |
| Kimi K2.7 Code | 78.2% | 49.7% |
| Kimi K3 | 93.4% | 60.8% |