Calibration
SWE-Rebench → SWE-bench Verified
SWE-bench Verified is estimated from SWE-Rebench with a Michaelis–Menten curve fitted on 8 models measured on both: y = 2.0000·x / (0.95233 + x), R² = 0.32, cross-validated error 4.8 pp. It is used for 3 estimates.
| Estimated model | SWE-Rebench | SWE-bench Verified | Source |
|---|---|---|---|
| Composer 2 | 58.0% | 75.7% | estimated ± 4.8 pp, medium confidence |
| DeepSeek V3.2 | 60.9% | 78.0% | estimated ± 4.8 pp, medium confidence |
| Gemma 4 31B | 41.6% | 60.8% | estimated ± 4.8 pp, low confidence |