Calibration
SWE-bench Pro → Vals SWE-bench
Vals SWE-bench is estimated from SWE-bench Pro with a Hill curve fitted on 35 models measured on both: y = 0.0000 + (1.0285 − 0.0000)·x^4.13 / (0.43114^4.13 + x^4.13), R² = 0.54, cross-validated error 7.5 pp. It is used for 2 estimates.
| Estimated model | SWE-bench Pro | Vals SWE-bench | Source |
|---|---|---|---|
| Atria Dawn Preview | 59.6% | 81.4% | estimated ± 7.5 pp, medium confidence |
| Laguna S 2.1 | 59.4% | 81.2% | estimated ± 7.5 pp, medium confidence |