Calibration
SWE-bench Pro → SciCode
SciCode is estimated from SWE-bench Pro with a linear curve fitted on 15 models measured on both: y = 0.4756·x + 0.2290, R² = 0.77, cross-validated error 4.8 pp. It is used for 4 estimates.
| Estimated model | SWE-bench Pro | SciCode | Source |
|---|---|---|---|
| Atria Dawn Preview | 59.6% | 51.2% | estimated ± 4.8 pp, high confidence |
| Hy4 preview | 65.7% | 54.1% | estimated ± 4.8 pp, high confidence |
| Laguna S 2.1 | 59.4% | 51.1% | estimated ± 4.8 pp, high confidence |
| Qwen3.8-Omni-Flash | 63.3% | 53.0% | estimated ± 4.8 pp, high confidence |