Calibration
SWE-Marathon → DeepSWE
DeepSWE is estimated from SWE-Marathon with a offset logistic curve fitted on 7 models measured on both: y = 0.6441 + (0.7526 − 0.6441) / (1 + exp(−71.43·(x − 0.4378))), R² = 0.63, cross-validated error 5.7 pp. It is used for 0 estimates.
Fitted on
| Model | SWE-Marathon | DeepSWE |
|---|---|---|
| GLM-5.3 | 42.5% | 66.9% |
| Hy4 preview | 31.9% | 64.3% |
| Kimi K3.1 Pro | 51.7% | 75.5% |
| Kimi K3 | 42.0% | 67.5% |
| Muse Wonder | 53.1% | 78.8% |
| Muse Wonder Full | 57.5% | 79.0% |
| Step 5 Preview | 72.7% | 67.7% |