Calibration
Vals LiveCodeBench → SWE-bench Pro
SWE-bench Pro is estimated from Vals LiveCodeBench with a offset logistic curve fitted on 32 models measured on both: y = 0.5697 + (0.8227 − 0.5697) / (1 + exp(−200.00·(x − 0.8833))), R² = 0.69, cross-validated error 4.8 pp. It is used for 4 estimates.
| Estimated model | Vals LiveCodeBench | SWE-bench Pro | Source |
|---|---|---|---|
| Gemini 3.7 Flash | 88.7% | 74.1% | estimated ± 4.8 pp, high confidence |
| GLM-4.5 | 67.4% | 57.0% | estimated ± 4.8 pp, medium confidence |
| Grok 4.3 | 84.5% | 57.0% | estimated ± 4.8 pp, high confidence |
| Qwen3.5 Flash | 83.3% | 57.0% | estimated ± 4.8 pp, high confidence |