Calibration
PostTrainBench v1.1 → Vals SWE-bench
Vals SWE-bench is estimated from PostTrainBench v1.1 with a Hill curve fitted on 10 models measured on both: y = 0.7868 + (1.2000 − 0.7868)·x^6.00 / (0.38364^6.00 + x^6.00), R² = 0.74, cross-validated error 4.1 pp. It is used for 1 estimate.
| Estimated model | PostTrainBench v1.1 | Vals SWE-bench | Source |
|---|---|---|---|
| MiniMax M3.1 Flash | 39.3% | 100.0% | estimated ± 4.1 pp, medium confidence |
Fitted on
| Model | PostTrainBench v1.1 | Vals SWE-bench |
|---|---|---|
| Claude Opus 4.7 | 28.6% | 82.0% |
| Claude Opus 4.8 | 32.9% | 88.6% |
| Claude Opus 5 | 35.0% | 97.0% |
| Gemini 3.1 Pro | 22.0% | 78.8% |
| GLM-5.2 | 31.7% | 82.8% |
| GPT-5.4 | 19.0% | 78.2% |
| GPT-5.5 | 27.2% | 82.6% |
| GPT-5.6 Sol | 36.2% | 96.2% |
| Grok 4.5 | 23.5% | 86.6% |
| Kimi K3 | 32.0% | 93.4% |