Calibration
FrontierSWE v2 → PostTrainBench
PostTrainBench is estimated from FrontierSWE v2 with a offset logistic curve fitted on 10 models measured on both: y = 0.3314 + (0.4223 − 0.3314) / (1 + exp(−78.14·(x − 0.5322))), R² = 0.86, cross-validated error 2.4 pp. It is used for 13 estimates.
| Estimated model | FrontierSWE v2 | PostTrainBench | Source |
|---|---|---|---|
| Claude Fable 5.2 | 69.9% | 42.2% | estimated ± 2.4 pp, medium confidence |
| Claude Sonnet 5.5 | 61.9% | 42.2% | estimated ± 2.4 pp, high confidence |
| Gemini 4 Hydrogen | 53.1% | 37.5% | estimated ± 2.4 pp, high confidence |
| Claude Fable 5 | 47.0% | 33.2% | estimated ± 2.4 pp, high confidence |
| Claude Haiku 5.5 | 43.8% | 33.1% | estimated ± 2.4 pp, high confidence |
| Grok 4.7 | 29.5% | 33.1% | estimated ± 2.4 pp, high confidence |
| Muse Spark 1.3 | 25.8% | 33.1% | estimated ± 2.4 pp, high confidence |
| Grok 4.6 | 25.3% | 33.1% | estimated ± 2.4 pp, high confidence |
| Gemini 3.7 Flash | 20.3% | 33.1% | estimated ± 2.4 pp, high confidence |
| Gemini 3.8 Flash | 19.6% | 33.1% | estimated ± 2.4 pp, high confidence |
| Qwen3.8 Max | 15.8% | 33.1% | estimated ± 2.4 pp, medium confidence |
| Muse Spark 1.2 | 12.0% | 33.1% | estimated ± 2.4 pp, medium confidence |
| Inkling | 4.1% | 33.1% | estimated ± 2.4 pp, medium confidence |
Fitted on
| Model | FrontierSWE v2 | PostTrainBench |
|---|---|---|
| Claude Opus 5 | 52.0% | 35.7% |
| Claude Opus 5.5 | 62.3% | 43.8% |
| Gemini 4 Argon | 55.0% | 40.4% |
| Gemini 4 Carbon | 61.2% | 41.0% |
| GLM-5.3 | 30.2% | 34.5% |
| GLM-5.3-Flash | 18.1% | 36.8% |
| GPT-5.6 Sol | 32.2% | 31.0% |
| GPT-6 Astra | 65.5% | 41.9% |
| Inkling 2 Small | 25.1% | 31.9% |
| Kimi K3 | 25.9% | 31.5% |