Calibration
SWE-bench Pro → FrontierCode 1.1 Main
FrontierCode 1.1 Main is estimated from SWE-bench Pro with a Michaelis–Menten curve fitted on 8 models measured on both: y = 2.0000·x / (2.39264 + x), R² = 0.73, cross-validated error 5.5 pp. It is used for 10 estimates.
| Estimated model | SWE-bench Pro | FrontierCode 1.1 Main | Source |
|---|---|---|---|
| Sakana Fugu-Ultra | 73.7% | 47.1% | estimated ± 5.5 pp, medium confidence |
| Sage-A1.5 | 61.2% | 40.7% | estimated ± 5.5 pp, medium confidence |
| Atria Dawn Preview | 59.6% | 39.9% | estimated ± 5.5 pp, medium confidence |
| Laguna S 2.1 | 59.4% | 39.8% | estimated ± 5.5 pp, medium confidence |
| Sakana Fugu | 59.0% | 39.6% | estimated ± 5.5 pp, medium confidence |
| Spark-X2.6-4B | 54.3% | 37.0% | estimated ± 5.5 pp, medium confidence |
| Granite 4.2 30B | 33.3% | 24.4% | estimated ± 5.5 pp, low confidence |
| LLaDA2.2-flash | 30.1% | 22.3% | estimated ± 5.5 pp, low confidence |
| Mellum2.1-12B-A2.5B-Thinking | 28.0% | 21.0% | estimated ± 5.5 pp, low confidence |
| Granite 4.2 8B | 19.1% | 14.8% | estimated ± 5.5 pp, low confidence |
Fitted on
| Model | SWE-bench Pro | FrontierCode 1.1 Main |
|---|---|---|
| Claude Fable 5 | 80.0% | 53.5% |
| Claude Opus 4.6 | 53.4% | 26.9% |
| Claude Opus 4.8 | 69.2% | 46.5% |
| Claude Opus 5 | 79.2% | 53.4% |
| Claude Opus 5.5 | 89.9% | 54.4% |
| Claude Sonnet 5 | 63.2% | 42.7% |
| Claude Sonnet 5.5 | 81.3% | 46.2% |
| GPT-5.5 | 58.6% | 43.0% |