Calibration

FrontierSWE v2 → SWE-bench Pro

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

SWE-bench Pro is estimated from FrontierSWE v2 with a linear curve fitted on 8 models measured on both: y = 0.4950·x + 0.5426, R² = 0.88, cross-validated error 5.1 pp. It is used for 3 estimates.

Estimated model	FrontierSWE v2	SWE-bench Pro	Source
Gemini 4 Argon	55.0%	81.5%	estimated ± 5.1 pp, medium confidence
GPT-6 Astra	65.5%	86.7%	estimated ± 5.1 pp, low confidence
Muse Spark 1.2	12.0%	60.2%	estimated ± 5.1 pp, medium confidence