Calibration

FrontierCode 1.1 Main → Vals SWE-bench

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

Vals SWE-bench is estimated from FrontierCode 1.1 Main with a Hill curve fitted on 9 models measured on both: y = 0.7448 + (1.2000 − 0.7448)·x^6.00 / (0.54965^6.00 + x^6.00), R² = 0.92, cross-validated error 2.8 pp. It is used for 3 estimates.

Estimated model	FrontierCode 1.1 Main	Vals SWE-bench	Source
Claude Haiku 5.5	46.4%	86.6%	estimated ± 2.8 pp, high confidence
SWE-1.7	42.3%	82.3%	estimated ± 2.8 pp, high confidence
SWE-2	50.0%	90.9%	estimated ± 2.8 pp, high confidence