Calibration

ProgramBench → Vals SWE-bench

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

Vals SWE-bench is estimated from ProgramBench with a offset logistic curve fitted on 5 models measured on both: y = 0.8547 + (0.9700 − 0.8547) / (1 + exp(−127.70·(x − 0.7718))), R² = 0.43, cross-validated error 10.6 pp. It is used for 0 estimates.