Calibration

SWE-Marathon → Bug Hunt Bench

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap
License: https://creativecommons.org/licenses/by-nc/4.0/

Bug Hunt Bench is estimated from SWE-Marathon with a linear curve fitted on 5 models measured on both: y = 1.3895·x + -0.3865, R² = 0.94, cross-validated error 3.2 pp. It is used for 2 estimates.

Estimated model	SWE-Marathon	Bug Hunt Bench	Source
Step 5 Preview	72.7%	62.4%	estimated ± 3.2 pp, low confidence
Hy4 preview	31.9%	5.7%	estimated ± 3.2 pp, low confidence

Fitted on

Model	SWE-Marathon	Bug Hunt Bench
GLM-5.3	42.5%	19.0%
Kimi K3.1 Pro	51.7%	36.0%
Kimi K3	42.0%	21.0%
Muse Wonder	53.1%	31.7%
Muse Wonder Full	57.5%	42.0%