benchgap
Calibration

PostTrainBench v1.1 → Vals SWE-bench

Vals SWE-bench is estimated from PostTrainBench v1.1 with a Hill curve fitted on 10 models measured on both: y = 0.7868 + (1.2000 − 0.7868)·x^6.00 / (0.38364^6.00 + x^6.00), R² = 0.74, cross-validated error 4.1 pp. It is used for 1 estimate.

Estimated modelPostTrainBench v1.1Vals SWE-benchSource
MiniMax M3.1 Flash39.3%100.0%estimated ± 4.1 pp, medium confidence

Fitted on

ModelPostTrainBench v1.1Vals SWE-bench
Claude Opus 4.728.6%82.0%
Claude Opus 4.832.9%88.6%
Claude Opus 535.0%97.0%
Gemini 3.1 Pro22.0%78.8%
GLM-5.231.7%82.8%
GPT-5.419.0%78.2%
GPT-5.527.2%82.6%
GPT-5.6 Sol36.2%96.2%
Grok 4.523.5%86.6%
Kimi K332.0%93.4%