benchgap
Calibration

Vals LiveCodeBench → FrontierCode 1.1 Main

FrontierCode 1.1 Main is estimated from Vals LiveCodeBench with a Hill curve fitted on 9 models measured on both: y = 0.0000 + (1.2000 − 0.0000)·x^6.00 / (0.95372^6.00 + x^6.00), R² = 0.66, cross-validated error 6.2 pp. It is used for 3 estimates.

Estimated modelVals LiveCodeBenchFrontierCode 1.1 MainSource
GPT-5.1-Codex85.6%41.2%estimated ± 6.2 pp, medium confidence
GPT-5.1-Codex-Max83.6%37.4%estimated ± 6.2 pp, medium confidence
GLM-4.681.0%32.7%estimated ± 6.2 pp, low confidence

Fitted on

ModelVals LiveCodeBenchFrontierCode 1.1 Main
Claude Fable 589.8%53.5%
Claude Opus 4.785.1%38.5%
Claude Opus 4.887.8%46.5%
Claude Opus 589.0%53.4%
Claude Sonnet 4.682.1%24.3%
Claude Sonnet 582.4%42.7%
Gemini 3.7 Flash88.7%43.6%
GPT-5.4 mini81.5%27.0%
GPT-5.585.3%43.0%