benchgap
Calibration

Chess Puzzles → EBR-bench

EBR-bench is estimated from Chess Puzzles with a inverse Michaelis–Menten curve fitted on 28 models measured on both: y = 0.46996·(x − 0.0000) / (0.0000 + 1.1932 − x), R² = 0.82, cross-validated error 12.3 pp. It is used for 15 estimates.

Estimated modelChess PuzzlesEBR-benchSource
GPT-5.5 Pro62.0%50.8%estimated ± 12.3 pp, low confidence
GPT-5.4 Pro59.0%46.0%estimated ± 12.3 pp, low confidence
Muse Spark 1.134.0%18.7%estimated ± 12.3 pp, low confidence
Grok 4.2029.0%15.1%estimated ± 12.3 pp, low confidence
Muse Spark 1.227.0%13.7%estimated ± 12.3 pp, low confidence
Seed 2.1 Turbo26.0%13.1%estimated ± 12.3 pp, low confidence
GLM-4.721.0%10.0%estimated ± 12.3 pp, low confidence
Inkling21.0%10.0%estimated ± 12.3 pp, low confidence
Seed 2.1 Pro21.0%10.0%estimated ± 12.3 pp, low confidence
Kimi K2.7 Code14.0%6.2%estimated ± 12.3 pp, low confidence
GPT-4.111.0%4.8%estimated ± 12.3 pp, low confidence
o110.0%4.3%estimated ± 12.3 pp, low confidence
GPT-4o9.0%3.8%estimated ± 12.3 pp, low confidence
Claude 3 Opus4.0%1.6%estimated ± 12.3 pp, low confidence
GPT-4.1 nano0.0%0.0%estimated ± 12.3 pp, low confidence

Fitted on

ModelChess PuzzlesEBR-bench
Claude Fable 547.0%39.5%
Claude Fable 5.148.0%57.1%
Claude Fable 5.279.0%90.5%
Claude Haiku 5.555.0%43.8%
Claude Opus 4.514.0%14.3%
Claude Opus 4.621.0%12.7%
Claude Opus 4.741.0%19.0%
Claude Opus 4.852.0%28.6%
Claude Opus 543.0%45.7%
Claude Opus 5.567.0%71.4%
Claude Sonnet 4.510.0%2.4%
Gemini 3.1 Pro51.0%14.3%
Gemini 3.5 Flash45.0%4.8%
Gemini 4 Argon59.0%64.8%
Gemini 4 Carbon78.0%87.6%
GLM-5.222.0%9.5%
GPT-536.0%12.7%
GPT-5.248.0%23.0%
GPT-5.448.0%25.4%
GPT-5.563.0%34.3%
GPT-5.6 Sol59.0%44.8%
GPT-6.1 Sol68.0%54.3%
GPT-6.2 Astra78.0%92.4%
GPT-6 Astra74.0%76.2%
GPT-6 Sol56.0%53.3%
Grok 4.633.0%30.5%
Kimi K2.620.0%2.4%
Qwen3.7 Max27.0%9.5%