benchgap
Calibration

Cookie-Bench → Chess Puzzles

Chess Puzzles is estimated from Cookie-Bench with a linear curve fitted on 5 models measured on both: y = 3.0245·x + -2.1013, R² = 0.90, cross-validated error 8.0 pp. It is used for 2 estimates.

Estimated modelCookie-BenchChess PuzzlesSource
Hy488.8%58.4%estimated ± 8.0 pp, low confidence
ERNIE 5.5 Flash Preview88.2%56.6%estimated ± 8.0 pp, low confidence

Fitted on

ModelCookie-BenchChess Puzzles
Claude Opus 5.591.5%67.0%
DeepSeek V4.1 Flash 101787.9%46.0%
GLM-5.380.4%37.0%
GLM-5.3-Flash76.9%21.0%
GPT-6.1 Sol89.7%68.0%