Calibration
Chess Puzzles → EBR-bench
EBR-bench is estimated from Chess Puzzles with a inverse Michaelis–Menten curve fitted on 28 models measured on both: y = 0.46996·(x − 0.0000) / (0.0000 + 1.1932 − x), R² = 0.82, cross-validated error 12.3 pp. It is used for 15 estimates.
| Estimated model | Chess Puzzles | EBR-bench | Source |
|---|---|---|---|
| GPT-5.5 Pro | 62.0% | 50.8% | estimated ± 12.3 pp, low confidence |
| GPT-5.4 Pro | 59.0% | 46.0% | estimated ± 12.3 pp, low confidence |
| Muse Spark 1.1 | 34.0% | 18.7% | estimated ± 12.3 pp, low confidence |
| Grok 4.20 | 29.0% | 15.1% | estimated ± 12.3 pp, low confidence |
| Muse Spark 1.2 | 27.0% | 13.7% | estimated ± 12.3 pp, low confidence |
| Seed 2.1 Turbo | 26.0% | 13.1% | estimated ± 12.3 pp, low confidence |
| GLM-4.7 | 21.0% | 10.0% | estimated ± 12.3 pp, low confidence |
| Inkling | 21.0% | 10.0% | estimated ± 12.3 pp, low confidence |
| Seed 2.1 Pro | 21.0% | 10.0% | estimated ± 12.3 pp, low confidence |
| Kimi K2.7 Code | 14.0% | 6.2% | estimated ± 12.3 pp, low confidence |
| GPT-4.1 | 11.0% | 4.8% | estimated ± 12.3 pp, low confidence |
| o1 | 10.0% | 4.3% | estimated ± 12.3 pp, low confidence |
| GPT-4o | 9.0% | 3.8% | estimated ± 12.3 pp, low confidence |
| Claude 3 Opus | 4.0% | 1.6% | estimated ± 12.3 pp, low confidence |
| GPT-4.1 nano | 0.0% | 0.0% | estimated ± 12.3 pp, low confidence |
Fitted on
| Model | Chess Puzzles | EBR-bench |
|---|---|---|
| Claude Fable 5 | 47.0% | 39.5% |
| Claude Fable 5.1 | 48.0% | 57.1% |
| Claude Fable 5.2 | 79.0% | 90.5% |
| Claude Haiku 5.5 | 55.0% | 43.8% |
| Claude Opus 4.5 | 14.0% | 14.3% |
| Claude Opus 4.6 | 21.0% | 12.7% |
| Claude Opus 4.7 | 41.0% | 19.0% |
| Claude Opus 4.8 | 52.0% | 28.6% |
| Claude Opus 5 | 43.0% | 45.7% |
| Claude Opus 5.5 | 67.0% | 71.4% |
| Claude Sonnet 4.5 | 10.0% | 2.4% |
| Gemini 3.1 Pro | 51.0% | 14.3% |
| Gemini 3.5 Flash | 45.0% | 4.8% |
| Gemini 4 Argon | 59.0% | 64.8% |
| Gemini 4 Carbon | 78.0% | 87.6% |
| GLM-5.2 | 22.0% | 9.5% |
| GPT-5 | 36.0% | 12.7% |
| GPT-5.2 | 48.0% | 23.0% |
| GPT-5.4 | 48.0% | 25.4% |
| GPT-5.5 | 63.0% | 34.3% |
| GPT-5.6 Sol | 59.0% | 44.8% |
| GPT-6.1 Sol | 68.0% | 54.3% |
| GPT-6.2 Astra | 78.0% | 92.4% |
| GPT-6 Astra | 74.0% | 76.2% |
| GPT-6 Sol | 56.0% | 53.3% |
| Grok 4.6 | 33.0% | 30.5% |
| Kimi K2.6 | 20.0% | 2.4% |
| Qwen3.7 Max | 27.0% | 9.5% |