Calibration
Terminal-Bench Hard → Terminal-Bench 2.1 (Vals AI)
Terminal-Bench 2.1 (Vals AI) is estimated from Terminal-Bench Hard with a Hill curve fitted on 14 models measured on both: y = 0.5082 + (1.2000 − 0.5082)·x^6.00 / (0.66194^6.00 + x^6.00), R² = 0.80, cross-validated error 5.4 pp. It is used for 9 estimates.
| Estimated model | Terminal-Bench Hard | Terminal-Bench 2.1 (Vals AI) | Source |
|---|---|---|---|
| GPT-5.4 | 57.6% | 71.8% | estimated ± 5.4 pp, medium confidence |
| Claude Opus 4.6 | 48.5% | 60.1% | estimated ± 5.4 pp, medium confidence |
| Claude Opus 4.5 Thinking | 47.0% | 58.7% | estimated ± 5.4 pp, medium confidence |
| GPT-5.2 | 47.0% | 58.7% | estimated ± 5.4 pp, medium confidence |
| GPT-5.1 | 45.5% | 57.4% | estimated ± 5.4 pp, medium confidence |
| Muse Spark | 45.5% | 57.4% | estimated ± 5.4 pp, medium confidence |
| Qwen 3.6 Max (preview) | 43.9% | 56.2% | estimated ± 5.4 pp, medium confidence |
| GLM-5 | 43.2% | 55.8% | estimated ± 5.4 pp, medium confidence |
| Solar Open 2 | 28.3% | 51.2% | estimated ± 5.4 pp, low confidence |
Fitted on
| Model | Terminal-Bench Hard | Terminal-Bench 2.1 (Vals AI) |
|---|---|---|
| Claude Fable 5 | 62.9% | 80.5% |
| Claude Opus 4.7 | 54.5% | 68.5% |
| Claude Opus 4.8 | 58.3% | 71.9% |
| Gemini 3.1 Pro | 53.8% | 70.8% |
| GLM-5.2 | 50.8% | 67.8% |
| GPT-5.4 mini | 52.3% | 54.7% |
| GPT-5.5 | 60.6% | 76.4% |
| GPT-5.6 Sol | 65.9% | 85.8% |
| Kimi K2.6 | 43.9% | 53.6% |
| Kimi K2.7 Code | 44.7% | 67.0% |
| MiMo-V2.5-Pro | 43.2% | 57.3% |
| Qwen3.6 Plus | 43.9% | 53.2% |
| Qwen3.7 Max | 50.8% | 61.0% |
| Qwen3.7 Plus | 47.0% | 52.8% |