Calibration
Terminal-Bench 4.0 → AA Terminal-Bench 4.0
AA Terminal-Bench 4.0 is estimated from Terminal-Bench 4.0 with a Hill curve fitted on 12 models measured on both: y = 0.2185 + (0.6834 − 0.2185)·x^6.00 / (0.47052^6.00 + x^6.00), R² = 0.92, cross-validated error 6.0 pp. It is used for 8 estimates.
| Estimated model | Terminal-Bench 4.0 | AA Terminal-Bench 4.0 | Source |
|---|---|---|---|
| Claude Mythos 5.1 | 60.9% | 60.2% | estimated ± 6.0 pp, medium confidence |
| Gemini 4 Argon | 57.4% | 57.5% | estimated ± 6.0 pp, medium confidence |
| Pareto 26.9 | 51.0% | 50.6% | estimated ± 6.0 pp, medium confidence |
| Pareto 26.10 Preview | 50.8% | 50.3% | estimated ± 6.0 pp, medium confidence |
| Ling 3.1 Flash | 46.0% | 43.6% | estimated ± 6.0 pp, medium confidence |
| DeepSeek V4.1 Flash 1017 | 42.3% | 37.9% | estimated ± 6.0 pp, medium confidence |
| MiMo-V2.6-Flash | 28.8% | 24.2% | estimated ± 6.0 pp, medium confidence |
| SWE-2 | 27.3% | 23.6% | estimated ± 6.0 pp, medium confidence |
Fitted on
| Model | Terminal-Bench 4.0 | AA Terminal-Bench 4.0 |
|---|---|---|
| Claude Fable 5.1 | 55.8% | 52.0% |
| Claude Haiku 5.5 | 39.2% | 32.8% |
| Claude Opus 5.5 | 66.4% | 59.6% |
| Claude Sonnet 5.5 | 70.6% | 63.6% |
| Gemini 3.8 Flash | 19.1% | 19.7% |
| GPT-6.1 Terra | 50.3% | 55.6% |
| GPT-6 Astra | 57.9% | 59.1% |
| Grok 4.7 | 38.0% | 25.8% |
| Kimi K3.1 Pro | 65.7% | 67.2% |
| MiMo-V2.6-Pro | 34.9% | 34.8% |
| MiniMax M3.1 Flash | 38.1% | 25.8% |
| Step 5 Preview | 33.3% | 33.3% |