Calibration
Terminal-Bench 3.0 → AA Terminal-Bench 2.1
AA Terminal-Bench 2.1 is estimated from Terminal-Bench 3.0 with a Michaelis–Menten curve fitted on 9 models measured on both: y = 0.9217·x / (0.01724 + x), R² = 0.54, cross-validated error 2.7 pp. It is used for 4 estimates.
| Estimated model | Terminal-Bench 3.0 | AA Terminal-Bench 2.1 | Source |
|---|---|---|---|
| GPT-5.6 Sol | 34.6% | 87.8% | estimated ± 2.7 pp, high confidence |
| Ornith-1.5-397B | 13.5% | 81.7% | estimated ± 2.7 pp, medium confidence |
| Ornith-1.5-35B-A3B | 5.1% | 68.9% | estimated ± 2.7 pp, medium confidence |
| GLM-5.2 | 4.6% | 67.0% | estimated ± 2.7 pp, medium confidence |
Fitted on
| Model | Terminal-Bench 3.0 | AA Terminal-Bench 2.1 |
|---|---|---|
| Claude Fable 5 | 34.1% | 84.6% |
| Claude Opus 4.8 | 21.1% | 84.6% |
| Claude Opus 5 | 42.7% | 89.1% |
| Claude Sonnet 5 | 14.6% | 80.5% |
| Gemini 3.7 Flash | 14.9% | 85.8% |
| GPT-5.6 Luna | 14.3% | 80.9% |
| GPT-5.6 Terra | 20.8% | 88.0% |
| Grok 4.5 | 15.7% | 81.6% |
| Grok 4.6 | 26.5% | 88.4% |