Calibration
Terminal-Bench 4.0 → AA Terminal-Bench 4.0
AA Terminal-Bench 4.0 is estimated from Terminal-Bench 4.0 with a linear curve fitted on 11 models measured on both: y = 0.9435·x + -0.0098, R² = 0.93, cross-validated error 4.7 pp. It is used for 6 estimates.
| Estimated model | Terminal-Bench 4.0 | AA Terminal-Bench 4.0 | Source |
|---|---|---|---|
| Claude Mythos 5.1 | 60.9% | 56.5% | estimated ± 4.7 pp, high confidence |
| Ling 3.1 Flash | 40.4% | 37.1% | estimated ± 4.7 pp, high confidence |
| MiMo-V2.6-Flash | 28.8% | 26.2% | estimated ± 4.7 pp, high confidence |
| Pareto 26.10 Preview | 50.8% | 47.0% | estimated ± 4.7 pp, high confidence |
| Pareto 26.9 | 51.0% | 47.1% | estimated ± 4.7 pp, high confidence |
| SWE-2 | 27.3% | 24.8% | estimated ± 4.7 pp, high confidence |