Calibration
τ³-Bench Banking → AutomationBench
AutomationBench is estimated from τ³-Bench Banking with a Hill curve fitted on 15 models measured on both: y = 0.0632 + (0.6904 − 0.0632)·x^6.00 / (0.35530^6.00 + x^6.00), R² = 0.87, cross-validated error 10.6 pp. It is used for 4 estimates.
| Estimated model | τ³-Bench Banking | AutomationBench | Source |
|---|---|---|---|
| Qwen3.8 Max | 51.3% | 62.8% | estimated ± 10.6 pp, low confidence |
| Qwen3.8 2.4T A95B | 49.1% | 61.2% | estimated ± 10.6 pp, low confidence |
| Qwen3.8 27B (medium) | 47.4% | 59.6% | estimated ± 10.6 pp, low confidence |
| Muse Spark 1.3 (xhigh) | 47.2% | 59.4% | estimated ± 10.6 pp, low confidence |