Calibration
AutomationBench → AA EnterpriseOps-Gym
AA EnterpriseOps-Gym is estimated from AutomationBench with a Hill curve fitted on 5 models measured on both: y = 0.4969 + (0.0000 − 0.4969)·x^6.00 / (0.57675^6.00 + x^6.00), R² = 0.73, cross-validated error 5.1 pp. It is used for 6 estimates.
| Estimated model | AutomationBench | AA EnterpriseOps-Gym | Source |
|---|---|---|---|
| Atria Dawn Preview | 53.8% | 30.0% | estimated ± 5.1 pp, low confidence |
| DeepSeek V4 Flash 0731 | 25.1% | 49.4% | estimated ± 5.1 pp, low confidence |
| Hy4 preview | 32.1% | 48.3% | estimated ± 5.1 pp, low confidence |
| Ling 3.1 Flash | 52.5% | 31.7% | estimated ± 5.1 pp, low confidence |
| MiMo-V2.6-Flash | 52.3% | 31.9% | estimated ± 5.1 pp, low confidence |
| Qwen3.8 Max | 27.3% | 49.1% | estimated ± 5.1 pp, low confidence |