Calibration
AA EnterpriseOps-Gym → AutomationBench
AutomationBench is estimated from AA EnterpriseOps-Gym with a linear curve fitted on 5 models measured on both: y = -0.9635·x + 0.8151, R² = 0.61, cross-validated error 7.0 pp. It is used for 8 estimates.
| Estimated model | AA EnterpriseOps-Gym | AutomationBench | Source |
|---|---|---|---|
| Claude Fable 5 | 51.1% | 32.3% | estimated ± 7.0 pp, low confidence |
| Gemini 3.5 Flash | 50.1% | 33.2% | estimated ± 7.0 pp, low confidence |
| Grok 4.6 | 48.3% | 35.0% | estimated ± 7.0 pp, low confidence |
| Inkling | 38.0% | 44.9% | estimated ± 7.0 pp, low confidence |
| MiniMax M3 | 32.1% | 50.6% | estimated ± 7.0 pp, low confidence |
| Muse Glimmer 30B | 34.7% | 48.1% | estimated ± 7.0 pp, low confidence |
| Nemotron 3 Ultra | 28.9% | 53.7% | estimated ± 7.0 pp, low confidence |
| Qwen3.8-27B | 44.2% | 38.9% | estimated ± 7.0 pp, low confidence |