Calibration
AutomationBench → CyberGym
CyberGym is estimated from AutomationBench with a linear curve fitted on 10 models measured on both: y = 0.4430·x + 0.6609, R² = 0.69, cross-validated error 3.7 pp. It is used for 7 estimates.
| Estimated model | AutomationBench | CyberGym | Source |
|---|---|---|---|
| Claude Fable 5.1 | 31.4% | 80.0% | estimated ± 3.7 pp, high confidence |
| Gemini 3.7 Flash | 30.4% | 79.6% | estimated ± 3.7 pp, high confidence |
| Gemini 4 Argon | 51.3% | 88.8% | estimated ± 3.7 pp, high confidence |
| GPT-6.1 Sol | 36.1% | 82.1% | estimated ± 3.7 pp, high confidence |
| GPT-6 Sol | 33.2% | 80.8% | estimated ± 3.7 pp, high confidence |
| Kimi K3 | 30.8% | 79.7% | estimated ± 3.7 pp, high confidence |
| Muse Spark 1.3 | 49.4% | 88.0% | estimated ± 3.7 pp, high confidence |