Calibration
CyberGym → OSWorld-Verified
OSWorld-Verified is estimated from CyberGym with a Michaelis–Menten curve fitted on 10 models measured on both: y = 1.0369·x / (0.25060 + x), R² = 0.53, cross-validated error 4.7 pp. It is used for 15 estimates.
| Estimated model | CyberGym | OSWorld-Verified | Source |
|---|---|---|---|
| Atria Dawn Preview | 86.5% | 80.4% | estimated ± 4.7 pp, high confidence |
| Claude Mythos Preview | 83.1% | 79.7% | estimated ± 4.7 pp, high confidence |
| DeepSeek V4.1 Flash | 88.1% | 80.7% | estimated ± 4.7 pp, high confidence |
| DeepSeek V4 Flash 0731 | 76.7% | 78.2% | estimated ± 4.7 pp, high confidence |
| DeepSeek V4 Pro 0813 | 83.3% | 79.7% | estimated ± 4.7 pp, high confidence |
| Gemini 3.5 Flash Cyber | 83.2% | 79.7% | estimated ± 4.7 pp, high confidence |
| Gemini 3.8 Flash Cyber | 86.2% | 80.3% | estimated ± 4.7 pp, high confidence |
| GLM-5 | 43.2% | 65.6% | estimated ± 4.7 pp, medium confidence |
| GLM-5.1 | 68.7% | 76.0% | estimated ± 4.7 pp, high confidence |
| GLM-5.3 | 84.5% | 80.0% | estimated ± 4.7 pp, high confidence |
| Hy4 preview | 78.4% | 78.6% | estimated ± 4.7 pp, high confidence |
| Ling 3.1 Flash | 87.9% | 80.7% | estimated ± 4.7 pp, high confidence |
| Muse Spark | 43.5% | 65.8% | estimated ± 4.7 pp, medium confidence |
| Fugu Cyber | 86.9% | 80.5% | estimated ± 4.7 pp, high confidence |
| Step 5 Preview | 84.7% | 80.0% | estimated ± 4.7 pp, high confidence |