Calibration
CyberGym → JobBench
JobBench is estimated from CyberGym with a Michaelis–Menten curve fitted on 12 models measured on both: y = 2.0000·x / (2.35768 + x), R² = 0.44, cross-validated error 8.6 pp. It is used for 8 estimates.
| Estimated model | CyberGym | JobBench | Source |
|---|---|---|---|
| DeepSeek V4.1 Flash 1017 | 88.1% | 54.4% | estimated ± 8.6 pp, low confidence |
| Fugu Cyber | 86.9% | 53.9% | estimated ± 8.6 pp, low confidence |
| Gemini 3.8 Flash Cyber | 86.2% | 53.5% | estimated ± 8.6 pp, low confidence |
| GLM-5.3 | 84.5% | 52.8% | estimated ± 8.6 pp, low confidence |
| Claude Mythos 5 | 83.8% | 52.4% | estimated ± 8.6 pp, low confidence |
| Gemini 3.5 Flash Cyber | 83.2% | 52.2% | estimated ± 8.6 pp, low confidence |
| Claude Mythos Preview | 83.1% | 52.1% | estimated ± 8.6 pp, low confidence |
| Muse Spark | 43.5% | 31.2% | estimated ± 8.6 pp, low confidence |
Fitted on
| Model | CyberGym | JobBench |
|---|---|---|
| Atria Dawn Preview | 86.5% | 50.3% |
| Claude Opus 4.5 | 50.6% | 32.3% |
| Claude Opus 4.6 | 66.6% | 36.7% |
| Claude Opus 4.7 | 73.1% | 45.9% |
| Claude Sonnet 4.6 | 65.2% | 36.9% |
| GPT-5.4 | 79.0% | 38.9% |
| GPT-5.5 | 81.8% | 42.7% |
| Hy4 preview | 78.4% | 61.7% |
| MiMo-V2.6-Flash | 95.1% | 61.2% |
| MiMo-V2.6-Pro | 94.0% | 62.0% |
| Muse Spark 1.1 | 59.0% | 54.7% |
| Step 5 Preview | 84.7% | 59.0% |