Calibration
CyberGym → Toolathlon-Verified
Toolathlon-Verified is estimated from CyberGym with a Michaelis–Menten curve fitted on 7 models measured on both: y = 0.9350·x / (0.22751 + x), R² = 0.43, cross-validated error 2.2 pp. It is used for 10 estimates.
| Estimated model | CyberGym | Toolathlon-Verified | Source |
|---|---|---|---|
| DeepSeek V4.1 Flash 1017 | 88.1% | 74.3% | estimated ± 2.2 pp, low confidence |
| Fugu Cyber | 86.9% | 74.1% | estimated ± 2.2 pp, low confidence |
| Atria Dawn Preview | 86.5% | 74.0% | estimated ± 2.2 pp, low confidence |
| Gemini 3.8 Flash Cyber | 86.2% | 74.0% | estimated ± 2.2 pp, low confidence |
| Claude Mythos 5 | 83.8% | 73.5% | estimated ± 2.2 pp, low confidence |
| Gemini 3.5 Flash Cyber | 83.2% | 73.4% | estimated ± 2.2 pp, low confidence |
| Claude Mythos Preview | 83.1% | 73.4% | estimated ± 2.2 pp, low confidence |
| GPT-5.4 | 79.0% | 72.6% | estimated ± 2.2 pp, low confidence |
| GLM-5.1 | 68.7% | 70.2% | estimated ± 2.2 pp, low confidence |
| Claude Opus 4.6 | 66.6% | 69.7% | estimated ± 2.2 pp, low confidence |
Fitted on
| Model | CyberGym | Toolathlon-Verified |
|---|---|---|
| DeepSeek V4 Flash 0731 | 76.7% | 70.3% |
| DeepSeek V4 Pro | 83.3% | 74.1% |
| GLM-5.3 | 84.5% | 73.0% |
| Hy4 preview | 78.4% | 74.1% |
| MiMo-V2.6-Flash | 95.1% | 73.6% |
| MiMo-V2.6-Pro | 94.0% | 76.9% |
| Step 5 Preview | 84.7% | 74.1% |