Calibration
CyberGym → MCP Atlas
MCP Atlas is estimated from CyberGym with a Hill curve fitted on 12 models measured on both: y = 0.0000 + (0.8126 − 0.0000)·x^6.00 / (0.46036^6.00 + x^6.00), R² = 0.75, cross-validated error 11.1 pp. It is used for 4 estimates.
| Estimated model | CyberGym | MCP Atlas | Source |
|---|---|---|---|
| Fugu Cyber | 86.9% | 79.5% | estimated ± 11.1 pp, low confidence |
| Gemini 3.8 Flash Cyber | 86.2% | 79.4% | estimated ± 11.1 pp, low confidence |
| Gemini 3.5 Flash Cyber | 83.2% | 79.0% | estimated ± 11.1 pp, low confidence |
| Claude Mythos Preview | 83.1% | 79.0% | estimated ± 11.1 pp, low confidence |
Fitted on
| Model | CyberGym | MCP Atlas |
|---|---|---|
| Claude Opus 4.5 | 50.6% | 42.3% |
| Claude Opus 4.7 | 73.1% | 77.3% |
| DeepSeek V4 Flash 0731 | 76.7% | 69.0% |
| DeepSeek V4 Pro | 83.3% | 73.6% |
| GLM-5 | 43.2% | 31.1% |
| GLM-5.1 | 68.7% | 71.8% |
| GPT-5.4 | 79.0% | 70.6% |
| GPT-5.5 | 81.8% | 75.3% |
| Hy4 preview | 78.4% | 83.7% |
| Ling 3.1 Flash | 87.9% | 83.0% |
| Muse Spark 1.1 | 59.0% | 88.1% |
| Step 5 Preview | 84.7% | 85.6% |