Calibration
CyberGym → HLE w/ tools
HLE w/ tools is estimated from CyberGym with a Hill curve fitted on 5 models measured on both: y = 0.0000 + (0.8911 − 0.0000)·x^6.00 / (0.74203^6.00 + x^6.00), R² = 0.86, cross-validated error 5.0 pp. It is used for 10 estimates.
| Estimated model | CyberGym | HLE w/ tools | Source |
|---|---|---|---|
| Claude Mythos Preview | 83.1% | 59.1% | estimated ± 5.0 pp, low confidence |
| Claude Sonnet 4.6 | 65.2% | 28.1% | estimated ± 5.0 pp, low confidence |
| Gemini 3.5 Flash Cyber | 83.2% | 59.3% | estimated ± 5.0 pp, low confidence |
| Gemini 3.8 Flash Cyber | 86.2% | 63.3% | estimated ± 5.0 pp, low confidence |
| Ling 3.1 Flash | 87.9% | 65.4% | estimated ± 5.0 pp, low confidence |
| MiMo-V2.6-Flash | 95.1% | 72.7% | estimated ± 5.0 pp, low confidence |
| MiMo-V2.6-Pro | 94.0% | 71.7% | estimated ± 5.0 pp, low confidence |
| Muse Spark | 43.5% | 3.5% | estimated ± 5.0 pp, low confidence |
| Muse Spark 1.1 | 59.0% | 18.0% | estimated ± 5.0 pp, low confidence |
| Fugu Cyber | 86.9% | 64.2% | estimated ± 5.0 pp, low confidence |