benchgap
Calibration

CyberGym → Toolathlon-Verified

Toolathlon-Verified is estimated from CyberGym with a Michaelis–Menten curve fitted on 7 models measured on both: y = 0.9350·x / (0.22751 + x), R² = 0.43, cross-validated error 2.2 pp. It is used for 10 estimates.

Estimated modelCyberGymToolathlon-VerifiedSource
DeepSeek V4.1 Flash 101788.1%74.3%estimated ± 2.2 pp, low confidence
Fugu Cyber86.9%74.1%estimated ± 2.2 pp, low confidence
Atria Dawn Preview86.5%74.0%estimated ± 2.2 pp, low confidence
Gemini 3.8 Flash Cyber86.2%74.0%estimated ± 2.2 pp, low confidence
Claude Mythos 583.8%73.5%estimated ± 2.2 pp, low confidence
Gemini 3.5 Flash Cyber83.2%73.4%estimated ± 2.2 pp, low confidence
Claude Mythos Preview83.1%73.4%estimated ± 2.2 pp, low confidence
GPT-5.479.0%72.6%estimated ± 2.2 pp, low confidence
GLM-5.168.7%70.2%estimated ± 2.2 pp, low confidence
Claude Opus 4.666.6%69.7%estimated ± 2.2 pp, low confidence

Fitted on

ModelCyberGymToolathlon-Verified
DeepSeek V4 Flash 073176.7%70.3%
DeepSeek V4 Pro83.3%74.1%
GLM-5.384.5%73.0%
Hy4 preview78.4%74.1%
MiMo-V2.6-Flash95.1%73.6%
MiMo-V2.6-Pro94.0%76.9%
Step 5 Preview84.7%74.1%