benchgap
Calibration

CyberGym → JobBench

JobBench is estimated from CyberGym with a Michaelis–Menten curve fitted on 12 models measured on both: y = 2.0000·x / (2.35768 + x), R² = 0.44, cross-validated error 8.6 pp. It is used for 8 estimates.

Estimated modelCyberGymJobBenchSource
DeepSeek V4.1 Flash 101788.1%54.4%estimated ± 8.6 pp, low confidence
Fugu Cyber86.9%53.9%estimated ± 8.6 pp, low confidence
Gemini 3.8 Flash Cyber86.2%53.5%estimated ± 8.6 pp, low confidence
GLM-5.384.5%52.8%estimated ± 8.6 pp, low confidence
Claude Mythos 583.8%52.4%estimated ± 8.6 pp, low confidence
Gemini 3.5 Flash Cyber83.2%52.2%estimated ± 8.6 pp, low confidence
Claude Mythos Preview83.1%52.1%estimated ± 8.6 pp, low confidence
Muse Spark43.5%31.2%estimated ± 8.6 pp, low confidence

Fitted on

ModelCyberGymJobBench
Atria Dawn Preview86.5%50.3%
Claude Opus 4.550.6%32.3%
Claude Opus 4.666.6%36.7%
Claude Opus 4.773.1%45.9%
Claude Sonnet 4.665.2%36.9%
GPT-5.479.0%38.9%
GPT-5.581.8%42.7%
Hy4 preview78.4%61.7%
MiMo-V2.6-Flash95.1%61.2%
MiMo-V2.6-Pro94.0%62.0%
Muse Spark 1.159.0%54.7%
Step 5 Preview84.7%59.0%