benchgap
Calibration

ExploitGym → Hermes Index

Hermes Index is estimated from ExploitGym with a Michaelis–Menten curve fitted on 6 models measured on both: y = 0.8648·x / (0.18140 + x), R² = 0.87, cross-validated error 5.5 pp. It is used for 7 estimates.

Estimated modelExploitGymHermes IndexSource
GPT-6 Cyber48.5%62.9%estimated ± 5.5 pp, low confidence
Claude Mythos 5.238.1%58.6%estimated ± 5.5 pp, low confidence
GPT-5.6 Cyber32.1%55.3%estimated ± 5.5 pp, low confidence
Claude Mythos 5.123.4%48.7%estimated ± 5.5 pp, low confidence
Claude Mythos Preview17.5%42.5%estimated ± 5.5 pp, low confidence
MiMo-V2.6-Flash6.0%21.5%estimated ± 5.5 pp, low confidence
Muse Spark 1.10.8%3.7%estimated ± 5.5 pp, low confidence

Fitted on

ModelExploitGymHermes Index
Claude Opus 5.533.3%63.3%
GLM-5.315.0%39.2%
GPT-6.1 Sol35.1%57.1%
GPT-6 Astra42.4%56.3%
GPT-6 Luna11.6%33.9%
GPT-6 Sol22.1%44.1%