benchgap
Calibration

Agents' Last Exam → Hermes Bench

Hermes Bench is estimated from Agents' Last Exam with a Hill curve fitted on 7 models measured on both: y = 0.7002 + (1.2000 − 0.7002)·x^6.00 / (0.90892^6.00 + x^6.00), R² = 0.63, cross-validated error 1.5 pp. It is used for 9 estimates.

Estimated modelAgents' Last ExamHermes BenchSource
GPT-6.1 Terra60.0%73.8%estimated ± 1.5 pp, low confidence
Qwen3.8-Flash-Next51.2%71.6%estimated ± 1.5 pp, medium confidence
Qwen3.8-27B42.9%70.6%estimated ± 1.5 pp, medium confidence
Hy441.0%70.4%estimated ± 1.5 pp, medium confidence
LongCat-2.538.5%70.3%estimated ± 1.5 pp, medium confidence
MiniMax M3.1 Flash29.6%70.1%estimated ± 1.5 pp, medium confidence
Gemini 3.7 Flash26.3%70.1%estimated ± 1.5 pp, medium confidence
DeepSeek V4 Pro25.7%70.1%estimated ± 1.5 pp, medium confidence
DeepSeek V4 Flash 073125.2%70.0%estimated ± 1.5 pp, medium confidence

Fitted on

ModelAgents' Last ExamHermes Bench
GLM-5.328.5%71.8%
GLM-5.3-Flash26.3%68.9%
GPT-6 Astra59.3%73.7%
GPT-6 Sol56.4%73.5%
Hy4 preview22.8%70.8%
Qwen3.8 Max52.4%70.5%
Step 5 Preview29.5%69.2%