Calibration
Agents' Last Exam → Hermes Bench
Hermes Bench is estimated from Agents' Last Exam with a Hill curve fitted on 7 models measured on both: y = 0.7002 + (1.2000 − 0.7002)·x^6.00 / (0.90892^6.00 + x^6.00), R² = 0.63, cross-validated error 1.5 pp. It is used for 9 estimates.
| Estimated model | Agents' Last Exam | Hermes Bench | Source |
|---|---|---|---|
| GPT-6.1 Terra | 60.0% | 73.8% | estimated ± 1.5 pp, low confidence |
| Qwen3.8-Flash-Next | 51.2% | 71.6% | estimated ± 1.5 pp, medium confidence |
| Qwen3.8-27B | 42.9% | 70.6% | estimated ± 1.5 pp, medium confidence |
| Hy4 | 41.0% | 70.4% | estimated ± 1.5 pp, medium confidence |
| LongCat-2.5 | 38.5% | 70.3% | estimated ± 1.5 pp, medium confidence |
| MiniMax M3.1 Flash | 29.6% | 70.1% | estimated ± 1.5 pp, medium confidence |
| Gemini 3.7 Flash | 26.3% | 70.1% | estimated ± 1.5 pp, medium confidence |
| DeepSeek V4 Pro | 25.7% | 70.1% | estimated ± 1.5 pp, medium confidence |
| DeepSeek V4 Flash 0731 | 25.2% | 70.0% | estimated ± 1.5 pp, medium confidence |
Fitted on
| Model | Agents' Last Exam | Hermes Bench |
|---|---|---|
| GLM-5.3 | 28.5% | 71.8% |
| GLM-5.3-Flash | 26.3% | 68.9% |
| GPT-6 Astra | 59.3% | 73.7% |
| GPT-6 Sol | 56.4% | 73.5% |
| Hy4 preview | 22.8% | 70.8% |
| Qwen3.8 Max | 52.4% | 70.5% |
| Step 5 Preview | 29.5% | 69.2% |