benchgap
Calibration

Terminal-Bench 4.1 → Terminal-Bench 2.1 (Vals AI)

Terminal-Bench 2.1 (Vals AI) is estimated from Terminal-Bench 4.1 with a Michaelis–Menten + offset curve fitted on 32 models measured on both: y = 0.5020 + 0.3353·x / (0.07787 + x), R² = 0.52, cross-validated error 8.3 pp. It is used for 2 estimates.

Estimated modelTerminal-Bench 4.1Terminal-Bench 2.1 (Vals AI)Source
GPT-6.1 Sol60.9%79.9%estimated ± 8.3 pp, medium confidence
Hy437.9%78.0%estimated ± 8.3 pp, medium confidence

Fitted on

ModelTerminal-Bench 4.1Terminal-Bench 2.1 (Vals AI)
Claude Fable 544.9%80.5%
Claude Fable 5.157.6%85.0%
Claude Opus 4.823.6%71.9%
Claude Opus 553.3%84.6%
Claude Opus 5.565.8%87.6%
Claude Sonnet 512.1%74.5%
Claude Sonnet 5.563.9%83.1%
DeepSeek V4.1 Flash 101738.5%74.5%
DeepSeek V4 Pro6.7%54.7%
Gemini 3.1 Pro2.7%70.8%
Gemini 3.7 Flash12.4%77.5%
Gemini 3.8 Flash18.2%81.3%
Gemini 4 Argon55.5%57.7%
GLM-5.341.2%71.5%
GLM-5.3-Flash37.0%62.9%
GPT-5.4 mini1.8%54.7%
GPT-5.6 Luna17.6%79.0%
GPT-5.6 Sol38.5%85.8%
GPT-5.6 Terra21.5%77.5%
GPT-6 Astra60.3%87.3%
GPT-6 Luna15.2%73.0%
GPT-6 Sol50.3%83.1%
Grok 4.512.4%67.8%
Grok 4.619.7%78.3%
Grok 4.738.8%73.4%
Inkling0.0%47.6%
Inkling-Small0.9%55.1%
Kimi K323.0%80.9%
MiMo-V2.6-Flash16.1%76.4%
MiMo-V2.6-Pro27.0%67.8%
MiniMax M37.6%53.6%
Muse Spark 1.314.9%72.3%