benchgap
Calibration

AA Agentic Index → MCP Atlas

MCP Atlas is estimated from AA Agentic Index with a offset logistic curve fitted on 23 models measured on both: y = 0.6456 + (0.7870 − 0.6456) / (1 + exp(−153.26·(x − 0.2328))), R² = 0.49, cross-validated error 7.8 pp. It is used for 30 estimates.

Estimated modelAA Agentic IndexMCP AtlasSource
Qwen3.8 Max Preview49.6%78.7%estimated ± 7.8 pp, low confidence
Muse Spark 1.244.0%78.7%estimated ± 7.8 pp, low confidence
Quasar 438B32.7%78.7%estimated ± 7.8 pp, low confidence
Gemini 3.6 Flash30.2%78.7%estimated ± 7.8 pp, low confidence
Hy325.6%78.3%estimated ± 7.8 pp, low confidence
Hy3 Preview25.6%78.3%estimated ± 7.8 pp, low confidence
Nemotron 3 Ultra21.7%65.7%estimated ± 7.8 pp, low confidence
Qwen3.6-27B20.1%64.7%estimated ± 7.8 pp, low confidence
Grok 4.317.2%64.6%estimated ± 7.8 pp, low confidence
MiniMax M2.716.8%64.6%estimated ± 7.8 pp, low confidence
Gemini 3.5 Flash-Lite15.9%64.6%estimated ± 7.8 pp, low confidence
Gemini 3.1 Pro10.3%64.6%estimated ± 7.8 pp, low confidence
Qwen3.5-122B-A10B9.6%64.6%estimated ± 7.8 pp, low confidence
Mistral Medium 3.5 128B9.4%64.6%estimated ± 7.8 pp, low confidence
GPT-OSS 120B6.2%64.6%estimated ± 7.8 pp, low confidence
Nemotron 3.5 Lightning 30B A3B NVFP46.1%64.6%estimated ± 7.8 pp, low confidence
Granite 4.2 8B3.7%64.6%estimated ± 7.8 pp, low confidence
Command A+3.6%64.6%estimated ± 7.8 pp, low confidence
Gemini 2.5 Pro3.6%64.6%estimated ± 7.8 pp, low confidence
Mistral Large 32.4%64.6%estimated ± 7.8 pp, low confidence
Mistral Small 41.4%64.6%estimated ± 7.8 pp, low confidence
GPT-OSS 20B1.4%64.6%estimated ± 7.8 pp, low confidence
Nemotron 3 Nano 30B1.0%64.6%estimated ± 7.8 pp, low confidence
Trinity-Large-Preview1.2%64.6%estimated ± 7.8 pp, low confidence
Trinity-Large-Thinking1.2%64.6%estimated ± 7.8 pp, low confidence
Celeris-10.7%64.6%estimated ± 7.8 pp, low confidence
DeepSeek V30.8%64.6%estimated ± 7.8 pp, low confidence
Gemma 3 27B0.1%64.6%estimated ± 7.8 pp, low confidence
Llama 4 Maverick0.6%64.6%estimated ± 7.8 pp, low confidence
Llama 4 Scout0.5%64.6%estimated ± 7.8 pp, low confidence

Fitted on

ModelAA Agentic IndexMCP Atlas
Claude Opus 4.739.5%77.3%
Claude Opus 4.842.6%82.2%
Claude Opus 556.2%85.8%
DeepSeek V4 Flash 073141.7%69.0%
DeepSeek V4 Pro49.6%73.6%
Gemini 3.5 Flash27.3%83.6%
GLM-5.125.2%71.8%
GLM-5.239.4%76.8%
GPT-5.4 mini19.7%57.7%
GPT-5.4 nano17.7%56.1%
GPT-5.537.3%75.3%
Inkling24.3%74.1%
Inkling-Small25.0%79.6%
Kimi K2.622.1%55.9%
Kimi K2.7 Code22.5%76.0%
Kimi K350.6%84.2%
Ling 3.0 Flash21.0%65.5%
MiniMax M330.8%74.2%
Muse Glimmer 30B10.5%75.5%
Muse Spark 1.127.5%88.1%
Qwen3.6-35B-A3B15.0%62.8%
Qwen3.7 Max23.9%76.4%
Qwen3.7 Plus19.7%73.2%