Agentic · tools

AA AutomationBench leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

As of 2026-10-07, the highest measured score on AA AutomationBench is 77.5% by Gemini 4 Argon. 135 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	Gemini 4 Argon	77.5%	measured
2	Claude Sonnet 5.5	71.8%	measured
3	Claude Opus 5.5	69.5%	measured
4	DeepSeek V4.1 Flash	68.9%	measured
5	GPT-6 Astra	68.5%	measured
6	Grok 4.6	66.7%	measured
7	GPT-5.6 Sol	65.6%	estimated ± 5.5 pp, low confidence
8	Grok 4.7	65.6%	measured
9	GPT-6.1 Sol	64.9%	measured
10	Qwen3.8 Max Preview	64.7%	estimated ± 7.0 pp, medium confidence
11	Ling 3.1 Flash	64.4%	estimated ± 7.0 pp, medium confidence
12	Claude Fable 5	64.3%	estimated ± 7.0 pp, medium confidence
13	GLM-5.2	64.3%	estimated ± 5.5 pp, low confidence
14	GPT-5.6 Terra	62.7%	estimated ± 5.5 pp, low confidence
15	GLM-5.3	62.2%	measured
16	GPT-6 Sol	61.6%	measured
17	Claude Opus 5	61.3%	estimated ± 5.7 pp, low confidence
18	Claude Mythos Preview	61.1%	estimated ± 5.5 pp, low confidence
19	GLM-5.3-Flash	60.4%	measured
20	Muse Spark 1.2	60.0%	estimated ± 7.0 pp, medium confidence
21	Gemini 3.8 Flash	59.9%	measured
22	Mistral Large 4	59.9%	measured
23	GPT-5.6 Luna	59.6%	estimated ± 5.5 pp, low confidence
24	Claude Fable 5.1	59.4%	measured
25	MiMo-V2.6-Pro	58.6%	measured
26	Kimi K3	58.3%	measured
27	GPT-5.5 Pro	58.0%	estimated ± 5.7 pp, low confidence
28	Muse Spark 1.3	57.9%	measured
29	Hy4 preview	56.8%	estimated ± 5.0 pp, medium confidence
30	MiMo-V2.6-Flash	56.7%	estimated ± 5.0 pp, medium confidence
31	Qwen3.8-Flash-Next	55.2%	estimated ± 5.0 pp, medium confidence
32	Muse Spark 1.1	54.9%	estimated ± 5.0 pp, medium confidence
33	Qwen3.8 Max	54.5%	estimated ± 5.0 pp, medium confidence
34	GPT-5.4 Pro	53.9%	estimated ± 5.7 pp, low confidence
35	Atria Dawn Preview	53.6%	estimated ± 5.0 pp, medium confidence
36	GPT-6 Luna	53.2%	measured
37	Claude Opus 4.7 (Adaptive)	52.4%	estimated ± 5.0 pp, medium confidence
38	GPT-5.5	51.5%	estimated ± 5.0 pp, medium confidence
39	Step 5 Preview	51.0%	measured
40	GPT-5.4	50.4%	estimated ± 5.0 pp, medium confidence
41	Claude Sonnet 4.6	49.9%	estimated ± 5.0 pp, medium confidence
42	Claude Opus 4.6	49.8%	estimated ± 5.0 pp, medium confidence
43	Gemini 3.7 Flash	49.7%	estimated ± 7.0 pp, medium confidence
44	Grok 4.5	49.3%	estimated ± 7.0 pp, medium confidence
45	GPT-5.2	49.2%	estimated ± 5.0 pp, medium confidence
46	GPT-5.3 Codex	49.0%	estimated ± 5.0 pp, medium confidence
47	Claude Opus 4.5	48.6%	estimated ± 5.0 pp, low confidence
48	Qwen3.8-27B	48.2%	measured
49	Claude Sonnet 4.5	47.3%	estimated ± 5.0 pp, low confidence
50	GPT-5.1-Codex	46.9%	estimated ± 5.0 pp, low confidence
51	GPT-5.2-Codex	46.8%	estimated ± 5.0 pp, low confidence
52	Claude Mythos 5	46.4%	estimated ± 5.7 pp, low confidence
53	Claude 4.1 Opus	45.7%	estimated ± 5.0 pp, low confidence
54	Qwen3.5 Plus	44.7%	estimated ± 5.0 pp, low confidence
55	Claude 4 Sonnet	44.7%	estimated ± 5.0 pp, low confidence
56	Claude Haiku 4.5	44.0%	estimated ± 5.0 pp, low confidence
57	Gemini 3 Flash	42.7%	estimated ± 5.0 pp, low confidence
58	Gemini 3 Pro	42.7%	estimated ± 5.0 pp, low confidence
59	Kimi K2.5	42.0%	estimated ± 5.0 pp, low confidence
60	GPT-5 (high)	41.9%	estimated ± 5.0 pp, low confidence
61	Claude Opus 4.7	41.7%	estimated ± 7.0 pp, medium confidence
62	Gemini 3.5 Flash	39.8%	estimated ± 7.0 pp, medium confidence
63	Ornith-1.5-397B	37.9%	estimated ± 5.7 pp, low confidence
64	Claude Haiku 5.5	35.4%	measured
65	Claude Sonnet 5	26.9%	estimated ± 5.7 pp, low confidence
66	Gemini 3.6 Flash	26.2%	estimated ± 7.0 pp, medium confidence
67	Claude Opus 4.8	24.8%	estimated ± 5.7 pp, low confidence
68	MiniMax M3	21.3%	measured
69	DeepSeek V4 Pro 0813	20.6%	estimated ± 5.7 pp, low confidence
70	dots3-note Preview	20.2%	estimated ± 5.7 pp, low confidence
71	Kimi K2.6	19.7%	estimated ± 5.7 pp, low confidence
72	Hy3 Preview	13.7%	estimated ± 7.0 pp, medium confidence
73	Apodex 1.1	11.4%	estimated ± 7.0 pp, medium confidence
74	Apodex 1.1 Mini	11.4%	estimated ± 7.0 pp, medium confidence
75	Quasar 438B	10.2%	estimated ± 7.0 pp, medium confidence
76	Ling 3.0 Flash VL	8.6%	estimated ± 7.0 pp, medium confidence
77	Qwen3.7 Max	6.8%	estimated ± 7.0 pp, medium confidence
78	Muse Glimmer 30B	6.8%	measured
79	MiMo-V2.5-Pro	6.5%	estimated ± 7.0 pp, medium confidence
80	Inkling-Small	5.9%	estimated ± 5.7 pp, low confidence
81	Beam	5.9%	estimated ± 5.7 pp, low confidence
82	Grok 4.3	5.2%	estimated ± 7.0 pp, medium confidence
83	Inkling	5.0%	measured
84	Hy3	4.9%	estimated ± 7.0 pp, medium confidence
85	Step 3.7 Flash	4.6%	estimated ± 5.7 pp, low confidence
86	Kimi K2.7 Code	4.5%	estimated ± 7.0 pp, medium confidence
87	Agents-A1	4.4%	estimated ± 5.7 pp, low confidence
88	GPT-5.4 mini	4.3%	estimated ± 7.0 pp, medium confidence
89	MiniMax M2.7	4.3%	estimated ± 7.0 pp, medium confidence
90	Muse Spark	4.2%	estimated ± 7.0 pp, medium confidence
91	Qwen3.6 Plus	4.1%	estimated ± 7.0 pp, medium confidence
92	Qwen3.6-27B	4.1%	estimated ± 7.0 pp, medium confidence
93	Gemini 3.5 Flash-Lite	4.1%	estimated ± 7.0 pp, medium confidence
94	A.X K2	4.0%	estimated ± 7.0 pp, medium confidence
95	GPT-5.4 nano	3.9%	estimated ± 7.0 pp, medium confidence
96	Ling 3.0 Flash FP8	3.9%	estimated ± 7.0 pp, medium confidence
97	Qwen3.6-35B-A3B	3.8%	estimated ± 7.0 pp, medium confidence
98	GPT-5.1	3.8%	estimated ± 7.0 pp, medium confidence
99	Gemini 3.1 Pro	3.8%	estimated ± 7.0 pp, medium confidence
100	Qwen3.7 Plus	3.8%	estimated ± 7.0 pp, low confidence
101	Mistral Medium 3.5 128B	3.8%	estimated ± 7.0 pp, low confidence
102	MiniCPM5-2B	3.7%	estimated ± 7.0 pp, low confidence
103	MiMo-V2-Flash	3.7%	estimated ± 7.0 pp, low confidence
104	Gemma 4 31B	3.7%	estimated ± 7.0 pp, low confidence
105	GPT-OSS 120B	3.7%	estimated ± 7.0 pp, low confidence
106	Granite 4.2 30B	3.7%	estimated ± 7.0 pp, low confidence
107	Gemma 4 26B A4B	3.7%	estimated ± 7.0 pp, low confidence
108	Ling 3.0 Tiny	3.7%	estimated ± 7.0 pp, low confidence
109	Celeris-1	3.7%	estimated ± 7.0 pp, low confidence
110	Command A+	3.7%	estimated ± 7.0 pp, low confidence
111	DeepSeek V3	3.7%	estimated ± 7.0 pp, low confidence
112	DeepSeek V3 0324	3.7%	estimated ± 7.0 pp, low confidence
113	Gemini 2.5 Pro	3.7%	estimated ± 7.0 pp, low confidence
114	Gemma 3 27B	3.7%	estimated ± 7.0 pp, low confidence
115	Gemma 4 12B	3.7%	estimated ± 7.0 pp, low confidence
116	Gemma 4 E2B	3.7%	estimated ± 7.0 pp, low confidence
117	Gemma 4 E4B	3.7%	estimated ± 7.0 pp, low confidence
118	GPT-4.1 mini	3.7%	estimated ± 7.0 pp, low confidence
119	GPT-4.1 nano	3.7%	estimated ± 7.0 pp, low confidence
120	GPT-4o	3.7%	estimated ± 7.0 pp, low confidence
121	GPT-4o mini	3.7%	estimated ± 7.0 pp, low confidence
122	GPT-OSS 20B	3.7%	estimated ± 7.0 pp, low confidence
123	Granite 4.2 3B	3.7%	estimated ± 7.0 pp, low confidence
124	Granite 4.2 8B	3.7%	estimated ± 7.0 pp, low confidence
125	K-Exaone	3.7%	estimated ± 7.0 pp, low confidence
126	LFM2.5-2.6B	3.7%	estimated ± 7.0 pp, low confidence
127	Ling 2.6 Flash	3.7%	estimated ± 7.0 pp, low confidence
128	Llama 4 Maverick	3.7%	estimated ± 7.0 pp, low confidence
129	Llama 4 Scout	3.7%	estimated ± 7.0 pp, low confidence
130	Mercury 2.5	3.7%	estimated ± 7.0 pp, low confidence
131	Mistral Large 3	3.7%	estimated ± 7.0 pp, low confidence
132	Mistral Small 4	3.7%	estimated ± 7.0 pp, low confidence
133	Mistral Small 4 (Reasoning)	3.7%	estimated ± 7.0 pp, low confidence
134	Nemotron 3 Nano 30B	3.7%	estimated ± 7.0 pp, low confidence
135	Nemotron 3 Nano Omni 30B A3B	3.7%	estimated ± 7.0 pp, low confidence
136	Nemotron 3 Super 100B	3.7%	estimated ± 7.0 pp, low confidence
137	North Mini Code	3.7%	estimated ± 7.0 pp, low confidence
138	Solar Pro 3	3.7%	estimated ± 7.0 pp, low confidence
139	Trinity-Large-Preview	3.7%	estimated ± 7.0 pp, low confidence
140	Trinity-Large-Thinking	3.7%	estimated ± 7.0 pp, low confidence
141	Ultravox v0.6 Llama 3.3 70B	3.7%	estimated ± 7.0 pp, low confidence
142	DeepSeek V4 Flash 0731	3.5%	estimated ± 5.7 pp, low confidence
143	Ling 3.0 Flash	3.2%	estimated ± 5.7 pp, low confidence
144	Nemotron 3 Ultra	3.0%	measured
145	GLM-5.1	2.7%	estimated ± 5.7 pp, low confidence
146	Ornith-1.5-35B-A3B	2.7%	estimated ± 5.7 pp, low confidence
147	Agents-A1-4B	2.7%	estimated ± 5.7 pp, low confidence
148	Qwen3.5-122B-A10B	2.6%	estimated ± 5.7 pp, low confidence
149	Qwen3.5 397B	2.6%	estimated ± 5.7 pp, low confidence
150	Qwen3.5-27B	2.6%	estimated ± 5.7 pp, low confidence
151	Qwen3.5-35B-A3B	2.6%	estimated ± 5.7 pp, low confidence
152	Kimi K2.5 (Reasoning)	2.6%	estimated ± 5.7 pp, low confidence
153	Ornith-1.5-9B	2.6%	estimated ± 5.7 pp, low confidence
154	GLM-4.7	2.6%	estimated ± 5.7 pp, low confidence
155	Solar Pro 4	2.6%	estimated ± 5.7 pp, low confidence
156	LongCat-Flash-Lite-Sparse	2.6%	estimated ± 5.7 pp, low confidence
157	Nemotron 3.5 Lightning 30B A3B NVFP4	2.6%	estimated ± 5.7 pp, low confidence
158	GLM-5	0.0%	estimated ± 13.0 pp, low confidence
159	LLaDA2.2-flash	0.0%	estimated ± 13.0 pp, low confidence
160	Solar Open 2	0.0%	estimated ± 13.0 pp, low confidence

AA AutomationBench leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General