Agentic · tools

AutomationBench leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

As of 2026-10-07, the highest measured score on AutomationBench is 54.8% by DeepSeek V4.1 Flash. 128 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	DeepSeek V4.1 Flash	54.8%	measured
2	Atria Dawn Preview	53.8%	measured
3	Nemotron 3 Ultra	53.7%	estimated ± 7.0 pp, low confidence
4	Fugu Cyber	53.2%	estimated ± 5.8 pp, medium confidence
5	MiMo-V2.6-Pro	53.1%	measured
6	Gemini 3.8 Flash Cyber	52.9%	estimated ± 5.8 pp, medium confidence
7	Ling 3.1 Flash	52.5%	measured
8	MiMo-V2.6-Flash	52.3%	measured
9	Gemini 4 Argon	51.3%	measured
10	MiniMax M3	50.6%	estimated ± 7.0 pp, low confidence
11	Muse Spark 1.3	49.4%	measured
12	GLM-5.3-Flash	48.8%	measured
13	GLM-5.3	48.2%	measured
14	Muse Glimmer 30B	48.1%	estimated ± 7.0 pp, low confidence
15	Mistral Large 4	45.5%	estimated ± 7.6 pp, low confidence
16	GPT-5.6 Sol	45.2%	estimated ± 5.8 pp, medium confidence
17	Inkling	44.9%	estimated ± 7.0 pp, low confidence
18	Qwen3.8 Max Preview	44.6%	estimated ± 9.7 pp, low confidence
19	Grok 4.7	44.6%	estimated ± 7.6 pp, low confidence
20	Qwen3.8-Flash-Next	44.5%	estimated ± 9.7 pp, low confidence
21	Step 5 Preview	44.0%	measured
22	Claude Haiku 5.5	44.0%	estimated ± 7.6 pp, low confidence
23	Gemini 3.8 Flash	43.8%	estimated ± 7.6 pp, low confidence
24	GPT-6 Luna	42.6%	estimated ± 7.6 pp, low confidence
25	GPT-6 Astra	41.4%	measured
26	Claude Sonnet 5.5	40.4%	estimated ± 7.6 pp, low confidence
27	Claude Opus 5.5	40.0%	measured
28	Qwen3.8-27B	38.9%	estimated ± 7.0 pp, low confidence
29	GPT-5.5 Pro	38.4%	estimated ± 10.3 pp, low confidence
30	GPT-5.4 Pro	37.5%	estimated ± 10.3 pp, low confidence
31	Claude Mythos 5	36.9%	estimated ± 5.8 pp, medium confidence
32	GPT-6.1 Sol	36.1%	measured
33	Grok 4.6	35.0%	estimated ± 7.0 pp, low confidence
34	Ornith-1.5-397B	34.8%	estimated ± 10.3 pp, low confidence
35	Gemini 3.5 Flash	33.2%	estimated ± 7.0 pp, low confidence
36	GPT-6 Sol	33.2%	measured
37	Claude Fable 5	32.3%	estimated ± 7.0 pp, low confidence
38	Hy4 preview	32.1%	measured
39	Gemini 3.5 Flash Cyber	31.8%	estimated ± 5.8 pp, medium confidence
40	DeepSeek V4 Pro 0813	31.8%	measured
41	dots3-note Preview	31.8%	estimated ± 10.3 pp, low confidence
42	Claude Fable 5.1	31.4%	measured
43	Claude Mythos Preview	31.3%	estimated ± 5.8 pp, medium confidence
44	Kimi K3	30.8%	measured
45	Gemini 3.7 Flash	30.4%	measured
46	Muse Spark 1.2	29.0%	estimated ± 9.7 pp, low confidence
47	Claude Sonnet 5	28.7%	estimated ± 9.7 pp, low confidence
48	GPT-5.5	28.7%	estimated ± 5.8 pp, medium confidence
49	GPT-5.6 Terra	28.7%	estimated ± 5.8 pp, medium confidence
50	Claude Opus 4.8	28.6%	estimated ± 9.7 pp, low confidence
51	Grok 4.5	28.4%	estimated ± 9.7 pp, low confidence
52	GLM-5.2	28.4%	estimated ± 9.7 pp, low confidence
53	Gemini 3.6 Flash	28.4%	estimated ± 9.7 pp, low confidence
54	A.X K2	28.4%	estimated ± 9.7 pp, low confidence
55	Apodex 1.1	28.4%	estimated ± 9.7 pp, low confidence
56	Apodex 1.1 Mini	28.4%	estimated ± 9.7 pp, low confidence
57	Celeris-1	28.4%	estimated ± 9.7 pp, low confidence
58	Command A+	28.4%	estimated ± 9.7 pp, low confidence
59	DeepSeek V3	28.4%	estimated ± 9.7 pp, low confidence
60	DeepSeek V3 0324	28.4%	estimated ± 9.7 pp, low confidence
61	Gemini 2.5 Pro	28.4%	estimated ± 9.7 pp, low confidence
62	Gemini 3.1 Pro	28.4%	estimated ± 9.7 pp, low confidence
63	Gemini 3.5 Flash-Lite	28.4%	estimated ± 9.7 pp, low confidence
64	Gemma 3 27B	28.4%	estimated ± 9.7 pp, low confidence
65	Gemma 4 12B	28.4%	estimated ± 9.7 pp, low confidence
66	Gemma 4 26B A4B	28.4%	estimated ± 9.7 pp, low confidence
67	Gemma 4 31B	28.4%	estimated ± 9.7 pp, low confidence
68	Gemma 4 E2B	28.4%	estimated ± 9.7 pp, low confidence
69	Gemma 4 E4B	28.4%	estimated ± 9.7 pp, low confidence
70	GLM-4.7	28.4%	estimated ± 9.7 pp, low confidence
71	GPT-4.1 mini	28.4%	estimated ± 9.7 pp, low confidence
72	GPT-4.1 nano	28.4%	estimated ± 9.7 pp, low confidence
73	GPT-4o	28.4%	estimated ± 9.7 pp, low confidence
74	GPT-4o mini	28.4%	estimated ± 9.7 pp, low confidence
75	GPT-5.1	28.4%	estimated ± 9.7 pp, low confidence
76	GPT-5.4 mini	28.4%	estimated ± 9.7 pp, low confidence
77	GPT-5.4 nano	28.4%	estimated ± 9.7 pp, low confidence
78	GPT-5 (high)	28.4%	estimated ± 9.7 pp, low confidence
79	GPT-OSS 120B	28.4%	estimated ± 9.7 pp, low confidence
80	GPT-OSS 20B	28.4%	estimated ± 9.7 pp, low confidence
81	Granite 4.2 30B	28.4%	estimated ± 9.7 pp, low confidence
82	Granite 4.2 3B	28.4%	estimated ± 9.7 pp, low confidence
83	Granite 4.2 8B	28.4%	estimated ± 9.7 pp, low confidence
84	Grok 4.3	28.4%	estimated ± 9.7 pp, low confidence
85	Hy3	28.4%	estimated ± 9.7 pp, low confidence
86	Hy3 Preview	28.4%	estimated ± 9.7 pp, low confidence
87	Inkling-Small	28.4%	estimated ± 9.7 pp, low confidence
88	K-Exaone	28.4%	estimated ± 9.7 pp, low confidence
89	Kimi K2.6	28.4%	estimated ± 9.7 pp, low confidence
90	Kimi K2.5	28.4%	estimated ± 9.7 pp, low confidence
91	Kimi K2.5 (Reasoning)	28.4%	estimated ± 9.7 pp, low confidence
92	Kimi K2.7 Code	28.4%	estimated ± 9.7 pp, low confidence
93	LFM2.5-2.6B	28.4%	estimated ± 9.7 pp, low confidence
94	Ling 2.6 Flash	28.4%	estimated ± 9.7 pp, low confidence
95	Ling 3.0 Flash	28.4%	estimated ± 9.7 pp, low confidence
96	Ling 3.0 Flash FP8	28.4%	estimated ± 9.7 pp, low confidence
97	Ling 3.0 Flash VL	28.4%	estimated ± 9.7 pp, low confidence
98	Ling 3.0 Tiny	28.4%	estimated ± 9.7 pp, low confidence
99	Llama 4 Maverick	28.4%	estimated ± 9.7 pp, low confidence
100	Llama 4 Scout	28.4%	estimated ± 9.7 pp, low confidence
101	Mercury 2.5	28.4%	estimated ± 9.7 pp, low confidence
102	MiMo-V2.5-Pro	28.4%	estimated ± 9.7 pp, low confidence
103	MiMo-V2-Flash	28.4%	estimated ± 9.7 pp, low confidence
104	MiniCPM5-2B	28.4%	estimated ± 9.7 pp, low confidence
105	MiniMax M2.7	28.4%	estimated ± 9.7 pp, low confidence
106	Mistral Large 3	28.4%	estimated ± 9.7 pp, low confidence
107	Mistral Medium 3.5 128B	28.4%	estimated ± 9.7 pp, low confidence
108	Mistral Small 4	28.4%	estimated ± 9.7 pp, low confidence
109	Mistral Small 4 (Reasoning)	28.4%	estimated ± 9.7 pp, low confidence
110	Nemotron 3.5 Lightning 30B A3B NVFP4	28.4%	estimated ± 9.7 pp, low confidence
111	Nemotron 3 Nano 30B	28.4%	estimated ± 9.7 pp, low confidence
112	Nemotron 3 Nano Omni 30B A3B	28.4%	estimated ± 9.7 pp, low confidence
113	Nemotron 3 Super 100B	28.4%	estimated ± 9.7 pp, low confidence
114	North Mini Code	28.4%	estimated ± 9.7 pp, low confidence
115	Quasar 438B	28.4%	estimated ± 9.7 pp, low confidence
116	Qwen3.5-122B-A10B	28.4%	estimated ± 9.7 pp, low confidence
117	Qwen3.6-27B	28.4%	estimated ± 9.7 pp, low confidence
118	Qwen3.6-35B-A3B	28.4%	estimated ± 9.7 pp, low confidence
119	Qwen3.6 Plus	28.4%	estimated ± 9.7 pp, low confidence
120	Qwen3.7 Max	28.4%	estimated ± 9.7 pp, low confidence
121	Qwen3.7 Plus	28.4%	estimated ± 9.7 pp, low confidence
122	Solar Pro 3	28.4%	estimated ± 9.7 pp, low confidence
123	Solar Pro 4	28.4%	estimated ± 9.7 pp, low confidence
124	Step 3.7 Flash	28.4%	estimated ± 9.7 pp, low confidence
125	Trinity-Large-Preview	28.4%	estimated ± 9.7 pp, low confidence
126	Trinity-Large-Thinking	28.4%	estimated ± 9.7 pp, low confidence
127	Ultravox v0.6 Llama 3.3 70B	28.4%	estimated ± 9.7 pp, low confidence
128	GPT-5.4	28.4%	estimated ± 5.8 pp, medium confidence
129	GPT-5.6 Luna	28.4%	estimated ± 5.8 pp, medium confidence
130	Claude Opus 4.5	28.4%	estimated ± 5.8 pp, low confidence
131	Claude Opus 4.6	28.4%	estimated ± 5.8 pp, low confidence
132	Claude Opus 4.7 (Adaptive)	28.4%	estimated ± 5.8 pp, low confidence
133	Claude Sonnet 4.6	28.4%	estimated ± 5.8 pp, low confidence
134	GLM-5	28.4%	estimated ± 5.8 pp, low confidence
135	GLM-5.1	28.4%	estimated ± 5.8 pp, low confidence
136	Muse Spark	28.4%	estimated ± 5.8 pp, low confidence
137	Muse Spark 1.1	28.4%	estimated ± 5.8 pp, low confidence
138	Qwen3.8 Max	27.3%	measured
139	Beam	27.1%	estimated ± 10.3 pp, low confidence
140	Claude Opus 5	26.0%	measured
141	Agents-A1	25.7%	estimated ± 10.3 pp, low confidence
142	DeepSeek V4 Flash 0731	25.1%	measured
143	Ornith-1.5-35B-A3B	20.8%	estimated ± 10.3 pp, low confidence
144	Agents-A1-4B	20.3%	estimated ± 10.3 pp, low confidence
145	GPT-5.2	19.8%	estimated ± 10.3 pp, low confidence
146	Qwen3.5 397B	17.8%	estimated ± 10.3 pp, low confidence
147	Qwen3.5-27B	17.3%	estimated ± 10.3 pp, low confidence
148	Qwen3.5-35B-A3B	17.3%	estimated ± 10.3 pp, low confidence
149	Ornith-1.5-9B	15.2%	estimated ± 10.3 pp, low confidence
150	LongCat-Flash-Lite-Sparse	12.1%	estimated ± 10.3 pp, low confidence

AutomationBench leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General