Agentic · tools

DeepsecBench-AA leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap
License: https://creativecommons.org/licenses/by-nc/4.0/

As of 2026-10-29, the highest measured score on DeepsecBench-AA is 46.3% by GPT-6 Sol. 147 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	GPT-6 Sol	46.3%	measured
2	Gemini 4 Carbon	39.4%	estimated ± 5.3 pp, low confidence
3	Gemini 4 Argon	39.1%	estimated ± 5.3 pp, low confidence
4	Atria Dawn Preview	37.7%	estimated ± 9.1 pp, low confidence
5	GPT-6 Astra	36.9%	measured
6	Kimi K3.1 Pro	36.7%	measured
7	MiniMax M3.1 Flash	34.3%	estimated ± 5.3 pp, medium confidence
8	DeepSeek V4.1 Flash 1017	31.2%	measured
9	Claude Mythos Preview	30.6%	estimated ± 8.1 pp, low confidence
10	GPT-6.1 Sol	27.1%	estimated ± 5.3 pp, medium confidence
11	Claude Opus 5.5	26.9%	measured
12	Grok 4.7	26.9%	measured
13	MiMo-V2.6-Pro	26.2%	measured
14	Claude Opus 5	25.9%	measured
15	Muse Spark 1.3	25.7%	estimated ± 7.9 pp, medium confidence
16	GPT-5.6 Terra	25.1%	measured
17	GLM-5.3	24.3%	measured
18	Grok 4.6	23.9%	estimated ± 7.9 pp, medium confidence
19	Claude Fable 5.1	23.5%	measured
20	GPT-6 Luna	23.5%	measured
21	Claude Sonnet 5.5	23.3%	estimated ± 5.2 pp, low confidence
22	GPT-6.1 Terra	22.6%	estimated ± 5.3 pp, medium confidence
23	Muse Spark 1.1	22.5%	estimated ± 6.0 pp, low confidence
24	GPT-5.5 Pro	22.3%	estimated ± 9.1 pp, medium confidence
25	Claude Fable 5	22.0%	estimated ± 7.9 pp, medium confidence
26	Claude Sonnet 5	21.8%	estimated ± 7.4 pp, medium confidence
27	GPT-5.6 Sol	21.6%	estimated ± 7.9 pp, medium confidence
28	Qwen3.8 Max Preview	20.9%	estimated ± 7.9 pp, medium confidence
29	Apodex 1.1	20.5%	estimated ± 7.4 pp, medium confidence
30	GLM-5.3-Flash	19.9%	measured
31	GPT-5.4 Pro	19.5%	estimated ± 9.1 pp, medium confidence
32	Sage-A1.5	19.4%	estimated ± 7.4 pp, medium confidence
33	DeepSeek V4 Pro 0813	19.2%	estimated ± 5.2 pp, low confidence
34	Hy4 preview	19.2%	estimated ± 5.2 pp, low confidence
35	MiMo-V2.6-Flash	18.7%	estimated ± 5.2 pp, low confidence
36	Qwen3.8-Flash-Next	18.6%	estimated ± 5.2 pp, low confidence
37	Muse Spark Open	17.8%	estimated ± 5.2 pp, low confidence
38	Qwen3.8 Max	17.5%	estimated ± 5.2 pp, low confidence
39	dots3-note Preview	16.8%	estimated ± 7.4 pp, medium confidence
40	Muse Spark 1.2	16.5%	estimated ± 7.9 pp, medium confidence
41	Claude Mythos 5	16.2%	estimated ± 9.1 pp, medium confidence
42	Ornith-1.5-397B	16.2%	estimated ± 5.2 pp, low confidence
43	Mistral Large 4	16.0%	measured
44	Step 5 Preview	15.8%	measured
45	GPT-5.6 Luna	15.5%	estimated ± 7.9 pp, medium confidence
46	DeepSeek V4 Flash 0731	15.3%	estimated ± 5.2 pp, low confidence
47	Grok 4.5	15.1%	estimated ± 7.9 pp, medium confidence
48	Gemini 3.5 Flash	14.9%	estimated ± 6.0 pp, low confidence
49	GLM-5	14.6%	estimated ± 7.4 pp, medium confidence
50	Gemini 3.8 Flash	14.5%	measured
51	Ling 3.1 Flash	13.9%	estimated ± 6.0 pp, low confidence
52	Claude Opus 4.8	12.6%	estimated ± 6.0 pp, low confidence
53	Kimi K3	12.5%	measured
54	Agents-A1	11.7%	estimated ± 7.4 pp, medium confidence
55	Qwen3.8-27B	11.5%	measured
56	Beam 1.0 501B-A23B FP8	11.4%	estimated ± 6.0 pp, low confidence
57	Step 3.7 Flash	11.3%	estimated ± 7.4 pp, medium confidence
58	Gemini 3.7 Flash	10.7%	estimated ± 7.9 pp, medium confidence
59	Claude Opus 4.6	10.1%	estimated ± 9.1 pp, medium confidence
60	Beam 1.0 501B-A23B NVFP4	9.7%	estimated ± 6.0 pp, low confidence
61	Claude Opus 4.7	8.8%	estimated ± 9.2 pp, medium confidence
62	Muse Wonder Full	8.5%	estimated ± 5.3 pp, low confidence
63	Inkling-Small	8.2%	estimated ± 6.0 pp, low confidence
64	Quasar 438B	7.7%	estimated ± 7.9 pp, medium confidence
65	Beam 1.0 501B-A23B	6.9%	measured
66	Apodex 1.1 Mini	6.0%	estimated ± 8.6 pp, medium confidence
67	Gemini 3.6 Flash	5.8%	estimated ± 7.9 pp, medium confidence
68	Claude Sonnet 4.6	5.5%	estimated ± 9.2 pp, medium confidence
69	Nemotron 3.5 Super VL 120B A12B	4.9%	estimated ± 8.6 pp, medium confidence
70	Nemotron 3.5 Super VL 120B A12B NVFP4	4.9%	estimated ± 8.6 pp, medium confidence
71	Ling 3.0 Flash VL	4.9%	estimated ± 8.6 pp, low confidence
72	Claude Opus 4.7 (Adaptive)	4.4%	estimated ± 6.0 pp, low confidence
73	Nemotron 3.5 Bolt 74B-A9B BF16	4.0%	estimated ± 9.1 pp, medium confidence
74	Qwen3.5 397B	3.8%	estimated ± 9.1 pp, medium confidence
75	GLM-5.2	3.5%	estimated ± 6.0 pp, low confidence
76	Agents-A1-4B	3.4%	estimated ± 9.1 pp, medium confidence
77	GPT-5.2	3.3%	estimated ± 9.1 pp, medium confidence
78	Solar Pro 4	3.1%	estimated ± 8.6 pp, low confidence
79	Qwen3.7 Max	2.8%	estimated ± 6.0 pp, low confidence
80	Qwen3.5-27B	2.6%	estimated ± 9.1 pp, medium confidence
81	Qwen3.5-35B-A3B	2.6%	estimated ± 9.1 pp, medium confidence
82	Muse Wonder	2.5%	estimated ± 5.3 pp, low confidence
83	Hy3	2.3%	estimated ± 7.9 pp, medium confidence
84	Hy3 Preview	2.3%	estimated ± 7.9 pp, medium confidence
85	Kimi K2.7 Code	2.2%	estimated ± 6.0 pp, low confidence
86	Sarvam 105B	1.6%	estimated ± 9.1 pp, medium confidence
87	LongCat-Flash-Lite-Sparse	1.5%	estimated ± 9.1 pp, medium confidence
88	Ling 3.0 Flash Sante	1.5%	estimated ± 6.0 pp, low confidence
89	Muse Glimmer 30B	1.3%	estimated ± 6.0 pp, low confidence
90	GLM-4.7	1.2%	estimated ± 8.6 pp, low confidence
91	MiniMax M3	1.1%	measured
92	Muse Spark	1.0%	estimated ± 8.6 pp, low confidence
93	Nemotron 3 Ultra	1.0%	measured
94	GPT-5.5	1.0%	estimated ± 6.0 pp, low confidence
95	Qwen3.6 Plus	0.9%	estimated ± 8.6 pp, low confidence
96	Sarvam 30B	0.9%	estimated ± 9.1 pp, low confidence
97	Nemotron 3.5 Bolt 74B-A9B	0.8%	estimated ± 8.6 pp, low confidence
98	Spark-X2.6-4B	0.7%	estimated ± 7.4 pp, low confidence
99	A.X K2	0.6%	estimated ± 8.6 pp, low confidence
100	GPT-5 (high)	0.4%	estimated ± 8.6 pp, low confidence
101	Kimi K2.5	0.1%	estimated ± 8.6 pp, low confidence
102	Kimi K2.5 (Reasoning)	0.1%	estimated ± 8.6 pp, low confidence
103	GPT-5.1	0.1%	estimated ± 8.6 pp, low confidence
104	K-EXAONE 2.0	0.1%	estimated ± 8.6 pp, low confidence
105	MiniCPM5-2B	0.0%	estimated ± 8.6 pp, low confidence
106	MiMo-V2-Flash	0.0%	estimated ± 8.6 pp, low confidence
107	Granite 4.2 30B	0.0%	estimated ± 8.6 pp, low confidence
108	Gemma 4 26B A4B	0.0%	estimated ± 8.6 pp, low confidence
109	Ling 3.0 Tiny	0.0%	estimated ± 8.6 pp, low confidence
110	Claude 3.5 Haiku	0.0%	estimated ± 8.6 pp, low confidence
111	DeepSeek V3 0324	0.0%	estimated ± 8.6 pp, low confidence
112	Gemma 4 12B	0.0%	estimated ± 8.6 pp, low confidence
113	Gemma 4 E2B	0.0%	estimated ± 8.6 pp, low confidence
114	Gemma 4 E4B	0.0%	estimated ± 8.6 pp, low confidence
115	GPT-4.1 mini	0.0%	estimated ± 8.6 pp, low confidence
116	GPT-4.1 nano	0.0%	estimated ± 8.6 pp, low confidence
117	GPT-4o	0.0%	estimated ± 8.6 pp, low confidence
118	GPT-4o mini	0.0%	estimated ± 8.6 pp, low confidence
119	Granite 4.2 3B	0.0%	estimated ± 8.6 pp, low confidence
120	K-Exaone	0.0%	estimated ± 8.6 pp, low confidence
121	LFM2.5-2.6B	0.0%	estimated ± 8.6 pp, low confidence
122	Ling 2.6 Flash	0.0%	estimated ± 8.6 pp, low confidence
123	Llama 3.3 70B	0.0%	estimated ± 8.6 pp, low confidence
124	Mercury 2.5	0.0%	estimated ± 8.6 pp, low confidence
125	Mistral Small 3.1	0.0%	estimated ± 8.6 pp, low confidence
126	Nemotron 3 Nano Omni 30B A3B	0.0%	estimated ± 8.6 pp, low confidence
127	North Mini Code	0.0%	estimated ± 8.6 pp, low confidence
128	Solar Pro 3	0.0%	estimated ± 8.6 pp, low confidence
129	Ultravox v0.6 Llama 3.3 70B	0.0%	estimated ± 8.6 pp, low confidence
130	Celeris-1	0.0%	estimated ± 7.9 pp, low confidence
131	Claude Haiku 5.5	0.0%	estimated ± 5.3 pp, low confidence
132	Command A+	0.0%	estimated ± 7.9 pp, low confidence
133	DeepSeek V3	0.0%	estimated ± 7.9 pp, low confidence
134	Gemini 2.5 Pro	0.0%	estimated ± 7.9 pp, low confidence
135	Gemini 3.1 Pro	0.0%	estimated ± 7.9 pp, low confidence
136	Gemini 3.5 Flash-Lite	0.0%	estimated ± 7.9 pp, low confidence
137	Gemma 3 27B	0.0%	estimated ± 7.9 pp, low confidence
138	Gemma 4 31B	0.0%	estimated ± 7.9 pp, low confidence
139	GLM-5.1	0.0%	estimated ± 6.0 pp, low confidence
140	GPT-5.4	0.0%	estimated ± 6.0 pp, low confidence
141	GPT-5.4 mini	0.0%	estimated ± 7.9 pp, low confidence
142	GPT-5.4 nano	0.0%	estimated ± 7.9 pp, low confidence
143	GPT-OSS 120B	0.0%	estimated ± 7.9 pp, low confidence
144	GPT-OSS 20B	0.0%	estimated ± 7.9 pp, low confidence
145	Granite 4.2 8B	0.0%	estimated ± 7.9 pp, low confidence
146	Grok 4.3	0.0%	estimated ± 7.9 pp, low confidence
147	Inkling	0.0%	estimated ± 6.0 pp, low confidence
148	Kimi K2.6	0.0%	estimated ± 7.9 pp, medium confidence
149	LFM3-1.7B	0.0%	estimated ± 7.4 pp, low confidence
150	Ling 3.0 Flash	0.0%	estimated ± 6.0 pp, low confidence
151	Llama 4 Maverick	0.0%	estimated ± 7.9 pp, low confidence
152	Llama 4 Scout	0.0%	estimated ± 7.9 pp, low confidence
153	MiMo-V2.5-Pro	0.0%	estimated ± 7.9 pp, medium confidence
154	MiniMax M2.7	0.0%	estimated ± 7.9 pp, low confidence
155	Mistral Large 3	0.0%	estimated ± 7.9 pp, low confidence
156	Mistral Medium 3.5 128B	0.0%	estimated ± 7.9 pp, low confidence
157	Mistral Small 4	0.0%	estimated ± 7.9 pp, low confidence
158	Mistral Small 4 (Reasoning)	0.0%	estimated ± 7.9 pp, low confidence
159	Nemotron 3.5 Lightning 30B A3B NVFP4	0.0%	estimated ± 7.9 pp, low confidence
160	Nemotron 3 Nano 30B	0.0%	estimated ± 7.9 pp, low confidence
161	Ornith-1.5-35B-A3B	0.0%	estimated ± 6.0 pp, low confidence
162	Ornith-1.5-9B	0.0%	estimated ± 7.4 pp, low confidence
163	Qwen3.5-122B-A10B	0.0%	estimated ± 7.9 pp, low confidence
164	Qwen3.6-27B	0.0%	estimated ± 7.9 pp, low confidence
165	Qwen3.6-35B-A3B	0.0%	estimated ± 6.0 pp, low confidence
166	Qwen3.7 Plus	0.0%	estimated ± 6.0 pp, low confidence
167	Trinity-Large-Preview	0.0%	estimated ± 7.9 pp, low confidence
168	Trinity-Large-Thinking	0.0%	estimated ± 7.9 pp, low confidence

DeepsecBench-AA leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General