Agentic · tools

CWE-Bench-AA leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap
License: https://creativecommons.org/licenses/by-nc/4.0/

As of 2026-10-29, the highest measured score on CWE-Bench-AA is 68.3% by DeepSeek V4.1 Flash 1017. 153 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	DeepSeek V4.1 Flash 1017	68.3%	measured
2	Grok 4.7	68.3%	measured
3	GPT-6 Sol	64.2%	measured
4	GPT-6 Astra	63.3%	measured
5	MiMo-V2.6-Pro	63.3%	measured
6	Gemini 4 Carbon	61.9%	estimated ± 6.0 pp, low confidence
7	Claude Opus 5	61.2%	estimated ± 4.5 pp, medium confidence
8	MiMo-V2.6-Flash	61.0%	estimated ± 6.0 pp, low confidence
9	Kimi K3.1 Pro	60.8%	measured
10	Gemini 4 Argon	60.8%	estimated ± 4.5 pp, medium confidence
11	Muse Spark 1.3	60.1%	estimated ± 4.5 pp, medium confidence
12	Ultramarine 1 Flash	59.9%	estimated ± 6.0 pp, low confidence
13	Muse Spark Open	59.7%	estimated ± 5.2 pp, medium confidence
14	GPT-6.1 Sol	59.7%	estimated ± 4.6 pp, medium confidence
15	Atria Dawn Preview	59.6%	estimated ± 5.8 pp, low confidence
16	Claude Sonnet 5.5	59.4%	estimated ± 4.6 pp, medium confidence
17	DeepSeek V4 Pro 0813	59.2%	estimated ± 5.2 pp, medium confidence
18	GPT-5.6 Sol	58.9%	estimated ± 4.5 pp, medium confidence
19	MiniMax M3.1 Flash	58.6%	estimated ± 4.5 pp, medium confidence
20	Ling 3.1 Flash	58.6%	estimated ± 5.8 pp, medium confidence
21	Qwen3.8 Max Preview	58.5%	estimated ± 9.8 pp, low confidence
22	Claude Opus 5.5	58.3%	measured
23	Claude Haiku 5.5	58.0%	estimated ± 4.6 pp, medium confidence
24	Claude Sonnet 5	57.8%	estimated ± 5.2 pp, medium confidence
25	Ling 3.0 Flash Sante	57.6%	estimated ± 5.2 pp, medium confidence
26	Grok 4.6	56.9%	estimated ± 9.8 pp, low confidence
27	Apodex 1.1	56.9%	estimated ± 5.2 pp, medium confidence
28	Ornith-1.5-397B	56.9%	estimated ± 5.2 pp, medium confidence
29	GPT-5.5 Pro	56.7%	estimated ± 5.8 pp, medium confidence
30	GPT-6 Luna	56.7%	measured
31	Claude Fable 5	56.7%	estimated ± 9.8 pp, low confidence
32	Sage-A1.5	56.1%	estimated ± 5.2 pp, medium confidence
33	GPT-5.4 Pro	55.8%	estimated ± 5.8 pp, medium confidence
34	GLM-5.3	55.8%	measured
35	GLM-5.3-Flash	55.8%	measured
36	GLM-5.2	55.8%	estimated ± 5.2 pp, medium confidence
37	LFM2.5-24B-A2B	55.7%	estimated ± 6.0 pp, low confidence
38	GPT-5.6 Terra	55.3%	estimated ± 4.5 pp, medium confidence
39	GPT-6.1 Terra	55.1%	estimated ± 8.4 pp, low confidence
40	Qwen3.7 Max	54.7%	estimated ± 5.2 pp, medium confidence
41	Gemini 3.7 Flash	54.7%	estimated ± 4.5 pp, medium confidence
42	Claude Mythos 5	54.4%	estimated ± 5.8 pp, medium confidence
43	GPT-5.6 Luna	54.0%	estimated ± 4.5 pp, medium confidence
44	dots3-note Preview	53.8%	estimated ± 5.2 pp, medium confidence
45	GLM-5.1	53.5%	estimated ± 5.2 pp, medium confidence
46	Kimi K3	53.3%	measured
47	Gemini 3.8 Flash	52.5%	measured
48	Step 5 Preview	52.5%	measured
49	Beam 1.0 501B-A23B FP8	52.2%	estimated ± 5.2 pp, medium confidence
50	Beam 1.0 501B-A23B NVFP4	52.2%	estimated ± 5.2 pp, medium confidence
51	Muse Spark 1.2	51.9%	estimated ± 9.8 pp, low confidence
52	GLM-5	51.4%	estimated ± 5.2 pp, medium confidence
53	Claude Fable 5.1	50.8%	measured
54	Mistral Large 4	50.8%	measured
55	Hy4 preview	50.4%	estimated ± 4.6 pp, low confidence
56	Claude Opus 4.6	49.8%	estimated ± 5.8 pp, medium confidence
57	Beam 1.0 501B-A23B	49.2%	measured
58	GPT-5.4	48.8%	estimated ± 5.8 pp, medium confidence
59	Grok 4.5	48.6%	estimated ± 9.8 pp, low confidence
60	Agents-A1	47.5%	estimated ± 5.2 pp, medium confidence
61	Ling 3.0 Flash	47.3%	estimated ± 4.6 pp, low confidence
62	Gemini 3.5 Flash	47.1%	estimated ± 9.8 pp, low confidence
63	Step 3.7 Flash	46.9%	estimated ± 5.2 pp, medium confidence
64	Claude Opus 4.8	46.8%	estimated ± 4.5 pp, medium confidence
65	Qwen3.8-Flash-Next	46.5%	estimated ± 4.5 pp, medium confidence
66	Qwen3.8 Max	46.5%	estimated ± 4.5 pp, medium confidence
67	Claude Opus 4.7 (Adaptive)	46.1%	estimated ± 4.5 pp, medium confidence
68	Muse Spark 1.1	45.0%	estimated ± 4.5 pp, medium confidence
69	Claude Opus 4.7	44.9%	estimated ± 4.5 pp, medium confidence
70	Gemini 3.6 Flash	44.7%	estimated ± 9.8 pp, low confidence
71	GPT-5.5	44.7%	estimated ± 4.5 pp, medium confidence
72	Muse Wonder Full	44.5%	estimated ± 8.4 pp, low confidence
73	Nemotron 3.5 Super VL 120B A12B	44.2%	estimated ± 5.8 pp, medium confidence
74	Claude Haiku 4.5	44.0%	estimated ± 11.3 pp, low confidence
75	GPT-5.3-Codex-Spark	44.0%	estimated ± 11.3 pp, low confidence
76	Grok 4.20	44.0%	estimated ± 11.3 pp, low confidence
77	Nemotron 3.5 Super VL 120B A12B NVFP4	43.8%	estimated ± 5.8 pp, medium confidence
78	Inkling-Small	43.7%	estimated ± 5.8 pp, medium confidence
79	Inkling	43.4%	estimated ± 5.8 pp, medium confidence
80	DeepSeek V4 Flash 0731	43.3%	estimated ± 5.2 pp, medium confidence
81	Claude Sonnet 4.6	43.3%	estimated ± 4.5 pp, medium confidence
82	Kimi K2.6	42.2%	estimated ± 4.5 pp, medium confidence
83	Hy3 Preview	41.8%	estimated ± 9.8 pp, low confidence
84	Qwen3.7 Plus	41.7%	estimated ± 4.5 pp, low confidence
85	MiniMax M3	41.7%	measured
86	Apodex 1.1 Mini	40.6%	estimated ± 9.8 pp, low confidence
87	Quasar 438B	40.3%	estimated ± 9.8 pp, low confidence
88	Muse Wonder	39.6%	estimated ± 8.4 pp, low confidence
89	Ling 3.0 Flash VL	39.3%	estimated ± 9.8 pp, low confidence
90	MiMo-V2.5-Pro	37.5%	estimated ± 9.8 pp, low confidence
91	Nemotron 3.5 Bolt 74B-A9B BF16	37.3%	estimated ± 5.8 pp, medium confidence
92	Qwen3.5 397B	36.5%	estimated ± 5.8 pp, medium confidence
93	Grok 4.3	35.8%	estimated ± 9.8 pp, low confidence
94	Hy3	34.9%	estimated ± 9.8 pp, low confidence
95	Agents-A1-4B	34.7%	estimated ± 5.8 pp, medium confidence
96	Nemotron 3.5 Bolt 74B-A9B	34.7%	estimated ± 5.8 pp, medium confidence
97	GPT-5.2	33.9%	estimated ± 5.8 pp, medium confidence
98	Kimi K2.7 Code	33.6%	estimated ± 9.8 pp, low confidence
99	Qwen3.5-122B-A10B	32.4%	estimated ± 5.8 pp, medium confidence
100	GPT-5.4 mini	32.3%	estimated ± 9.8 pp, low confidence
101	MiniMax M2.7	32.2%	estimated ± 9.8 pp, low confidence
102	Muse Spark	31.6%	estimated ± 9.8 pp, low confidence
103	Qwen3.6 Plus	31.2%	estimated ± 9.8 pp, low confidence
104	Qwen3.6-27B	31.0%	estimated ± 9.8 pp, low confidence
105	Gemini 3.5 Flash-Lite	30.9%	estimated ± 9.8 pp, low confidence
106	Qwen3.5-27B	30.4%	estimated ± 5.8 pp, medium confidence
107	Qwen3.5-35B-A3B	30.4%	estimated ± 5.8 pp, medium confidence
108	Kimi K2.5	30.1%	estimated ± 5.8 pp, medium confidence
109	Kimi K2.5 (Reasoning)	30.1%	estimated ± 5.8 pp, medium confidence
110	A.X K2	29.6%	estimated ± 9.8 pp, low confidence
111	GPT-5.4 nano	29.0%	estimated ± 9.8 pp, low confidence
112	Nemotron 3 Ultra	27.5%	measured
113	GPT-5 (high)	27.4%	estimated ± 9.8 pp, low confidence
114	Qwen3.8-27B	26.7%	measured
115	Spark-X2.6-4B	26.3%	estimated ± 5.2 pp, low confidence
116	Qwen3.6-35B-A3B	26.0%	estimated ± 9.8 pp, low confidence
117	GLM-4.7	24.3%	estimated ± 5.8 pp, medium confidence
118	Sarvam 105B	22.8%	estimated ± 5.8 pp, medium confidence
119	Solar Pro 4	22.6%	estimated ± 5.8 pp, medium confidence
120	LongCat-Flash-Lite-Sparse	22.2%	estimated ± 5.8 pp, medium confidence
121	GPT-5.1	22.2%	estimated ± 9.8 pp, low confidence
122	K-EXAONE 2.0	21.1%	estimated ± 9.8 pp, low confidence
123	Gemini 3.1 Pro	20.2%	estimated ± 9.8 pp, low confidence
124	Muse Glimmer 30B	19.7%	estimated ± 9.8 pp, low confidence
125	Ornith-1.5-35B-A3B	19.3%	estimated ± 5.2 pp, low confidence
126	Mistral Medium 3.5 128B	18.2%	estimated ± 9.8 pp, low confidence
127	Nemotron 3.5 Lightning 30B A3B NVFP4	15.6%	estimated ± 5.8 pp, low confidence
128	Sarvam 30B	14.9%	estimated ± 5.8 pp, low confidence
129	MiniCPM5-2B	14.9%	estimated ± 9.8 pp, low confidence
130	Ornith-1.5-9B	14.6%	estimated ± 5.2 pp, low confidence
131	MiMo-V2-Flash	9.5%	estimated ± 9.8 pp, low confidence
132	Gemma 4 31B	8.8%	estimated ± 9.8 pp, low confidence
133	GPT-OSS 120B	8.1%	estimated ± 9.8 pp, low confidence
134	LFM3-1.7B	6.4%	estimated ± 5.2 pp, low confidence
135	Granite 4.2 30B	5.6%	estimated ± 9.8 pp, low confidence
136	Gemma 4 26B A4B	4.9%	estimated ± 9.8 pp, low confidence
137	Ling 3.0 Tiny	4.6%	estimated ± 9.8 pp, low confidence
138	Command A+	1.1%	estimated ± 9.8 pp, low confidence
139	Celeris-1	0.0%	estimated ± 9.8 pp, low confidence
140	Claude 3.5 Haiku	0.0%	estimated ± 9.8 pp, low confidence
141	DeepSeek V3	0.0%	estimated ± 9.8 pp, low confidence
142	DeepSeek V3 0324	0.0%	estimated ± 9.8 pp, low confidence
143	Gemini 2.5 Pro	0.0%	estimated ± 9.8 pp, low confidence
144	Gemma 3 27B	0.0%	estimated ± 9.8 pp, low confidence
145	Gemma 4 12B	0.0%	estimated ± 9.8 pp, low confidence
146	Gemma 4 E2B	0.0%	estimated ± 9.8 pp, low confidence
147	Gemma 4 E4B	0.0%	estimated ± 9.8 pp, low confidence
148	GPT-4.1 mini	0.0%	estimated ± 9.8 pp, low confidence
149	GPT-4.1 nano	0.0%	estimated ± 9.8 pp, low confidence
150	GPT-4o	0.0%	estimated ± 9.8 pp, low confidence
151	GPT-4o mini	0.0%	estimated ± 9.8 pp, low confidence
152	GPT-OSS 20B	0.0%	estimated ± 9.8 pp, low confidence
153	Granite 4.2 3B	0.0%	estimated ± 9.8 pp, low confidence
154	Granite 4.2 8B	0.0%	estimated ± 9.8 pp, low confidence
155	K-Exaone	0.0%	estimated ± 9.8 pp, low confidence
156	LFM2.5-2.6B	0.0%	estimated ± 9.8 pp, low confidence
157	Ling 2.6 Flash	0.0%	estimated ± 9.8 pp, low confidence
158	Llama 3.3 70B	0.0%	estimated ± 9.8 pp, low confidence
159	Llama 4 Maverick	0.0%	estimated ± 9.8 pp, low confidence
160	Llama 4 Scout	0.0%	estimated ± 9.8 pp, low confidence
161	Mercury 2.5	0.0%	estimated ± 9.8 pp, low confidence
162	Mistral Large 3	0.0%	estimated ± 9.8 pp, low confidence
163	Mistral Small 3.1	0.0%	estimated ± 9.8 pp, low confidence
164	Mistral Small 4	0.0%	estimated ± 9.8 pp, low confidence
165	Mistral Small 4 (Reasoning)	0.0%	estimated ± 9.8 pp, low confidence
166	Nemotron 3 Nano 30B	0.0%	estimated ± 9.8 pp, low confidence
167	Nemotron 3 Nano Omni 30B A3B	0.0%	estimated ± 9.8 pp, low confidence
168	North Mini Code	0.0%	estimated ± 9.8 pp, low confidence
169	Solar Pro 3	0.0%	estimated ± 9.8 pp, low confidence
170	Trinity-Large-Preview	0.0%	estimated ± 9.8 pp, low confidence
171	Trinity-Large-Thinking	0.0%	estimated ± 9.8 pp, low confidence
172	Ultravox v0.6 Llama 3.3 70B	0.0%	estimated ± 9.8 pp, low confidence

CWE-Bench-AA leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General