Coding

SWE-bench Verified leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

As of 2026-10-07, the highest measured score on SWE-bench Verified is 96.0% by Claude Opus 5. 140 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	Claude Opus 5.5	96.5%	estimated ± 0.8 pp, low confidence
2	Claude Opus 5	96.0%	measured
3	Gemini 3.8 Flash	95.8%	estimated ± 4.4 pp, high confidence
4	Claude Mythos 5	95.5%	measured
5	GPT-6 Astra	95.3%	estimated ± 0.8 pp, medium confidence
6	Claude Fable 5	95.0%	measured
7	Claude Fable 5.1	94.6%	estimated ± 2.5 pp, medium confidence
8	Muse Spark 1.3	94.3%	estimated ± 5.2 pp, low confidence
9	Pareto 26.9	93.4%	estimated ± 5.2 pp, low confidence
10	Ember-1	92.2%	measured
11	GPT-6.1 Sol	91.9%	estimated ± 5.2 pp, low confidence
12	MiMo-V2.6-Pro	91.9%	estimated ± 5.2 pp, low confidence
13	SWE-2	91.8%	estimated ± 0.8 pp, medium confidence
14	Grok 4.7	91.3%	estimated ± 5.2 pp, low confidence
15	Sakana Fugu-Ultra	91.1%	estimated ± 2.5 pp, high confidence
16	Pareto 26.10 Preview	90.6%	estimated ± 5.2 pp, low confidence
17	Grok 4.6	90.5%	estimated ± 4.4 pp, high confidence
18	GPT-6 Sol	89.8%	estimated ± 5.2 pp, low confidence
19	Gemini 3.6 Flash	89.6%	estimated ± 4.4 pp, high confidence
20	MiMo-V2.6-Flash	89.2%	estimated ± 5.2 pp, low confidence
21	Step 5 Preview	89.1%	estimated ± 5.2 pp, low confidence
22	Gemini 4 Argon	89.0%	estimated ± 4.4 pp, medium confidence
23	Claude Opus 4.8	88.6%	measured
24	Claude Haiku 5.5	88.4%	estimated ± 0.8 pp, medium confidence
25	GPT-6 Luna	88.4%	estimated ± 5.2 pp, low confidence
26	Claude Sonnet 5.5	88.3%	estimated ± 0.8 pp, medium confidence
27	Claude Opus 4.7 (Adaptive)	87.6%	measured
28	Ling 3.1 Flash	87.2%	estimated ± 6.6 pp, medium confidence
29	Mistral Large 4	86.4%	estimated ± 4.4 pp, medium confidence
30	Gemini 3.7 Flash	86.2%	estimated ± 0.8 pp, medium confidence
31	Ornith-1.5-397B	86.0%	measured
32	Grok 4.5	85.8%	estimated ± 2.5 pp, high confidence
33	GPT-5.5	85.7%	estimated ± 0.8 pp, medium confidence
34	GPT-5.6 Sol	85.7%	estimated ± 2.5 pp, high confidence
35	SWE-1.7	85.3%	estimated ± 0.8 pp, medium confidence
36	Qwen3.8 Max Preview	85.2%	estimated ± 5.5 pp, medium confidence
37	Claude Sonnet 5	85.2%	measured
38	GPT-5.3 Codex	85.0%	measured
39	Muse Spark 1.2	84.9%	estimated ± 4.5 pp, high confidence
40	GPT-5.6 Terra	84.9%	estimated ± 2.5 pp, high confidence
41	GPT-5.6 Luna	84.4%	estimated ± 2.5 pp, high confidence
42	DeepSeek V4.1 Flash	84.1%	estimated ± 2.4 pp, medium confidence
43	Muse Spark 1.1	83.5%	estimated ± 2.5 pp, high confidence
44	Claude Opus 4.7	83.1%	estimated ± 0.8 pp, medium confidence
45	Hy4 preview	82.9%	estimated ± 2.4 pp, high confidence
46	GLM-5.3	82.7%	estimated ± 2.4 pp, high confidence
47	Ornith-1.0-397B	82.4%	measured
48	GLM-5.3-Flash	82.3%	estimated ± 2.4 pp, high confidence
49	Qwen3.8 Max	82.2%	estimated ± 2.4 pp, high confidence
50	Atria Dawn Preview	82.2%	estimated ± 2.5 pp, high confidence
51	Laguna S 2.1	82.0%	estimated ± 2.5 pp, high confidence
52	Quasar 438B	81.9%	estimated ± 5.5 pp, medium confidence
53	Apodex 1.1 Mini	81.8%	estimated ± 5.5 pp, medium confidence
54	Claude Opus 4.6 (Adaptive)	81.7%	estimated ± 4.4 pp, high confidence
55	Sakana Fugu	81.7%	estimated ± 2.5 pp, high confidence
56	Composer 2.5	81.4%	estimated ± 4.5 pp, high confidence
57	Kimi K3	81.2%	estimated ± 4.4 pp, high confidence
58	Hy3	81.0%	estimated ± 5.5 pp, medium confidence
59	Claude Opus 4.5	80.9%	measured
60	Beam	80.9%	measured
61	Claude Opus 4.6	80.8%	measured
62	GPT-5.4	80.7%	estimated ± 2.5 pp, high confidence
63	DeepSeek V4 Pro 0813	80.6%	measured
64	MiniMax M3	80.5%	measured
65	GLM-5.2	80.4%	estimated ± 2.4 pp, high confidence
66	Qwen3.8-Omni-Flash	80.4%	estimated ± 2.4 pp, high confidence
67	Qwen3.7 Max	80.4%	measured
68	MiMo-V2.5-Pro	80.3%	estimated ± 2.5 pp, high confidence
69	GPT-5.4 mini	80.2%	estimated ± 0.8 pp, medium confidence
70	Inkling-Small	80.2%	measured
71	Kimi K2.6	80.2%	measured
72	Qwen3.8-Flash-Next	80.2%	estimated ± 2.4 pp, high confidence
73	GPT-5.2	80.0%	measured
74	Seed 2.1 Pro	79.9%	estimated ± 2.4 pp, high confidence
75	Ling 3.0 Flash	79.8%	estimated ± 2.5 pp, high confidence
76	Claude Sonnet 4.6	79.6%	measured
77	Step 3.7 Flash	79.6%	estimated ± 2.5 pp, high confidence
78	MiMo-V2.5	79.4%	estimated ± 2.5 pp, high confidence
79	DeepSeek V4 Flash 0731	79.0%	measured
80	Ornith-1.5-35B-A3B	79.0%	measured
81	Seed 2.1 Turbo	78.8%	estimated ± 2.4 pp, high confidence
82	GPT-5.2-Codex	78.8%	estimated ± 4.4 pp, high confidence
83	Qwen3.6 Plus	78.8%	measured
84	Gemini 3.5 Flash	78.6%	estimated ± 2.5 pp, high confidence
85	Qwen 3.6 Max (preview)	78.5%	estimated ± 2.4 pp, high confidence
86	GLM-5.1	78.5%	estimated ± 2.4 pp, high confidence
87	BTL-4	78.4%	measured
88	dots3-note Preview	78.4%	measured
89	Qwen3.8-27B	78.3%	estimated ± 2.4 pp, high confidence
90	DeepSeek V3.2	78.0%	estimated ± 4.8 pp, medium confidence
91	MiMo-V2-Pro	78.0%	measured
92	Gemini 3.5 Flash-Lite	77.8%	estimated ± 2.5 pp, high confidence
93	GLM-5	77.8%	measured
94	Apodex 1.1	77.7%	measured
95	Qwen3.7 Plus	77.7%	measured
96	Gemini 3.1 Pro	77.7%	estimated ± 4.4 pp, high confidence
97	Inkling	77.6%	measured
98	Mistral Medium 3.5 128B	77.6%	measured
99	Muse Spark	77.4%	measured
100	MiniMax M2.7	77.4%	estimated ± 2.4 pp, high confidence
101	Claude Sonnet 4.5	77.2%	measured
102	Qwen3.6-27B	77.2%	measured
103	Ling 3.0 Flash FP8	76.8%	estimated ± 5.5 pp, medium confidence
104	Kimi K2.5	76.8%	measured
105	Kimi K2.5 (Reasoning)	76.8%	measured
106	Grok 4.20	76.7%	measured
107	GPT-5.4 nano	76.6%	estimated ± 4.4 pp, high confidence
108	GPT-5.1	76.3%	estimated ± 4.4 pp, high confidence
109	Grok 4.3	76.3%	estimated ± 4.4 pp, high confidence
110	Qwen3.5 Flash	76.3%	estimated ± 4.4 pp, high confidence
111	Kimi K2.7 Code	76.3%	estimated ± 4.4 pp, high confidence
112	GLM-4.5	76.3%	estimated ± 4.4 pp, high confidence
113	Qwen3.5 397B	76.2%	measured
114	GLM-5 (Reasoning)	76.1%	estimated ± 4.4 pp, high confidence
115	Muse Glimmer 30B	76.0%	measured
116	Claude Sonnet 4.5 Thinking	75.9%	estimated ± 4.4 pp, high confidence
117	GPT-5.1-Codex-Max	75.8%	estimated ± 4.4 pp, high confidence
118	Composer 2	75.7%	estimated ± 4.8 pp, medium confidence
119	Ornith-1.0-35B	75.6%	measured
120	Claude Opus 4.5 Thinking	75.5%	estimated ± 4.4 pp, high confidence
121	Gemini 3 Flash	75.5%	estimated ± 4.4 pp, high confidence
122	GPT-5 (high)	75.4%	estimated ± 4.4 pp, high confidence
123	MiMo-V2-Omni	74.8%	measured
124	Qwen3.5 Plus	74.6%	estimated ± 4.4 pp, high confidence
125	Laguna M.1	74.6%	measured
126	Claude 4.1 Opus	74.5%	measured
127	MiniMax M2.5	74.5%	estimated ± 4.4 pp, high confidence
128	Hy3 Preview	74.4%	measured
129	Gemini 3 Pro	74.4%	estimated ± 4.4 pp, high confidence
130	GPT-5 mini	74.3%	estimated ± 4.4 pp, high confidence
131	GPT-5.1-Codex	74.1%	estimated ± 4.4 pp, high confidence
132	Claude Haiku 4.5 Thinking	73.8%	estimated ± 4.4 pp, high confidence
133	GLM-4.7	73.8%	measured
134	MAI-Thinking-1	73.5%	measured
135	MiMo-V2-Flash	73.4%	measured
136	Qwen3.6-35B-A3B	73.4%	measured
137	Claude Haiku 4.5	73.3%	measured
138	Claude 4 Sonnet	72.7%	measured
139	DeepSeek V3.2 (Thinking)	72.6%	estimated ± 4.4 pp, high confidence
140	MAI-Code-1.1-Flash	72.6%	measured
141	Qwen3.5-27B	72.4%	measured
142	Ling 3.0 Flash VL	72.4%	estimated ± 6.6 pp, medium confidence
143	Qwen3 Max	72.3%	estimated ± 4.4 pp, high confidence
144	GLM-4.6	72.2%	estimated ± 4.4 pp, high confidence
145	BTL-3	72.2%	estimated ± 10.5 pp, low confidence
146	Qwen3.5-122B-A10B	72.0%	measured
147	Nemotron 3 Ultra	71.9%	measured
148	Grok 4.1 Fast (Reasoning)	71.9%	estimated ± 4.4 pp, high confidence
149	Gemini 3.1 Flash-Lite	71.7%	estimated ± 4.4 pp, medium confidence
150	Grok 4 Fast (Reasoning)	71.7%	estimated ± 4.4 pp, medium confidence
151	Laguna XS 2.1	70.9%	measured
152	Grok Code Fast 1	70.8%	measured
153	Ornith-1.5-9B	70.6%	measured
154	Solar Pro 4	70.6%	measured
155	Solar Open 2	70.4%	measured
156	Laguna XS.2	69.9%	measured
157	Ornith-1.0-9B	69.4%	measured
158	Qwen3.5-35B-A3B	69.2%	measured
159	K-EXAONE 2.0	68.2%	measured
160	LongCat-Flash-Lite-Sparse	68.2%	measured
161	o1	68.1%	estimated ± 5.5 pp, medium confidence
162	Gemma 4 26B A4B	67.7%	estimated ± 5.5 pp, medium confidence
163	A.X K2	66.4%	estimated ± 5.7 pp, medium confidence
164	Nemotron 3 Super 100B	65.9%	estimated ± 5.5 pp, medium confidence
165	DeepSeek V3 0324	64.6%	estimated ± 6.6 pp, medium confidence
166	North Mini Code	64.3%	estimated ± 6.6 pp, medium confidence
167	Gemini 2.5 Pro	63.8%	measured
168	Mercury 2.5	62.2%	estimated ± 5.7 pp, medium confidence
169	Mellum2-12B-A2.5B-Thinking	61.9%	estimated ± 10.5 pp, low confidence
170	o1-preview	61.3%	estimated ± 5.5 pp, medium confidence
171	Gemma 4 31B	60.8%	estimated ± 4.8 pp, low confidence
172	Ternary Bonsai 2 27B	60.8%	measured
173	ZAYA1-8B	59.4%	estimated ± 10.5 pp, low confidence
174	K-Exaone	58.5%	estimated ± 5.5 pp, medium confidence
175	Granite 4.2 30B	57.0%	measured
176	Gemma 4 12B	56.8%	estimated ± 5.5 pp, medium confidence
177	GPT-OSS 120B	55.9%	estimated ± 5.5 pp, medium confidence
178	GPT-4.1	54.6%	measured
179	ZAYA1-74B-Preview	53.2%	measured
180	Nemotron 3.5 Lightning 30B A3B NVFP4	52.8%	measured
181	Command A+	51.5%	estimated ± 5.5 pp, medium confidence
182	Agents-A1-4B	50.4%	estimated ± 5.7 pp, medium confidence
183	o3-mini	49.3%	measured
184	LLaDA2.2-flash	49.3%	measured
185	Mistral Small 4	49.2%	estimated ± 5.5 pp, medium confidence
186	Mistral Small 4 (Reasoning)	49.2%	estimated ± 5.5 pp, medium confidence
187	Claude 3.5 Sonnet	49.0%	measured
188	Granite 4.2 8B	47.7%	measured
189	Trinity-Large-Preview	47.5%	estimated ± 5.5 pp, medium confidence
190	Trinity-Large-Thinking	47.5%	estimated ± 5.5 pp, medium confidence
191	Ling 2.6 Flash	46.5%	estimated ± 5.5 pp, medium confidence
192	MiniCPM5-2B	46.4%	measured
193	Solar Pro 3	44.4%	estimated ± 6.6 pp, low confidence
194	Gemini 1.5 Pro	43.1%	estimated ± 5.5 pp, medium confidence
195	Granite 4.2 3B	42.7%	estimated ± 5.7 pp, low confidence
196	Ling 3.0 Tiny	42.4%	estimated ± 6.6 pp, low confidence
197	DeepSeek V3	42.0%	measured
198	Mellum2-12B-A2.5B-Instruct	38.5%	estimated ± 10.5 pp, low confidence
199	GPT-4 Turbo	38.3%	estimated ± 5.5 pp, medium confidence
200	GPT-OSS 20B	36.5%	estimated ± 5.5 pp, medium confidence
201	MiniCPM5-1B	35.4%	estimated ± 10.5 pp, low confidence
202	Mistral Large 3	35.0%	estimated ± 5.5 pp, low confidence
203	Claude 3 Opus	33.7%	estimated ± 5.5 pp, low confidence
204	LLaDA2.2-mini	30.6%	estimated ± 10.5 pp, low confidence
205	Llama 4 Maverick	25.7%	estimated ± 5.5 pp, low confidence
206	GPT-4.1 mini	23.6%	measured
207	Celeris-1	21.0%	estimated ± 5.5 pp, low confidence
208	Nemotron 3 Nano 30B	21.0%	estimated ± 5.5 pp, low confidence
209	Nemotron 3 Nano Omni 30B A3B	19.5%	estimated ± 5.5 pp, low confidence
210	Ultravox v0.6 Llama 3.3 70B	15.1%	estimated ± 5.5 pp, low confidence
211	GPT-4o mini	13.9%	estimated ± 5.5 pp, low confidence
212	GPT-4.1 nano	13.3%	estimated ± 5.5 pp, low confidence
213	Gemma 3 27B	11.0%	estimated ± 5.5 pp, low confidence
214	Gemma 4 E4B	9.6%	estimated ± 5.5 pp, low confidence
215	Llama 4 Scout	7.3%	estimated ± 5.5 pp, low confidence
216	LFM2.5-2.6B	6.6%	estimated ± 5.5 pp, low confidence
217	Gemma 4 E2B	5.7%	estimated ± 5.5 pp, low confidence

SWE-bench Verified leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General