benchgap
Coding

SWE-bench Verified leaderboard

As of 2026-11-21, the highest measured score on SWE-bench Verified is 96.0% by Claude Opus 5. 172 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#ModelScoreSource
1Claude Fable 5.2100.0%estimated ± 3.6 pp, medium confidence
2Claude Fable 5.197.8%estimated ± 3.6 pp, medium confidence
3Claude Opus 596.0%measured
4Claude Mythos 595.5%measured
5Claude Fable 595.0%measured
6Claude Opus 5.594.5%estimated ± 2.2 pp, low confidence
7GPT-6 Astra93.9%estimated ± 2.2 pp, medium confidence
8Muse Wonder Full92.3%estimated ± 5.1 pp, low confidence
9Muse Wonder92.3%estimated ± 5.1 pp, low confidence
10SWE-292.2%estimated ± 2.2 pp, medium confidence
11Ember-192.2%measured
12Sakana Fugu-Ultra91.8%estimated ± 3.6 pp, high confidence
13Qwen4 Max90.8%estimated ± 5.1 pp, low confidence
14Kimi K3.1 Pro90.8%estimated ± 5.1 pp, low confidence
15Muse Spark 1.390.7%estimated ± 5.1 pp, low confidence
16Hy490.5%estimated ± 5.1 pp, medium confidence
17Claude Haiku 5.590.4%estimated ± 2.2 pp, medium confidence
18Claude Sonnet 5.590.3%estimated ± 2.2 pp, medium confidence
19Gemini 4 Argon90.2%estimated ± 4.5 pp, medium confidence
20Pareto 26.990.0%estimated ± 5.1 pp, medium confidence
21SWE-2 Flash89.9%estimated ± 2.2 pp, medium confidence
22Grok 4.689.9%estimated ± 4.4 pp, high confidence
23MAI Code 289.7%estimated ± 5.1 pp, medium confidence
24Particle-189.4%estimated ± 5.1 pp, medium confidence
25GPT-6.1 Sol89.0%estimated ± 5.1 pp, medium confidence
26MiMo-V2.6-Pro89.0%estimated ± 5.1 pp, medium confidence
27Gemini 3.7 Flash89.0%estimated ± 2.2 pp, medium confidence
28Kimi K388.7%estimated ± 4.4 pp, high confidence
29GPT-5.588.7%estimated ± 2.2 pp, medium confidence
30Grok 4.788.6%estimated ± 5.1 pp, medium confidence
31Claude Opus 4.888.6%measured
32SWE-1.788.3%estimated ± 2.2 pp, medium confidence
33Pareto 26.10 Preview88.1%estimated ± 5.1 pp, medium confidence
34Ling 3.1 Flash87.8%estimated ± 3.6 pp, high confidence
35GPT-6.1 Terra87.8%estimated ± 6.3 pp, medium confidence
36Claude Opus 4.787.6%measured
37GPT-6 Sol87.5%estimated ± 5.1 pp, medium confidence
38Mistral Large 487.3%estimated ± 4.5 pp, medium confidence
39MiMo-V2.6-Flash87.1%estimated ± 5.1 pp, medium confidence
40Step 5 Preview87.0%estimated ± 5.1 pp, medium confidence
41GPT-6 Luna86.5%estimated ± 5.1 pp, medium confidence
42Ornith-1.5-397B86.0%measured
43Hy4 flash85.6%estimated ± 5.1 pp, medium confidence
44Muse Spark 1.285.3%estimated ± 4.4 pp, high confidence
45Claude Sonnet 585.2%measured
46GPT-5.3 Codex85.0%measured
47Grok 4.584.5%estimated ± 3.6 pp, high confidence
48GPT-5.6 Sol84.5%estimated ± 3.6 pp, high confidence
49Ultramarine 1 Flash84.2%estimated ± 5.1 pp, medium confidence
50Muse Spark Open83.6%estimated ± 5.1 pp, medium confidence
51MAI Code 2 Flash83.6%estimated ± 5.1 pp, medium confidence
52GPT-5.6 Terra83.5%estimated ± 3.6 pp, high confidence
53Qwen3.8 Max Preview83.4%estimated ± 6.3 pp, medium confidence
54Composer 2 Fast83.2%estimated ± 5.0 pp, low confidence
55Gemini 3.1 Pro82.9%estimated ± 3.2 pp, medium confidence
56GPT-5.6 Luna82.9%estimated ± 3.6 pp, high confidence
57Ornith-1.0-397B82.4%measured
58GPT-5.482.4%estimated ± 3.2 pp, medium confidence
59DeepSeek V4.1 Flash 101782.3%estimated ± 2.3 pp, medium confidence
60Inkling 2 Small82.2%measured
61ERNIE 5.5 Flash Preview82.2%estimated ± 2.3 pp, medium confidence
62GPT-5.1-Codex-Max82.1%estimated ± 3.2 pp, medium confidence
63Muse Spark 1.182.0%estimated ± 3.6 pp, high confidence
64MiniMax M3.1 Flash82.0%estimated ± 2.3 pp, high confidence
65LongCat-2.581.9%estimated ± 2.3 pp, high confidence
66Gemini 3.8 Flash81.9%estimated ± 4.4 pp, high confidence
67Hy4 preview81.8%estimated ± 2.3 pp, high confidence
68GLM-5.381.7%estimated ± 2.3 pp, high confidence
69Sage-A1.581.7%estimated ± 3.6 pp, high confidence
70Composer 2.581.7%estimated ± 4.4 pp, high confidence
71Gemini 3.6 Flash81.7%estimated ± 4.4 pp, high confidence
72Beam 1.0 501B-A23B81.6%measured
73GLM-5.3-Flash81.6%estimated ± 2.3 pp, high confidence
74Qwen3.8 Max81.6%estimated ± 2.3 pp, high confidence
75Gemini 3 Pro81.5%estimated ± 3.2 pp, medium confidence
76o381.4%estimated ± 13.1 pp, low confidence
77GPT-5.2-Codex81.1%estimated ± 3.2 pp, medium confidence
78Claude Opus 4.6 (Adaptive)80.9%estimated ± 4.4 pp, high confidence
79Kimi K2.7 Code80.9%estimated ± 4.4 pp, high confidence
80Claude Opus 4.580.9%measured
81Claude Opus 4.680.8%measured
82GLM-5.280.7%estimated ± 2.3 pp, high confidence
83Qwen3.8-Omni-Flash80.7%estimated ± 2.3 pp, high confidence
84Beam 1.0 501B-A23B NVFP480.7%measured
85Qwen3.8-Flash-Next80.6%estimated ± 2.3 pp, high confidence
86DeepSeek V4 Pro80.6%measured
87MiniMax M380.5%measured
88GPT-5.4 mini80.5%estimated ± 2.2 pp, medium confidence
89Atria Dawn Preview80.4%estimated ± 3.6 pp, high confidence
90Seed 2.1 Pro80.4%estimated ± 2.3 pp, high confidence
91Qwen3.7 Max80.4%measured
92Laguna S 2.180.3%estimated ± 3.6 pp, high confidence
93Inkling-Small80.2%measured
94Kimi K2.680.2%measured
95Beam 1.0 501B-A23B FP880.2%measured
96Claude Opus 4.5 Thinking80.0%estimated ± 4.4 pp, high confidence
97GPT-5.280.0%measured
98Sakana Fugu80.0%estimated ± 3.6 pp, high confidence
99Mercury 380.0%estimated ± 5.1 pp, medium confidence
100Seed 2.1 Turbo79.7%estimated ± 2.3 pp, high confidence
101Claude Sonnet 4.679.6%measured
102Qwen 3.6 Max (preview)79.5%estimated ± 2.3 pp, high confidence
103GLM-5.179.5%estimated ± 2.3 pp, high confidence
104Qwen3.8-27B79.4%estimated ± 2.3 pp, high confidence
105Gemini 3 Flash79.3%estimated ± 4.4 pp, high confidence
106Inkling 2 Mini79.2%measured
107DeepSeek V4 Flash 073179.0%measured
108Ornith-1.5-35B-A3B79.0%measured
109Qwen3.6 Plus78.8%measured
110MiniMax M2.778.6%estimated ± 2.3 pp, high confidence
111o4-mini78.6%estimated ± 13.1 pp, low confidence
112MiMo-V2.5-Pro78.5%estimated ± 3.6 pp, high confidence
113BTL-478.4%measured
114dots3-note Preview78.4%measured
115Hy378.4%estimated ± 6.3 pp, medium confidence
116Ling 3.0 Flash78.0%estimated ± 3.6 pp, high confidence
117DeepSeek V3.278.0%estimated ± 4.8 pp, medium confidence
118MiMo-V2-Pro78.0%measured
119GLM-577.8%measured
120Step 3.7 Flash77.8%estimated ± 3.6 pp, high confidence
121Apodex 1.177.7%measured
122Qwen3.7 Plus77.7%measured
123Quasar 438B77.6%estimated ± 6.3 pp, medium confidence
124MiMo-V2.577.6%estimated ± 3.6 pp, high confidence
125Inkling77.6%measured
126Mistral Medium 3.5 128B77.6%measured
127Grok 4.377.4%estimated ± 4.4 pp, high confidence
128Muse Spark77.4%measured
129Claude Sonnet 4.577.2%measured
130Qwen3.6-27B77.2%measured
131Gemini 3.5 Flash76.8%estimated ± 3.6 pp, high confidence
132Kimi K2.576.8%measured
133Claude Sonnet 4.5 Thinking76.7%estimated ± 4.4 pp, high confidence
134Grok 4.2076.7%measured
135GPT-5.176.6%estimated ± 4.4 pp, high confidence
136GPT-5.4 nano76.6%estimated ± 4.4 pp, high confidence
137Ripple-65B Preview76.5%measured
138GPT-576.2%estimated ± 4.4 pp, high confidence
139Qwen3.5 397B76.2%measured
140Gemma 4 31B76.1%estimated ± 5.0 pp, medium confidence
141Gemini 3.5 Flash-Lite76.1%estimated ± 3.6 pp, high confidence
142Muse Glimmer 30B76.0%measured
143Composer 275.7%estimated ± 4.8 pp, medium confidence
144Ornith-1.0-35B75.6%measured
145MiMo-V2-Omni74.8%measured
146Laguna M.174.6%measured
147Claude 4.1 Opus74.5%measured
148Hy3 Preview74.4%measured
149Step 3.5 Flash74.4%measured
150Flint74.2%measured
151Qwen3.5 Plus74.2%estimated ± 4.5 pp, high confidence
152MiniMax M2.574.0%estimated ± 4.5 pp, high confidence
153Apodex 1.1 Mini73.9%estimated ± 6.3 pp, medium confidence
154Qwen3.5 Flash73.9%estimated ± 4.4 pp, high confidence
155GPT-5 mini73.9%estimated ± 4.5 pp, high confidence
156GLM-4.773.8%measured
157GPT-5.1-Codex73.6%estimated ± 4.5 pp, high confidence
158BTL-373.5%estimated ± 9.7 pp, low confidence
159MAI-Thinking-173.5%measured
160MiMo-V2-Flash73.4%measured
161Qwen3.6-35B-A3B73.4%measured
162Claude Haiku 4.573.3%measured
163Claude Haiku 4.5 Thinking73.3%estimated ± 4.5 pp, high confidence
164Gemini 3.1 Flash-Lite73.0%estimated ± 4.4 pp, high confidence
165Jais-3-MoVA-36B-A4B72.8%measured
166Claude 4 Sonnet72.7%measured
167MAI-Code-1.1-Flash72.6%measured
168Claude 4 Opus72.5%measured
169Jais-3-32B72.4%measured
170Qwen3.5-27B72.4%measured
171Ling 3.0 Flash VL72.0%estimated ± 6.3 pp, medium confidence
172Qwen3.5-122B-A10B72.0%measured
173Nemotron 3 Ultra71.9%measured
174Qwen3 Max71.6%estimated ± 4.5 pp, high confidence
175GLM-4.671.5%estimated ± 4.5 pp, high confidence
176Nemotron 3.5 Super VL 120B A12B71.1%measured
177Nemotron 3.5 Super VL 120B A12B NVFP471.1%measured
178Laguna XS 2.170.9%measured
179Grok Code Fast 170.8%measured
180Ornith-1.5-9B70.6%measured
181Solar Pro 470.6%measured
182Grok 470.5%estimated ± 4.4 pp, high confidence
183Solar Open 270.4%measured
184Jais-3-375B-A23B70.2%measured
185Jais-3-7B70.2%measured
186Laguna XS.269.9%measured
187Qwen3-Coder 480B-A35B69.6%measured
188A.X K269.5%estimated ± 5.6 pp, medium confidence
189Ornith-1.0-9B69.4%measured
190Qwen3.5-35B-A3B69.2%measured
191Ling 3.0 Flash FP868.4%estimated ± 5.6 pp, medium confidence
192K-EXAONE 2.068.2%measured
193LongCat-Flash-Lite-Sparse68.2%measured
194Trinity-Large-Preview66.8%estimated ± 6.3 pp, medium confidence
195Trinity-Large-Thinking66.8%estimated ± 6.3 pp, medium confidence
196o166.2%estimated ± 6.5 pp, medium confidence
197DeepSeek V3.166.0%measured
198Gemma 4 26B A4B65.9%estimated ± 6.3 pp, medium confidence
199Kimi K265.8%measured
200Nemotron 3.5 Bolt 74B-A9B65.5%estimated ± 6.3 pp, medium confidence
201DeepSeek V3 032464.5%estimated ± 6.3 pp, medium confidence
202Mistral Small 464.2%estimated ± 6.3 pp, medium confidence
203North Mini Code64.2%estimated ± 6.3 pp, medium confidence
204Grok 4 Fast (Reasoning)64.1%estimated ± 4.4 pp, medium confidence
205Mercury 2.564.0%estimated ± 5.6 pp, medium confidence
206Grok 3 Mini63.8%estimated ± 9.3 pp, low confidence
207Gemini 2.5 Pro63.8%measured
208Command A+63.8%estimated ± 6.3 pp, medium confidence
209Spark-X2.6-4B62.8%measured
210GPT-OSS 120B62.4%measured
211Claude 3.7 Sonnet62.3%measured
212Grok 4.1 Fast (Reasoning)62.0%estimated ± 4.4 pp, medium confidence
213Mistral Large 362.0%estimated ± 4.4 pp, medium confidence
214Ternary Bonsai 2 27B60.8%measured
215o1-preview60.8%estimated ± 6.5 pp, medium confidence
216GPT-OSS 20B60.7%measured
217Nemotron Ultra 253B60.5%estimated ± 9.3 pp, low confidence
218Nemotron 3 Super 100B60.5%measured
219GLM-4.7-Flash59.2%measured
220DeepSeek-R1-0528-Qwen3-8B58.0%estimated ± 9.3 pp, low confidence
221DeepSeek-R1-052857.6%measured
222Mellum2-12B-A2.5B-Thinking57.5%estimated ± 9.7 pp, low confidence
223ZAYA1-8B57.5%estimated ± 9.7 pp, low confidence
224Qwen3 235B 250757.5%estimated ± 9.7 pp, low confidence
225Magistral Medium57.5%estimated ± 9.7 pp, low confidence
226Magistral Small57.5%estimated ± 9.7 pp, low confidence
227LLaDA2.2-mini57.5%estimated ± 9.7 pp, low confidence
228Mellum2-12B-A2.5B-Instruct57.5%estimated ± 9.7 pp, low confidence
229MiniCPM5-1B57.5%estimated ± 9.7 pp, low confidence
230Mistral Medium 357.5%estimated ± 9.7 pp, low confidence
231Gemma 4 12B57.5%estimated ± 6.5 pp, medium confidence
232GLM-4.557.2%estimated ± 13.1 pp, low confidence
233Granite 4.2 30B57.0%measured
234DeepSeek R1 Distill Llama 70B56.6%estimated ± 9.3 pp, low confidence
235Grok 3 [Beta]56.3%estimated ± 9.3 pp, low confidence
236MiniMax M1 80k56.0%measured
237Phi-4-reasoning54.7%estimated ± 9.3 pp, low confidence
238GPT-4.154.6%measured
239DeepSeek R1 Distill Qwen 14B54.4%estimated ± 9.3 pp, low confidence
240Phi-4-reasoning-plus54.4%estimated ± 9.3 pp, low confidence
241Llama 4 Maverick54.0%estimated ± 6.3 pp, medium confidence
242ZAYA1-74B-Preview53.2%measured
243Nemotron 3.5 Lightning 30B A3B NVFP452.8%measured
244Nemotron 3 Nano Omni 30B A3B51.8%estimated ± 5.6 pp, medium confidence
245LFM2.5-24B-A2B51.0%measured
246K-Exaone49.4%measured
247o3-mini49.3%measured
248LLaDA2.2-flash49.3%measured
249DeepSeek-R149.2%measured
250Claude 3.5 Sonnet49.0%measured
251Claude 3.7 Sonnet Thinking48.7%estimated ± 13.1 pp, low confidence
252Gemini 1.5 Pro48.5%estimated ± 6.5 pp, medium confidence
253Mistral Small 3.148.3%estimated ± 6.3 pp, medium confidence
254Granite 4.2 8B47.7%measured
255Agents-A1-4B47.5%estimated ± 5.6 pp, medium confidence
256Mellum2.1-12B-A2.5B-Thinking47.0%measured
257Devstral Small46.8%measured
258MiniCPM5-2B46.4%measured
259Qwen2.5-Max45.7%estimated ± 9.3 pp, low confidence
260GPT-4 Turbo45.5%estimated ± 6.5 pp, medium confidence
261Solar Pro 345.0%estimated ± 6.3 pp, low confidence
262Sarvam 105B45.0%measured
263Ling 2.6 Flash43.7%estimated ± 5.6 pp, medium confidence
264Gemma 4 E4B43.4%estimated ± 6.3 pp, low confidence
265LFM3-1.7B43.4%measured
266Ling 3.0 Tiny43.1%estimated ± 6.3 pp, low confidence
267Claude 3 Opus42.6%estimated ± 6.5 pp, low confidence
268DeepSeek V342.0%measured
269Gemma 3 27B41.8%estimated ± 6.3 pp, low confidence
270Granite 4.2 3B40.8%estimated ± 5.6 pp, low confidence
271Claude 3.5 Haiku40.6%measured
272Celeris-139.4%estimated ± 6.3 pp, low confidence
273Llama 4 Scout38.9%estimated ± 6.3 pp, low confidence
274Nemotron 3 Nano 30B34.0%estimated ± 6.5 pp, low confidence
275Sarvam 30B34.0%measured
276Gemini 2.5 Flash-Lite31.6%measured
277GPT-4o31.2%estimated ± 13.1 pp, low confidence
278Ultravox v0.6 Llama 3.3 70B29.4%estimated ± 6.5 pp, low confidence
279LFM2.5-2.6B29.0%estimated ± 6.3 pp, low confidence
280GPT-4o mini28.3%estimated ± 6.5 pp, low confidence
281GPT-4.1 nano27.9%estimated ± 6.5 pp, low confidence
282GPT-4.1 mini23.6%measured
283Gemini 2.0 Flash21.4%measured
284Gemma 4 E2B19.4%estimated ± 6.5 pp, low confidence
285Gemini 2.0 Flash-Lite12.5%measured
286Gemini 1.5 Flash9.6%measured

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General