benchgap
Coding

SWE-bench Pro leaderboard

As of 2026-11-21, the highest measured score on SWE-bench Pro is 93.0% by Claude Fable 5.2. 191 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#ModelScoreSource
1Claude Fable 5.293.0%measured
2Gemini 4 Carbon92.1%estimated ± 4.2 pp, medium confidence
3GPT-6.2 Astra90.0%estimated ± 4.3 pp, high confidence
4Claude Opus 5.589.9%measured
5GPT-6 Astra87.2%estimated ± 4.3 pp, high confidence
6Gemini 4 Neon86.1%estimated ± 4.8 pp, high confidence
7Pareto 26.985.7%estimated ± 7.7 pp, medium confidence
8Gemini 4 Hydrogen84.2%estimated ± 4.2 pp, high confidence
9MAI Code 282.8%estimated ± 7.7 pp, medium confidence
10GPT-6.1 Sol81.4%estimated ± 4.3 pp, high confidence
11Claude Sonnet 5.581.3%measured
12Claude Fable 5.181.2%measured
13Muse Wonder Full80.7%estimated ± 6.3 pp, medium confidence
14Particle-180.6%estimated ± 7.7 pp, medium confidence
15Claude Mythos 580.3%measured
16Claude Fable 580.0%measured
17Claude Opus 579.2%measured
18Muse Wonder78.6%estimated ± 4.2 pp, high confidence
19MiMo-V2.6-Pro77.5%estimated ± 6.3 pp, medium confidence
20Qwen4 Max77.4%estimated ± 4.3 pp, high confidence
21SWE-277.3%estimated ± 6.8 pp, medium confidence
22Kimi K3.1 Pro77.0%estimated ± 4.3 pp, high confidence
23Claude Haiku 5.575.7%estimated ± 4.2 pp, high confidence
24Hy474.5%estimated ± 4.3 pp, high confidence
25Gemini 3.7 Flash73.9%estimated ± 4.7 pp, high confidence
26Step 5 Preview73.8%estimated ± 6.3 pp, medium confidence
27Sakana Fugu-Ultra73.7%measured
28Gemini 3.8 Flash73.4%estimated ± 4.5 pp, high confidence
29Pareto 26.10 Preview73.0%estimated ± 7.7 pp, medium confidence
30Muse Spark Open72.9%estimated ± 6.3 pp, medium confidence
31Grok 4.772.8%estimated ± 4.2 pp, high confidence
32Ember-172.0%estimated ± 4.5 pp, high confidence
33GPT-6 Sol71.5%estimated ± 6.3 pp, medium confidence
34SWE-2 Flash69.8%estimated ± 6.8 pp, medium confidence
35Claude Opus 4.869.2%measured
36Inkling 2 Small68.9%estimated ± 4.3 pp, high confidence
37Ling 3.1 Flash68.7%measured
38Kimi K368.3%estimated ± 4.3 pp, high confidence
39Qwen3.8 Max67.7%measured
40Muse Spark 1.367.2%estimated ± 4.2 pp, high confidence
41GPT-6.1 Terra67.2%estimated ± 6.3 pp, medium confidence
42Grok 4.667.0%estimated ± 4.2 pp, high confidence
43GPT-6 Luna66.3%estimated ± 6.3 pp, medium confidence
44Beam 1.0 501B-A23B66.1%measured
45Beam 1.0 501B-A23B FP865.9%measured
46Hy4 preview65.7%measured
47Beam 1.0 501B-A23B NVFP465.6%measured
48Hy4 flash65.5%estimated ± 7.7 pp, medium confidence
49GPT-6.2 Sol65.5%estimated ± 8.1 pp, medium confidence
50Ornith-1.5-397B65.1%measured
51SWE-1.765.0%estimated ± 6.8 pp, medium confidence
52Grok 4.564.7%measured
53GPT-5.6 Sol64.6%measured
54Qwen3.8 Max Preview64.4%estimated ± 6.0 pp, medium confidence
55Claude Opus 4.764.3%measured
56Ultramarine 1 Flash63.9%estimated ± 7.7 pp, medium confidence
57MAI Code 2 Flash63.5%estimated ± 7.7 pp, medium confidence
58GPT-5.6 Terra63.4%measured
59Qwen3.8-Omni-Flash63.3%measured
60Claude Sonnet 563.2%measured
61GPT-5.6 Luna62.7%measured
62DeepSeek V4.1 Flash 101762.7%estimated ± 3.5 pp, medium confidence
63Mercury 362.6%estimated ± 7.7 pp, medium confidence
64ERNIE 5.5 Flash Preview62.6%estimated ± 3.5 pp, medium confidence
65Qwen3.8-Flash-Next62.5%measured
66MiniMax M3.1 Flash62.4%estimated ± 3.5 pp, high confidence
67LongCat-2.562.4%estimated ± 3.5 pp, high confidence
68Ornith-1.0-397B62.2%measured
69GLM-5.362.2%estimated ± 3.5 pp, high confidence
70GLM-5.262.1%measured
71GLM-5.3-Flash62.1%estimated ± 3.5 pp, high confidence
72Qwen3.8-27B61.7%measured
73Muse Spark 1.161.5%measured
74Ripple-65B Preview61.4%measured
75Sage-A1.561.2%measured
76dots3-note Preview61.0%measured
77MiMo-V2.6-Flash60.8%estimated ± 6.3 pp, medium confidence
78Qwen3.7 Max60.6%measured
79Seed 2.1 Pro60.4%estimated ± 3.5 pp, high confidence
80Gemini 4 Argon60.2%estimated ± 2.9 pp, low confidence
81Muse Spark 1.259.8%estimated ± 4.8 pp, high confidence
82Atria Dawn Preview59.6%measured
83Ornith-1.5-35B-A3B59.6%measured
84Laguna S 2.159.4%measured
85Composer 2.559.4%estimated ± 4.2 pp, medium confidence
86Seed 2.1 Turbo59.3%estimated ± 3.5 pp, high confidence
87Mistral Large 459.1%estimated ± 2.9 pp, low confidence
88MiniMax M359.0%measured
89Sakana Fugu59.0%measured
90Quasar 438B58.9%estimated ± 6.0 pp, medium confidence
91Apodex 1.1 Mini58.7%estimated ± 6.0 pp, medium confidence
92GPT-5.558.6%measured
93Kimi K2.658.6%measured
94GLM-5.158.4%measured
95GPT-5.457.7%measured
96Inkling 2 Mini57.6%estimated ± 4.5 pp, high confidence
97Qwen3.7 Plus57.6%measured
98Hy357.6%estimated ± 6.0 pp, medium confidence
99Qwen 3.6 Max (preview)57.3%measured
100Grok 4.357.2%estimated ± 4.7 pp, high confidence
101o357.2%estimated ± 4.7 pp, high confidence
102Qwen3.5 Flash57.2%estimated ± 4.7 pp, high confidence
103Grok 457.2%estimated ± 4.7 pp, high confidence
104o4-mini57.2%estimated ± 4.7 pp, high confidence
105Claude 3.7 Sonnet Thinking57.2%estimated ± 4.7 pp, medium confidence
106GLM-4.557.2%estimated ± 4.7 pp, medium confidence
107Mistral Large 357.2%estimated ± 4.7 pp, medium confidence
108o157.2%estimated ± 4.7 pp, medium confidence
109MiMo-V2.5-Pro57.2%measured
110Claude Opus 4.6 (Adaptive)57.2%estimated ± 2.9 pp, medium confidence
111Claude Opus 4.557.1%measured
112Gemini 3.6 Flash57.0%estimated ± 4.5 pp, high confidence
113Claude Sonnet 4.657.0%estimated ± 2.9 pp, medium confidence
114GPT-5.3 Codex56.8%measured
115GPT-5.4 mini56.8%estimated ± 2.9 pp, medium confidence
116BTL-456.7%estimated ± 4.5 pp, high confidence
117Ling 3.0 Flash56.6%measured
118Qwen3.6 Plus56.6%measured
119Step 3.7 Flash56.3%measured
120MiMo-V2-Pro56.3%estimated ± 4.5 pp, high confidence
121MiniMax M2.756.2%measured
122MiMo-V2.556.1%measured
123GPT-5.2-Codex56.0%estimated ± 2.9 pp, medium confidence
124Apodex 1.155.9%estimated ± 4.5 pp, high confidence
125Inkling-Small55.9%measured
126Mistral Medium 3.5 128B55.8%estimated ± 4.5 pp, high confidence
127BTL-355.6%estimated ± 6.0 pp, medium confidence
128GPT-5.255.6%measured
129Gemini 3.1 Pro55.5%estimated ± 2.9 pp, medium confidence
130DeepSeek V4 Pro55.4%measured
131Claude Sonnet 4.555.4%estimated ± 4.5 pp, high confidence
132Gemini 3.5 Flash55.1%measured
133GLM-555.1%measured
134GPT-5.4 nano55.0%estimated ± 2.9 pp, medium confidence
135GPT-5.154.9%estimated ± 2.9 pp, medium confidence
136Kimi K2.7 Code54.8%estimated ± 4.5 pp, high confidence
137Claude Sonnet 4.5 Thinking54.8%estimated ± 2.9 pp, medium confidence
138GPT-5.1-Codex-Max54.7%estimated ± 2.9 pp, medium confidence
139Claude Opus 4.5 Thinking54.6%estimated ± 2.9 pp, medium confidence
140Gemini 3 Flash54.6%estimated ± 2.9 pp, medium confidence
141GPT-554.6%estimated ± 2.9 pp, medium confidence
142Inkling54.3%measured
143Spark-X2.6-4B54.3%measured
144Qwen3.5 Plus54.2%estimated ± 2.9 pp, medium confidence
145Gemini 3.5 Flash-Lite54.2%measured
146MiniMax M2.554.2%estimated ± 2.9 pp, medium confidence
147Gemini 3 Pro54.1%estimated ± 2.9 pp, medium confidence
148GPT-5 mini54.1%estimated ± 2.9 pp, medium confidence
149GPT-5.1-Codex54.0%estimated ± 2.9 pp, medium confidence
150Claude Haiku 4.5 Thinking53.9%estimated ± 2.9 pp, medium confidence
151Qwen3.6-27B53.5%measured
152Claude Opus 4.653.4%measured
153Qwen3 Max53.3%estimated ± 2.9 pp, low confidence
154GLM-4.653.2%estimated ± 2.9 pp, low confidence
155Grok 4.1 Fast (Reasoning)53.1%estimated ± 2.9 pp, low confidence
156Gemini 2.5 Pro53.0%estimated ± 2.9 pp, low confidence
157Gemini 3.1 Flash-Lite53.0%estimated ± 2.9 pp, low confidence
158Grok 4 Fast (Reasoning)53.0%estimated ± 2.9 pp, low confidence
159MAI-Thinking-152.8%measured
160MiMo-V2-Omni52.7%estimated ± 4.5 pp, high confidence
161DeepSeek V4 Flash 073152.6%measured
162Muse Spark52.4%measured
163Claude 4.1 Opus52.4%estimated ± 4.5 pp, high confidence
164Hy3 Preview52.3%estimated ± 4.5 pp, high confidence
165Step 3.5 Flash52.3%estimated ± 4.5 pp, high confidence
166Flint52.1%estimated ± 4.5 pp, high confidence
167Grok 4.2051.8%measured
168Nemotron 3.5 Super VL 120B A12B51.8%measured
169GLM-4.751.6%estimated ± 4.5 pp, high confidence
170Nemotron 3.5 Super VL 120B A12B NVFP451.3%measured
171Muse Glimmer 30B51.2%measured
172MiMo-V2-Flash51.2%estimated ± 4.5 pp, high confidence
173Claude Haiku 4.551.1%estimated ± 4.5 pp, high confidence
174Qwen3.5 397B50.9%measured
175Kimi K2.550.7%measured
176Jais-3-MoVA-36B-A4B50.5%estimated ± 4.5 pp, high confidence
177Claude 4 Sonnet50.4%estimated ± 4.5 pp, high confidence
178Ornith-1.0-35B50.4%measured
179MAI-Code-1.1-Flash50.3%estimated ± 4.5 pp, high confidence
180Claude 4 Opus50.2%estimated ± 4.5 pp, high confidence
181Jais-3-32B50.1%estimated ± 4.5 pp, high confidence
182Qwen3.5-27B50.1%estimated ± 4.5 pp, high confidence
183Ling 3.0 Flash VL49.8%estimated ± 6.3 pp, medium confidence
184Qwen3.5-122B-A10B49.6%estimated ± 4.5 pp, high confidence
185Nemotron 3 Ultra49.5%estimated ± 4.5 pp, high confidence
186Qwen3.6-35B-A3B49.5%measured
187Laguna M.149.2%measured
188Ling 3.0 Flash FP849.1%estimated ± 10.1 pp, low confidence
189Grok Code Fast 148.3%estimated ± 4.5 pp, high confidence
190Solar Pro 448.1%estimated ± 4.5 pp, high confidence
191Solar Open 247.9%estimated ± 4.5 pp, high confidence
192Jais-3-375B-A23B47.6%estimated ± 4.5 pp, high confidence
193Jais-3-7B47.6%estimated ± 4.5 pp, high confidence
194Laguna XS 2.147.6%measured
195Ornith-1.5-9B47.5%measured
196A.X K247.1%estimated ± 6.0 pp, medium confidence
197Qwen3-Coder 480B-A35B47.0%estimated ± 4.5 pp, high confidence
198Qwen3.5-35B-A3B46.5%estimated ± 4.5 pp, high confidence
199Laguna XS.246.3%measured
200K-EXAONE 2.045.4%estimated ± 4.5 pp, high confidence
201Nemotron 3.5 Bolt 74B-A9B43.3%estimated ± 6.3 pp, medium confidence
202DeepSeek V3.143.0%estimated ± 4.5 pp, high confidence
203Ornith-1.0-9B42.9%measured
204Kimi K242.8%estimated ± 4.5 pp, high confidence
205DeepSeek V3 032442.3%estimated ± 6.3 pp, medium confidence
206North Mini Code42.1%estimated ± 6.3 pp, medium confidence
207Mercury 2.541.6%estimated ± 6.3 pp, medium confidence
208LongCat-Flash-Lite-Sparse40.6%measured
209o1-preview40.4%estimated ± 6.0 pp, medium confidence
210GPT-OSS 120B39.0%estimated ± 4.5 pp, high confidence
211Claude 3.7 Sonnet38.9%estimated ± 4.5 pp, high confidence
212Gemma 4 31B38.9%estimated ± 6.0 pp, medium confidence
213Ternary Bonsai 2 27B37.2%estimated ± 4.5 pp, high confidence
214GPT-OSS 20B37.1%estimated ± 4.5 pp, high confidence
215Nemotron 3 Super 100B36.9%estimated ± 4.5 pp, high confidence
216GLM-4.7-Flash35.5%estimated ± 4.5 pp, high confidence
217Command A+34.9%estimated ± 6.0 pp, medium confidence
218Mistral Small 433.7%estimated ± 6.0 pp, medium confidence
219DeepSeek-R1-052833.7%estimated ± 4.5 pp, high confidence
220Granite 4.2 30B33.3%measured
221Trinity-Large-Preview32.9%estimated ± 6.0 pp, medium confidence
222Trinity-Large-Thinking32.9%estimated ± 6.0 pp, medium confidence
223Gemma 4 26B A4B32.8%estimated ± 6.0 pp, medium confidence
224Ling 2.6 Flash32.4%estimated ± 6.0 pp, medium confidence
225MiniMax M1 80k31.9%estimated ± 4.5 pp, high confidence
226Gemini 1.5 Pro30.8%estimated ± 6.0 pp, medium confidence
227GPT-4.130.4%estimated ± 4.5 pp, high confidence
228LLaDA2.2-flash30.1%measured
229ZAYA1-74B-Preview28.8%estimated ± 4.5 pp, high confidence
230GPT-4 Turbo28.6%estimated ± 6.0 pp, low confidence
231Nemotron 3.5 Lightning 30B A3B NVFP428.4%estimated ± 4.5 pp, high confidence
232Mellum2.1-12B-A2.5B-Thinking28.0%measured
233Mistral Small 3.127.8%estimated ± 6.3 pp, medium confidence
234Claude 3 Opus26.4%estimated ± 6.0 pp, low confidence
235LFM2.5-24B-A2B26.4%estimated ± 4.5 pp, high confidence
236Solar Pro 325.0%estimated ± 6.3 pp, low confidence
237K-Exaone24.6%estimated ± 4.5 pp, high confidence
238o3-mini24.5%estimated ± 4.5 pp, high confidence
239DeepSeek-R124.4%estimated ± 4.5 pp, high confidence
240Claude 3.5 Sonnet24.2%estimated ± 4.5 pp, high confidence
241Ling 3.0 Tiny23.5%estimated ± 6.3 pp, low confidence
242Llama 4 Maverick22.8%estimated ± 6.0 pp, low confidence
243Gemma 4 12B22.3%estimated ± 6.0 pp, medium confidence
244Devstral Small21.7%estimated ± 4.5 pp, high confidence
245Celeris-120.5%estimated ± 6.0 pp, low confidence
246Nemotron 3 Nano Omni 30B A3B19.7%estimated ± 6.0 pp, low confidence
247Sarvam 105B19.7%estimated ± 4.5 pp, medium confidence
248Granite 4.2 8B19.1%measured
249LFM3-1.7B18.0%estimated ± 4.5 pp, medium confidence
250Mellum2-12B-A2.5B-Thinking17.9%estimated ± 6.0 pp, medium confidence
251Granite 4.2 3B17.5%estimated ± 6.0 pp, medium confidence
252Ultravox v0.6 Llama 3.3 70B17.4%estimated ± 6.0 pp, low confidence
253GPT-4o mini16.7%estimated ± 6.0 pp, low confidence
254GPT-4.1 nano16.4%estimated ± 6.0 pp, low confidence
255DeepSeek V316.4%estimated ± 4.5 pp, medium confidence
256Gemma 3 27B15.0%estimated ± 6.0 pp, low confidence
257Claude 3.5 Haiku14.9%estimated ± 4.5 pp, medium confidence
258Nemotron 3 Nano 30B14.6%estimated ± 6.0 pp, low confidence
259MiniCPM5-2B14.4%measured
260Llama 4 Scout12.4%estimated ± 6.0 pp, low confidence
261Gemma 4 E2B11.1%estimated ± 6.0 pp, low confidence
262ZAYA1-8B9.4%estimated ± 6.0 pp, low confidence
263Sarvam 30B7.5%estimated ± 4.5 pp, medium confidence
264Gemini 2.5 Flash-Lite4.9%estimated ± 4.5 pp, medium confidence
265Agents-A1-4B0.0%estimated ± 6.0 pp, low confidence
266Gemini 1.5 Flash0.0%estimated ± 4.5 pp, medium confidence
267Gemini 2.0 Flash0.0%estimated ± 4.5 pp, medium confidence
268Gemini 2.0 Flash-Lite0.0%estimated ± 4.5 pp, medium confidence
269Gemma 4 E4B0.0%estimated ± 6.0 pp, low confidence
270GPT-4.1 mini0.0%estimated ± 4.5 pp, medium confidence
271LFM2.5-2.6B0.0%estimated ± 6.0 pp, low confidence
272Magistral Medium0.0%estimated ± 6.0 pp, low confidence
273Magistral Small0.0%estimated ± 6.0 pp, low confidence
274Qwen3 235B 25070.0%estimated ± 6.0 pp, low confidence

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General