benchgap
Coding

DeepSWE leaderboard

As of 2026-11-21, the highest measured score on DeepSWE is 79.0% by Muse Wonder Full. 153 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#ModelScoreSource
1Muse Wonder Full79.0%measured
2Muse Wonder78.8%measured
3Gemini 4 Argon77.9%measured
4GPT-6.2 Astra77.6%estimated ± 6.9 pp, low confidence
5DeepSeek V4.1 Flash 101777.0%measured
6Qwen4 Max75.7%measured
7Gemini 4 Carbon75.6%estimated ± 4.1 pp, low confidence
8Kimi K3.1 Pro75.5%measured
9Muse Spark 1.375.4%measured
10Ember-175.2%measured
11Hy474.9%measured
12Claude Fable 5.274.3%measured
13Claude Opus 5.574.2%measured
14GPT-6 Astra74.1%measured
15Pareto 26.974.0%measured
16Gemini 4 Hydrogen74.0%estimated ± 4.1 pp, medium confidence
17Gemini 3.8 Flash73.8%measured
18MiniMax M3.1 Flash73.7%measured
19Gemini 4 Neon73.5%estimated ± 4.3 pp, high confidence
20MAI Code 273.2%measured
21SWE-273.0%measured
22GLM-5.272.9%estimated ± 3.8 pp, medium confidence
23Kimi K2.7 Code72.9%estimated ± 3.8 pp, medium confidence
24GPT-5.6 Sol72.7%measured
25GPT-6.1 Terra72.6%estimated ± 6.5 pp, medium confidence
26Particle-172.6%measured
27Claude Fable 572.3%estimated ± 4.3 pp, high confidence
28GPT-6.2 Sol72.2%estimated ± 6.5 pp, medium confidence
29Claude Haiku 5.572.2%estimated ± 4.1 pp, medium confidence
30Inkling 2 Small72.1%measured
31GPT-6.1 Sol71.9%measured
32MiMo-V2.6-Pro71.9%measured
33ERNIE 5.5 Flash Preview71.2%measured
34Claude Sonnet 5.571.0%measured
35Grok 4.771.0%measured
36Pareto 26.10 Preview69.9%measured
37GPT-5.6 Terra69.6%measured
38Claude Opus 568.8%measured
39GPT-6 Sol68.8%measured
40MiMo-V2.6-Flash67.9%measured
41Step 5 Preview67.7%measured
42Kimi K367.5%measured
43Claude Fable 5.167.4%measured
44GPT-5.6 Luna67.2%measured
45GLM-5.366.9%measured
46GPT-6 Luna66.6%measured
47Sakana Fugu66.4%estimated ± 8.4 pp, low confidence
48Claude Opus 4.866.2%estimated ± 4.3 pp, high confidence
49Grok 4.665.9%measured
50Claude Sonnet 565.9%estimated ± 4.1 pp, low confidence
51Gemini 3.7 Flash65.3%measured
52Hy4 flash64.6%measured
53LongCat-2.564.6%measured
54Claude Mythos 564.6%estimated ± 11.2 pp, low confidence
55Hy4 preview64.3%measured
56Sakana Fugu-Ultra63.8%estimated ± 8.4 pp, low confidence
57GLM-5.3-Flash63.4%measured
58SWE-1.763.3%estimated ± 4.5 pp, low confidence
59DeepSeek V4 Pro62.7%measured
60Claude Opus 4.762.1%estimated ± 4.3 pp, high confidence
61Composer 2.561.7%estimated ± 4.1 pp, low confidence
62Ultramarine 1 Flash61.5%measured
63GPT-5.560.7%estimated ± 4.3 pp, high confidence
64SWE-2 Flash60.1%measured
65Muse Spark Open60.0%measured
66MAI Code 2 Flash59.9%measured
67Ling 3.1 Flash59.7%measured
68Muse Spark 1.259.3%measured
69Qwen3.8 Max Preview59.3%estimated ± 6.4 pp, medium confidence
70Qwen3.8-Flash-Next58.7%measured
71Qwen3.8-Omni-Flash57.8%measured
72Qwen3.8 Max56.6%measured
73Inkling-Small56.3%estimated ± 7.2 pp, low confidence
74Claude Opus 4.6 (Adaptive)56.3%estimated ± 7.2 pp, low confidence
75GPT-5.3 Codex56.3%estimated ± 7.2 pp, low confidence
76Claude Sonnet 4.656.3%estimated ± 7.2 pp, low confidence
77Claude Opus 4.5 Thinking56.3%estimated ± 7.2 pp, low confidence
78Gemini 3 Pro56.3%estimated ± 7.2 pp, low confidence
79GPT-5.256.3%estimated ± 7.2 pp, low confidence
80Gemini 3.5 Flash-Lite56.3%estimated ± 7.2 pp, low confidence
81Gemini 3 Flash56.3%estimated ± 7.2 pp, low confidence
82Claude Haiku 4.556.3%estimated ± 7.2 pp, low confidence
83Claude Sonnet 4.5 Thinking56.3%estimated ± 7.2 pp, low confidence
84Gemini 3.1 Flash-Lite56.3%estimated ± 7.2 pp, low confidence
85GLM-4.756.3%estimated ± 7.2 pp, low confidence
86GPT-556.3%estimated ± 7.2 pp, low confidence
87GPT-5.156.3%estimated ± 7.2 pp, low confidence
88GPT-5.2-Codex56.3%estimated ± 7.2 pp, low confidence
89Grok 4.2056.3%estimated ± 7.2 pp, low confidence
90Grok 4.356.3%estimated ± 7.2 pp, low confidence
91Ling 3.0 Flash56.3%estimated ± 7.2 pp, low confidence
92MiMo-V2.556.3%estimated ± 7.2 pp, low confidence
93MiniMax M2.756.3%estimated ± 7.2 pp, low confidence
94Mistral Medium 3.5 128B56.3%estimated ± 7.2 pp, low confidence
95Nemotron 3 Ultra56.3%estimated ± 7.2 pp, low confidence
96Qwen3.5 Flash56.3%estimated ± 7.2 pp, low confidence
97Qwen3.6-27B56.3%estimated ± 7.2 pp, low confidence
98Qwen 3.6 Max (preview)56.3%estimated ± 7.2 pp, low confidence
99Qwen3.6 Plus56.3%estimated ± 7.2 pp, low confidence
100Ornith-1.5-397B56.0%measured
101Gemini 3.5 Flash55.8%estimated ± 6.4 pp, medium confidence
102DeepSeek V4 Flash 073154.4%measured
103Gemini 3.1 Pro54.4%estimated ± 4.3 pp, medium confidence
104Muse Spark 1.153.3%measured
105Solar Pro 453.3%estimated ± 7.5 pp, low confidence
106Grok 4.553.0%measured
107Nemotron 3.5 Super VL 120B A12B52.8%estimated ± 7.5 pp, low confidence
108Nemotron 3.5 Super VL 120B A12B NVFP452.8%estimated ± 7.5 pp, low confidence
109Atria Dawn Preview52.3%estimated ± 11.2 pp, low confidence
110GPT-5.450.1%estimated ± 4.3 pp, medium confidence
111dots3-note Preview49.6%estimated ± 8.7 pp, medium confidence
112Gemini 2.5 Pro49.3%estimated ± 7.5 pp, low confidence
113Gemini 3.6 Flash49.0%measured
114Inkling 2 Mini48.2%measured
115Claude Opus 4.648.2%estimated ± 11.2 pp, low confidence
116MAI-Thinking-147.7%estimated ± 11.2 pp, low confidence
117Kimi K2.547.6%estimated ± 8.4 pp, low confidence
118Ornith-1.0-397B47.4%estimated ± 8.7 pp, medium confidence
119Gemma 4 31B47.2%estimated ± 7.5 pp, low confidence
120Qwen3.7 Max46.6%estimated ± 6.4 pp, low confidence
121Mercury 346.3%measured
122Seed 2.1 Pro45.8%estimated ± 8.7 pp, medium confidence
123Muse Glimmer 30B45.7%estimated ± 7.5 pp, low confidence
124Qwen3.5 397B45.4%estimated ± 7.5 pp, low confidence
125Laguna M.145.2%estimated ± 11.2 pp, low confidence
126BTL-445.0%estimated ± 12.7 pp, low confidence
127Ripple-65B Preview44.3%measured
128MiMo-V2-Pro44.2%estimated ± 12.7 pp, low confidence
129Laguna XS 2.144.1%estimated ± 11.2 pp, low confidence
130Ling 3.0 Flash VL43.8%estimated ± 7.5 pp, low confidence
131Beam 1.0 501B-A23B43.3%measured
132Laguna XS.243.1%estimated ± 11.2 pp, low confidence
133Beam 1.0 501B-A23B FP843.0%measured
134Step 3.7 Flash43.0%estimated ± 7.5 pp, low confidence
135Beam 1.0 501B-A23B NVFP442.7%measured
136Claude Sonnet 4.542.7%estimated ± 12.7 pp, low confidence
137Qwen3.8-27B42.2%measured
138Inkling42.1%estimated ± 4.3 pp, medium confidence
139Sage-A1.541.5%estimated ± 8.4 pp, low confidence
140Seed 2.1 Turbo41.5%estimated ± 8.7 pp, medium confidence
141Claude Opus 4.540.8%estimated ± 8.7 pp, medium confidence
142Laguna S 2.140.4%measured
143LongCat-Flash-Lite-Sparse38.9%estimated ± 11.2 pp, low confidence
144MiMo-V2-Omni38.1%estimated ± 12.7 pp, low confidence
145K-EXAONE 2.037.6%estimated ± 7.5 pp, low confidence
146Claude 4.1 Opus37.5%estimated ± 12.7 pp, low confidence
147Step 3.5 Flash37.3%estimated ± 12.7 pp, low confidence
148Kimi K2.637.0%estimated ± 6.4 pp, low confidence
149Flint36.9%estimated ± 12.7 pp, low confidence
150Ling 3.0 Flash FP836.5%estimated ± 8.4 pp, low confidence
151Quasar 438B35.7%estimated ± 6.4 pp, low confidence
152MiMo-V2-Flash35.4%estimated ± 12.7 pp, low confidence
153A.X K234.8%estimated ± 7.5 pp, low confidence
154Apodex 1.134.7%estimated ± 6.4 pp, low confidence
155Apodex 1.1 Mini34.7%estimated ± 6.4 pp, low confidence
156Jais-3-MoVA-36B-A4B34.3%estimated ± 12.7 pp, low confidence
157Claude 4 Sonnet34.1%estimated ± 12.7 pp, low confidence
158MAI-Code-1.1-Flash33.9%estimated ± 12.7 pp, low confidence
159Claude 4 Opus33.7%estimated ± 12.7 pp, low confidence
160Trinity-Large-Preview33.6%estimated ± 7.5 pp, low confidence
161Trinity-Large-Thinking33.6%estimated ± 7.5 pp, low confidence
162Jais-3-32B33.5%estimated ± 12.7 pp, low confidence
163Qwen3.5-27B33.5%estimated ± 12.7 pp, low confidence
164MiMo-V2.5-Pro33.4%estimated ± 6.4 pp, low confidence
165Spark-X2.6-4B32.2%estimated ± 8.4 pp, low confidence
166GLM-532.0%estimated ± 8.7 pp, low confidence
167Gemma 4 26B A4B31.9%estimated ± 7.5 pp, low confidence
168Nemotron 3.5 Bolt 74B-A9B31.0%estimated ± 7.5 pp, low confidence
169Qwen3.5-122B-A10B31.0%estimated ± 7.5 pp, low confidence
170Grok Code Fast 130.6%estimated ± 12.7 pp, low confidence
171Ornith-1.0-35B30.5%estimated ± 8.7 pp, low confidence
172Hy330.4%estimated ± 6.4 pp, low confidence
173Hy3 Preview30.4%estimated ± 6.4 pp, low confidence
174LLaDA2.2-flash30.3%estimated ± 11.2 pp, low confidence
175Muse Spark30.0%estimated ± 6.4 pp, low confidence
176MiniMax M329.9%estimated ± 6.4 pp, low confidence
177Solar Open 229.8%estimated ± 12.7 pp, low confidence
178Jais-3-375B-A23B29.5%estimated ± 12.7 pp, low confidence
179Jais-3-7B29.5%estimated ± 12.7 pp, low confidence
180DeepSeek V3 032429.0%estimated ± 7.5 pp, low confidence
181GPT-OSS 20B28.7%estimated ± 7.5 pp, low confidence
182Mellum2.1-12B-A2.5B-Thinking28.5%estimated ± 11.2 pp, low confidence
183Qwen3-Coder 480B-A35B28.4%estimated ± 12.7 pp, low confidence
184Mistral Small 428.4%estimated ± 7.5 pp, low confidence
185North Mini Code28.4%estimated ± 7.5 pp, low confidence
186Ornith-1.5-9B28.0%estimated ± 8.7 pp, low confidence
187Qwen3.5-35B-A3B27.7%estimated ± 12.7 pp, low confidence
188LFM2.5-24B-A2B27.6%estimated ± 8.4 pp, low confidence
189Command A+27.6%estimated ± 7.5 pp, low confidence
190Mercury 2.527.6%estimated ± 7.5 pp, low confidence
191Nemotron 3 Nano Omni 30B A3B26.8%estimated ± 8.4 pp, low confidence
192Granite 4.2 30B25.6%estimated ± 7.5 pp, low confidence
193GPT-5.4 mini24.7%estimated ± 6.4 pp, low confidence
194GPT-5.4 nano24.7%estimated ± 6.4 pp, low confidence
195Agents-A1-4B24.3%estimated ± 8.4 pp, low confidence
196Qwen3.7 Plus24.3%estimated ± 6.4 pp, low confidence
197GLM-5.124.1%estimated ± 6.4 pp, low confidence
198LFM3-1.7B23.2%estimated ± 8.4 pp, low confidence
199Ornith-1.0-9B22.4%estimated ± 8.7 pp, low confidence
200DeepSeek V3.122.4%estimated ± 12.7 pp, low confidence
201Mistral Large 322.3%estimated ± 7.5 pp, low confidence
202Qwen3.6-35B-A3B22.3%estimated ± 7.5 pp, low confidence
203Kimi K222.0%estimated ± 12.7 pp, low confidence
204Ornith-1.5-35B-A3B22.0%measured
205Ling 2.6 Flash21.7%estimated ± 8.4 pp, low confidence
206DeepSeek V320.3%estimated ± 7.5 pp, low confidence
207Granite 4.2 3B18.9%estimated ± 8.4 pp, low confidence
208Claude 3.7 Sonnet16.9%estimated ± 12.7 pp, low confidence
209GPT-OSS 120B15.9%estimated ± 7.5 pp, low confidence
210Ternary Bonsai 2 27B14.9%estimated ± 12.7 pp, low confidence
211Nemotron 3 Super 100B14.5%estimated ± 12.7 pp, low confidence
212GLM-4.7-Flash13.0%estimated ± 12.7 pp, low confidence
213Nemotron 3.5 Lightning 30B A3B NVFP412.0%estimated ± 7.5 pp, low confidence
214DeepSeek-R1-052811.2%estimated ± 12.7 pp, low confidence
215Llama 4 Maverick11.2%estimated ± 7.5 pp, low confidence
216Granite 4.2 8B10.9%estimated ± 7.5 pp, low confidence
217Mistral Small 3.15.5%estimated ± 7.5 pp, low confidence
218MiniCPM5-2B4.0%estimated ± 7.5 pp, low confidence
219Solar Pro 33.4%estimated ± 7.5 pp, low confidence

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General