Coding

DeepSWE leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

As of 2026-10-07, the highest measured score on DeepSWE is 77.9% by Gemini 4 Argon. 107 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	Gemini 4 Argon	77.9%	measured
2	Muse Spark 1.3	75.4%	measured
3	Ember-1	75.2%	measured
4	Claude Opus 5.5	74.2%	measured
5	DeepSeek V4.1 Flash	74.2%	measured
6	GPT-6 Astra	74.1%	measured
7	Pareto 26.9	74.0%	measured
8	Gemini 3.8 Flash	73.8%	measured
9	SWE-2	73.0%	measured
10	Claude Fable 5	72.9%	estimated ± 3.4 pp, medium confidence
11	GPT-5.6 Sol	72.7%	measured
12	GPT-6.1 Sol	71.9%	measured
13	MiMo-V2.6-Pro	71.9%	measured
14	Claude Sonnet 5.5	71.0%	measured
15	Grok 4.7	71.0%	measured
16	Pareto 26.10 Preview	69.9%	measured
17	GPT-5.6 Terra	69.6%	measured
18	Claude Opus 4.8	69.4%	estimated ± 3.4 pp, medium confidence
19	Claude Haiku 5.5	69.3%	estimated ± 3.4 pp, medium confidence
20	Claude Opus 5	68.8%	measured
21	GPT-6 Sol	68.8%	measured
22	MiMo-V2.6-Flash	67.9%	measured
23	Step 5 Preview	67.7%	measured
24	Kimi K3	67.5%	measured
25	Claude Fable 5.1	67.4%	measured
26	GPT-5.6 Luna	67.2%	measured
27	GLM-5.3	66.9%	measured
28	GPT-6 Luna	66.6%	measured
29	GPT-5.5	66.0%	estimated ± 3.4 pp, low confidence
30	Grok 4.6	65.9%	measured
31	Claude Sonnet 5	65.7%	estimated ± 3.4 pp, low confidence
32	Gemini 3.7 Flash	65.3%	measured
33	SWE-1.7	65.2%	estimated ± 3.4 pp, low confidence
34	GLM-5.2	64.9%	estimated ± 4.4 pp, high confidence
35	Hy4 preview	64.3%	measured
36	GLM-5.3-Flash	63.4%	measured
37	Claude Opus 4.7 (Adaptive)	62.8%	estimated ± 6.4 pp, medium confidence
38	DeepSeek V4 Pro 0813	62.7%	measured
39	Muse Spark 1.2	59.3%	measured
40	Qwen3.8 Max Preview	59.3%	estimated ± 6.4 pp, medium confidence
41	Claude Opus 4.7	58.9%	estimated ± 3.4 pp, low confidence
42	Qwen3.8-Flash-Next	58.7%	measured
43	Qwen3.8-Omni-Flash	57.8%	measured
44	Qwen3.8 Max	56.6%	measured
45	Claude Haiku 4.5	56.3%	estimated ± 7.2 pp, low confidence
46	Composer 2.5	56.3%	estimated ± 7.2 pp, low confidence
47	Gemini 3.1 Flash-Lite	56.3%	estimated ± 7.2 pp, low confidence
48	Gemini 3 Flash	56.3%	estimated ± 7.2 pp, low confidence
49	GPT-5.2-Codex	56.3%	estimated ± 7.2 pp, low confidence
50	GPT-5.3 Codex	56.3%	estimated ± 7.2 pp, low confidence
51	Grok 4.20	56.3%	estimated ± 7.2 pp, low confidence
52	Laguna M.1	56.3%	estimated ± 7.2 pp, low confidence
53	Laguna XS.2	56.3%	estimated ± 7.2 pp, low confidence
54	MiMo-V2.5	56.3%	estimated ± 7.2 pp, low confidence
55	Qwen3.5 Flash	56.3%	estimated ± 7.2 pp, low confidence
56	Qwen 3.6 Max (preview)	56.3%	estimated ± 7.2 pp, low confidence
57	Ornith-1.5-397B	56.0%	measured
58	Gemini 3.5 Flash	55.8%	estimated ± 6.4 pp, medium confidence
59	DeepSeek V4 Flash 0731	54.4%	measured
60	Gemini 3.1 Pro	54.2%	estimated ± 4.4 pp, medium confidence
61	Muse Spark 1.1	53.3%	measured
62	Grok 4.5	53.0%	measured
63	GPT-5.4	50.0%	estimated ± 4.4 pp, medium confidence
64	dots3-note Preview	49.3%	estimated ± 9.6 pp, medium confidence
65	Gemini 3.6 Flash	49.0%	measured
66	Ornith-1.0-397B	47.2%	estimated ± 9.6 pp, medium confidence
67	Qwen3.7 Max	46.6%	estimated ± 6.4 pp, low confidence
68	Seed 2.1 Pro	45.6%	estimated ± 9.6 pp, medium confidence
69	Beam	44.4%	measured
70	Qwen3.8-27B	42.2%	measured
71	Seed 2.1 Turbo	41.3%	estimated ± 9.6 pp, medium confidence
72	Inkling	40.8%	estimated ± 4.2 pp, medium confidence
73	Claude Opus 4.5	40.6%	estimated ± 9.6 pp, medium confidence
74	Laguna S 2.1	40.4%	measured
75	Kimi K2.6	37.0%	estimated ± 6.4 pp, low confidence
76	Quasar 438B	35.7%	estimated ± 6.4 pp, low confidence
77	Apodex 1.1	34.7%	estimated ± 6.4 pp, low confidence
78	Apodex 1.1 Mini	34.7%	estimated ± 6.4 pp, low confidence
79	Kimi K2.7 Code	34.7%	estimated ± 6.4 pp, low confidence
80	MiMo-V2.5-Pro	33.4%	estimated ± 6.4 pp, low confidence
81	Hy3	30.4%	estimated ± 6.4 pp, low confidence
82	Hy3 Preview	30.4%	estimated ± 6.4 pp, low confidence
83	Ornith-1.0-35B	30.3%	estimated ± 9.6 pp, low confidence
84	Muse Spark	30.0%	estimated ± 6.4 pp, low confidence
85	MiniMax M3	29.9%	estimated ± 6.4 pp, low confidence
86	Ornith-1.5-9B	27.8%	estimated ± 9.6 pp, low confidence
87	GPT-5.4 nano	24.7%	estimated ± 6.4 pp, low confidence
88	Qwen3.7 Plus	24.3%	estimated ± 6.4 pp, low confidence
89	GLM-5.1	24.1%	estimated ± 6.4 pp, low confidence
90	Ornith-1.0-9B	22.2%	estimated ± 9.6 pp, low confidence
91	GPT-5.4 mini	22.2%	estimated ± 3.4 pp, low confidence
92	Ornith-1.5-35B-A3B	22.0%	measured
93	Claude Opus 4.6	21.8%	estimated ± 3.4 pp, low confidence
94	Qwen3.6 Plus	21.7%	estimated ± 6.4 pp, low confidence
95	Qwen3.6-27B	20.2%	estimated ± 6.4 pp, low confidence
96	Inkling-Small	18.8%	estimated ± 6.4 pp, low confidence
97	MiniMax M2.7	18.3%	estimated ± 6.4 pp, low confidence
98	Ling 3.0 Flash	15.1%	estimated ± 6.4 pp, low confidence
99	Ling 3.0 Flash FP8	15.1%	estimated ± 6.4 pp, low confidence
100	MiMo-V2-Flash	13.9%	estimated ± 6.4 pp, low confidence
101	Claude Sonnet 4.6	13.7%	estimated ± 3.4 pp, low confidence
102	GPT-5.1	13.3%	estimated ± 6.4 pp, low confidence
103	Gemini 3.5 Flash-Lite	13.2%	estimated ± 6.4 pp, low confidence
104	Nemotron 3 Ultra	13.1%	estimated ± 6.4 pp, low confidence
105	Muse Glimmer 30B	12.7%	estimated ± 6.4 pp, low confidence
106	Mistral Medium 3.5 128B	10.1%	estimated ± 6.4 pp, low confidence
107	Kimi K2.5	9.9%	estimated ± 6.4 pp, low confidence
108	Kimi K2.5 (Reasoning)	9.9%	estimated ± 6.4 pp, low confidence
109	Qwen3.5-122B-A10B	8.8%	estimated ± 6.4 pp, low confidence
110	GLM-4.7	8.3%	estimated ± 6.4 pp, low confidence
111	Gemma 4 31B	6.6%	estimated ± 6.4 pp, low confidence
112	Grok 4.3	5.6%	estimated ± 6.4 pp, low confidence
113	Qwen3.6-35B-A3B	5.4%	estimated ± 6.4 pp, low confidence
114	o1	4.0%	estimated ± 6.4 pp, low confidence
115	Step 3.7 Flash	3.9%	estimated ± 6.4 pp, low confidence
116	Gemma 4 26B A4B	3.7%	estimated ± 6.4 pp, low confidence
117	GPT-5 (high)	3.0%	estimated ± 6.4 pp, low confidence
118	Nemotron 3 Super 100B	2.9%	estimated ± 6.4 pp, low confidence
119	o1-preview	1.6%	estimated ± 6.4 pp, low confidence
120	Gemini 2.5 Pro	1.4%	estimated ± 6.4 pp, low confidence
121	K-Exaone	1.1%	estimated ± 6.4 pp, low confidence
122	Gemma 4 12B	0.9%	estimated ± 6.4 pp, low confidence
123	GPT-OSS 120B	0.8%	estimated ± 6.4 pp, low confidence
124	Command A+	0.5%	estimated ± 6.4 pp, low confidence
125	Nemotron 3.5 Lightning 30B A3B NVFP4	0.4%	estimated ± 6.4 pp, low confidence
126	Mistral Small 4	0.4%	estimated ± 6.4 pp, low confidence
127	Mistral Small 4 (Reasoning)	0.4%	estimated ± 6.4 pp, low confidence
128	Trinity-Large-Preview	0.3%	estimated ± 6.4 pp, low confidence
129	Trinity-Large-Thinking	0.3%	estimated ± 6.4 pp, low confidence
130	Ling 2.6 Flash	0.3%	estimated ± 6.4 pp, low confidence
131	Gemini 1.5 Pro	0.2%	estimated ± 6.4 pp, low confidence
132	DeepSeek V3	0.2%	estimated ± 6.4 pp, low confidence
133	Granite 4.2 8B	0.1%	estimated ± 6.4 pp, low confidence
134	GPT-4 Turbo	0.1%	estimated ± 6.4 pp, low confidence
135	GPT-OSS 20B	0.1%	estimated ± 6.4 pp, low confidence
136	GPT-4.1 mini	0.1%	estimated ± 6.4 pp, low confidence
137	Mistral Large 3	0.1%	estimated ± 6.4 pp, low confidence
138	Claude 3 Opus	0.1%	estimated ± 6.4 pp, low confidence
139	Llama 4 Maverick	0.0%	estimated ± 6.4 pp, low confidence
140	Celeris-1	0.0%	estimated ± 6.4 pp, low confidence
141	Nemotron 3 Nano 30B	0.0%	estimated ± 6.4 pp, low confidence
142	Nemotron 3 Nano Omni 30B A3B	0.0%	estimated ± 6.4 pp, low confidence
143	Ultravox v0.6 Llama 3.3 70B	0.0%	estimated ± 6.4 pp, low confidence
144	GPT-4o mini	0.0%	estimated ± 6.4 pp, low confidence
145	GPT-4.1 nano	0.0%	estimated ± 6.4 pp, low confidence
146	Gemma 3 27B	0.0%	estimated ± 6.4 pp, low confidence
147	Gemma 4 E4B	0.0%	estimated ± 6.4 pp, low confidence
148	Llama 4 Scout	0.0%	estimated ± 6.4 pp, low confidence
149	Gemma 4 E2B	0.0%	estimated ± 6.4 pp, low confidence
150	LFM2.5-2.6B	0.0%	estimated ± 6.4 pp, low confidence

DeepSWE leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General