Coding

SWE-bench Pro leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

As of 2026-10-07, the highest measured score on SWE-bench Pro is 89.9% by Claude Opus 5.5. 129 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	Claude Opus 5.5	89.9%	measured
2	GPT-6 Astra	86.7%	estimated ± 5.1 pp, low confidence
3	Gemini 4 Argon	81.5%	estimated ± 5.1 pp, medium confidence
4	Claude Sonnet 5.5	81.3%	measured
5	Claude Fable 5.1	81.2%	measured
6	Claude Mythos 5	80.3%	measured
7	Claude Fable 5	80.0%	measured
8	Claude Opus 5	79.2%	measured
9	Pareto 26.9	77.1%	estimated ± 7.8 pp, low confidence
10	SWE-2	76.6%	estimated ± 7.5 pp, medium confidence
11	MiMo-V2.6-Pro	76.1%	estimated ± 6.8 pp, medium confidence
12	Ember-1	74.7%	estimated ± 3.2 pp, high confidence
13	Grok 4.7	74.5%	estimated ± 4.3 pp, high confidence
14	Gemini 3.7 Flash	74.1%	estimated ± 4.8 pp, high confidence
15	Sakana Fugu-Ultra	73.7%	measured
16	Step 5 Preview	73.0%	estimated ± 6.8 pp, medium confidence
17	Pareto 26.10 Preview	73.0%	estimated ± 7.8 pp, low confidence
18	GPT-6 Sol	71.0%	estimated ± 6.8 pp, medium confidence
19	Muse Spark 1.3	69.6%	estimated ± 4.3 pp, high confidence
20	Grok 4.6	69.4%	estimated ± 4.3 pp, high confidence
21	Claude Opus 4.8	69.2%	measured
22	SWE-1.7	68.8%	estimated ± 7.5 pp, medium confidence
23	Qwen3.8 Max	67.7%	measured
24	Gemini 3.8 Flash	67.4%	estimated ± 4.3 pp, high confidence
25	Claude Haiku 5.5	67.0%	estimated ± 6.8 pp, medium confidence
26	GPT-6 Luna	66.3%	estimated ± 6.8 pp, medium confidence
27	GPT-6.1 Sol	65.7%	estimated ± 6.8 pp, medium confidence
28	Mistral Large 4	65.7%	estimated ± 6.8 pp, medium confidence
29	Hy4 preview	65.7%	measured
30	Ling 3.1 Flash	65.6%	estimated ± 6.8 pp, medium confidence
31	Beam	65.5%	measured
32	Ornith-1.5-397B	65.1%	measured
33	Grok 4.5	64.7%	measured
34	GPT-5.6 Sol	64.6%	measured
35	Qwen3.8 Max Preview	64.4%	estimated ± 6.0 pp, medium confidence
36	Claude Opus 4.7 (Adaptive)	64.3%	measured
37	GPT-5.6 Terra	63.4%	measured
38	Qwen3.8-Omni-Flash	63.3%	measured
39	Kimi K3	63.3%	estimated ± 4.5 pp, high confidence
40	Claude Sonnet 5	63.2%	measured
41	DeepSeek V4.1 Flash	62.8%	estimated ± 3.6 pp, medium confidence
42	GPT-5.6 Luna	62.7%	measured
43	Qwen3.8-Flash-Next	62.5%	measured
44	GLM-5.3	62.4%	estimated ± 3.6 pp, high confidence
45	GLM-5.3-Flash	62.2%	estimated ± 3.6 pp, high confidence
46	Ornith-1.0-397B	62.2%	measured
47	GLM-5.2	62.1%	measured
48	Qwen3.8-27B	61.7%	measured
49	Muse Spark 1.1	61.5%	measured
50	MiMo-V2.6-Flash	61.2%	estimated ± 6.8 pp, medium confidence
51	dots3-note Preview	61.0%	measured
52	Qwen3.7 Max	60.6%	measured
53	Seed 2.1 Pro	60.4%	estimated ± 3.6 pp, high confidence
54	Muse Spark 1.2	60.2%	estimated ± 5.1 pp, medium confidence
55	Atria Dawn Preview	59.6%	measured
56	Ornith-1.5-35B-A3B	59.6%	measured
57	Laguna S 2.1	59.4%	measured
58	Seed 2.1 Turbo	59.0%	estimated ± 3.6 pp, high confidence
59	MiniMax M3	59.0%	measured
60	Sakana Fugu	59.0%	measured
61	Quasar 438B	58.9%	estimated ± 6.0 pp, medium confidence
62	Apodex 1.1 Mini	58.7%	estimated ± 6.0 pp, medium confidence
63	GPT-5.5	58.6%	measured
64	Kimi K2.6	58.6%	measured
65	GLM-5.1	58.4%	measured
66	BTL-3	58.3%	estimated ± 4.7 pp, high confidence
67	GPT-5.4	57.7%	measured
68	Qwen3.7 Plus	57.6%	measured
69	Hy3	57.6%	estimated ± 6.0 pp, medium confidence
70	Qwen 3.6 Max (preview)	57.3%	measured
71	Claude Opus 4.7	57.2%	estimated ± 2.1 pp, high confidence
72	MiMo-V2.5-Pro	57.2%	measured
73	Claude Sonnet 4.6	57.1%	estimated ± 2.1 pp, high confidence
74	Claude Opus 4.5	57.1%	measured
75	GPT-5.4 mini	57.1%	estimated ± 2.1 pp, high confidence
76	Gemini 3.6 Flash	57.0%	estimated ± 4.5 pp, high confidence
77	Grok 4.3	57.0%	estimated ± 4.8 pp, high confidence
78	Qwen3.5 Flash	57.0%	estimated ± 4.8 pp, high confidence
79	GLM-4.5	57.0%	estimated ± 4.8 pp, medium confidence
80	GPT-5.2-Codex	56.9%	estimated ± 2.1 pp, high confidence
81	Gemini 3.1 Pro	56.8%	estimated ± 2.1 pp, high confidence
82	GPT-5.3 Codex	56.8%	measured
83	GPT-5.4 nano	56.6%	estimated ± 2.1 pp, high confidence
84	Ling 3.0 Flash	56.6%	measured
85	Qwen3.6 Plus	56.6%	measured
86	GPT-5.1-Codex-Max	56.5%	estimated ± 2.1 pp, high confidence
87	Gemini 3 Flash	56.4%	estimated ± 2.1 pp, high confidence
88	Step 3.7 Flash	56.3%	measured
89	MiniMax M2.7	56.2%	measured
90	MiMo-V2.5	56.1%	measured
91	Inkling-Small	55.9%	measured
92	GPT-5.1-Codex	55.7%	estimated ± 2.1 pp, high confidence
93	GPT-5.2	55.6%	measured
94	DeepSeek V4 Pro 0813	55.4%	measured
95	Gemini 3.5 Flash	55.1%	measured
96	GLM-5	55.1%	measured
97	BTL-4	55.0%	estimated ± 3.2 pp, high confidence
98	Kimi K2.7 Code	54.8%	estimated ± 4.5 pp, high confidence
99	MiMo-V2-Pro	54.4%	estimated ± 3.2 pp, high confidence
100	Inkling	54.3%	measured
101	Gemini 3.5 Flash-Lite	54.2%	measured
102	Apodex 1.1	54.1%	estimated ± 3.2 pp, high confidence
103	Mistral Medium 3.5 128B	53.9%	estimated ± 3.2 pp, high confidence
104	Qwen3.6-27B	53.5%	measured
105	Claude Sonnet 4.5	53.4%	estimated ± 3.2 pp, high confidence
106	Claude Opus 4.6	53.4%	measured
107	Kimi K2.5 (Reasoning)	52.9%	estimated ± 3.2 pp, high confidence
108	MAI-Thinking-1	52.8%	measured
109	Ling 3.0 Flash FP8	52.6%	estimated ± 6.0 pp, medium confidence
110	DeepSeek V4 Flash 0731	52.6%	measured
111	Composer 2.5	52.4%	estimated ± 4.3 pp, medium confidence
112	Muse Spark	52.4%	measured
113	GPT-5.1	51.8%	estimated ± 6.0 pp, medium confidence
114	Grok 4.20	51.8%	measured
115	Muse Glimmer 30B	51.2%	measured
116	Qwen3.5 397B	50.9%	measured
117	A.X K2	50.8%	estimated ± 4.7 pp, high confidence
118	Kimi K2.5	50.7%	measured
119	MiMo-V2-Omni	50.4%	estimated ± 3.2 pp, high confidence
120	Ornith-1.0-35B	50.4%	measured
121	GLM-4.6	50.3%	estimated ± 2.1 pp, high confidence
122	Ling 3.0 Flash VL	50.2%	estimated ± 6.8 pp, medium confidence
123	Claude 4.1 Opus	50.1%	estimated ± 3.2 pp, high confidence
124	Hy3 Preview	50.0%	estimated ± 3.2 pp, high confidence
125	Qwen3.6-35B-A3B	49.5%	measured
126	GLM-4.7	49.2%	estimated ± 3.2 pp, high confidence
127	Laguna M.1	49.2%	measured
128	MiMo-V2-Flash	48.7%	estimated ± 3.2 pp, high confidence
129	Claude Haiku 4.5	48.6%	estimated ± 3.2 pp, high confidence
130	Claude 4 Sonnet	47.9%	estimated ± 3.2 pp, high confidence
131	MAI-Code-1.1-Flash	47.8%	estimated ± 3.2 pp, high confidence
132	Gemma 4 31B	47.7%	estimated ± 6.0 pp, medium confidence
133	Laguna XS 2.1	47.6%	measured
134	Qwen3.5-27B	47.6%	estimated ± 3.2 pp, high confidence
135	Ornith-1.5-9B	47.5%	measured
136	Qwen3.5-122B-A10B	47.1%	estimated ± 3.2 pp, high confidence
137	Nemotron 3 Ultra	47.0%	estimated ± 3.2 pp, high confidence
138	Laguna XS.2	46.3%	measured
139	Grok Code Fast 1	45.7%	estimated ± 3.2 pp, high confidence
140	Solar Pro 4	45.5%	estimated ± 3.2 pp, high confidence
141	Solar Open 2	45.2%	estimated ± 3.2 pp, high confidence
142	o1	44.9%	estimated ± 6.0 pp, medium confidence
143	Gemma 4 26B A4B	44.6%	estimated ± 6.0 pp, medium confidence
144	Qwen3.5-35B-A3B	43.9%	estimated ± 3.2 pp, high confidence
145	GPT-5 (high)	43.4%	estimated ± 6.0 pp, medium confidence
146	Nemotron 3 Super 100B	43.4%	estimated ± 6.0 pp, medium confidence
147	Ornith-1.0-9B	42.9%	measured
148	K-EXAONE 2.0	42.7%	estimated ± 3.2 pp, high confidence
149	DeepSeek V3 0324	42.2%	estimated ± 6.8 pp, medium confidence
150	North Mini Code	41.9%	estimated ± 6.8 pp, medium confidence
151	Mercury 2.5	41.4%	estimated ± 6.8 pp, medium confidence
152	LongCat-Flash-Lite-Sparse	40.6%	measured
153	o1-preview	40.4%	estimated ± 6.0 pp, medium confidence
154	K-Exaone	38.7%	estimated ± 6.0 pp, medium confidence
155	Gemini 2.5 Pro	38.0%	estimated ± 3.2 pp, high confidence
156	GPT-OSS 120B	37.3%	estimated ± 6.0 pp, medium confidence
157	Command A+	34.9%	estimated ± 6.0 pp, medium confidence
158	Ternary Bonsai 2 27B	34.9%	estimated ± 3.2 pp, high confidence
159	Mistral Small 4	33.7%	estimated ± 6.0 pp, medium confidence
160	Mistral Small 4 (Reasoning)	33.7%	estimated ± 6.0 pp, medium confidence
161	Granite 4.2 30B	33.3%	measured
162	Trinity-Large-Preview	32.9%	estimated ± 6.0 pp, medium confidence
163	Trinity-Large-Thinking	32.9%	estimated ± 6.0 pp, medium confidence
164	Ling 2.6 Flash	32.4%	estimated ± 6.0 pp, medium confidence
165	Gemini 1.5 Pro	30.8%	estimated ± 6.0 pp, medium confidence
166	LLaDA2.2-flash	30.1%	measured
167	GPT-4.1	28.8%	estimated ± 3.2 pp, high confidence
168	GPT-4 Turbo	28.6%	estimated ± 6.0 pp, low confidence
169	GPT-OSS 20B	27.7%	estimated ± 6.0 pp, low confidence
170	ZAYA1-74B-Preview	27.5%	estimated ± 3.2 pp, high confidence
171	Nemotron 3.5 Lightning 30B A3B NVFP4	27.2%	estimated ± 3.2 pp, high confidence
172	Mistral Large 3	27.0%	estimated ± 6.0 pp, low confidence
173	Claude 3 Opus	26.4%	estimated ± 6.0 pp, low confidence
174	o3-mini	24.0%	estimated ± 3.2 pp, high confidence
175	Claude 3.5 Sonnet	23.8%	estimated ± 3.2 pp, high confidence
176	Llama 4 Maverick	22.8%	estimated ± 6.0 pp, low confidence
177	Solar Pro 3	21.3%	estimated ± 6.8 pp, low confidence
178	Celeris-1	20.5%	estimated ± 6.0 pp, low confidence
179	Nemotron 3 Nano 30B	20.5%	estimated ± 6.0 pp, low confidence
180	Gemma 4 12B	20.1%	estimated ± 4.7 pp, high confidence
181	Nemotron 3 Nano Omni 30B A3B	19.7%	estimated ± 6.0 pp, low confidence
182	Ling 3.0 Tiny	19.3%	estimated ± 6.8 pp, low confidence
183	Granite 4.2 8B	19.1%	measured
184	DeepSeek V3	17.9%	estimated ± 3.2 pp, medium confidence
185	Ultravox v0.6 Llama 3.3 70B	17.4%	estimated ± 6.0 pp, low confidence
186	GPT-4o mini	16.7%	estimated ± 6.0 pp, low confidence
187	GPT-4.1 nano	16.4%	estimated ± 6.0 pp, low confidence
188	Mellum2-12B-A2.5B-Thinking	15.8%	estimated ± 4.7 pp, high confidence
189	Granite 4.2 3B	15.4%	estimated ± 4.7 pp, high confidence
190	Gemma 3 27B	15.0%	estimated ± 6.0 pp, low confidence
191	MiniCPM5-2B	14.4%	measured
192	Gemma 4 E4B	14.1%	estimated ± 6.0 pp, low confidence
193	Llama 4 Scout	12.4%	estimated ± 6.0 pp, low confidence
194	Gemma 4 E2B	11.1%	estimated ± 6.0 pp, low confidence
195	ZAYA1-8B	9.3%	estimated ± 4.7 pp, medium confidence
196	GPT-4.1 mini	4.5%	estimated ± 3.2 pp, medium confidence
197	Agents-A1-4B	3.8%	estimated ± 4.7 pp, medium confidence
198	LFM2.5-2.6B	3.7%	estimated ± 4.7 pp, medium confidence
199	Mellum2-12B-A2.5B-Instruct	0.1%	estimated ± 4.7 pp, medium confidence
200	MiniCPM5-1B	0.1%	estimated ± 4.7 pp, medium confidence
201	LLaDA2.2-mini	0.0%	estimated ± 4.7 pp, medium confidence
202	Gemini 3.1 Flash-Lite	0.0%	estimated ± 2.1 pp, high confidence

SWE-bench Pro leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General