Coding

FrontierSWE v2 leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

As of 2026-10-07, the highest measured score on FrontierSWE v2 is 65.5% by GPT-6 Astra. 150 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	GPT-6 Astra	65.5%	measured
2	Claude Opus 5.5	62.3%	measured
3	Claude Sonnet 5.5	61.9%	measured
4	Claude Fable 5.1	56.3%	measured
5	Gemini 4 Argon	55.0%	measured
6	Claude Opus 5	52.0%	measured
7	Claude Mythos 5	51.3%	estimated ± 9.0 pp, medium confidence
8	Claude Fable 5	47.0%	measured
9	MiMo-V2.6-Pro	45.2%	estimated ± 14.6 pp, low confidence
10	Sakana Fugu-Ultra	39.5%	estimated ± 9.0 pp, medium confidence
11	Step 5 Preview	39.1%	estimated ± 14.6 pp, low confidence
12	GPT-6 Sol	35.1%	estimated ± 14.6 pp, low confidence
13	GPT-5.6 Sol	32.2%	measured
14	Claude Opus 4.8	31.5%	estimated ± 9.0 pp, medium confidence
15	GLM-5.3	30.2%	measured
16	Grok 4.7	29.5%	measured
17	Muse Spark 1.3	27.9%	estimated ± 7.6 pp, medium confidence
18	GPT-5.6 Terra	27.1%	estimated ± 7.6 pp, medium confidence
19	Claude Haiku 5.5	27.1%	estimated ± 14.6 pp, low confidence
20	Kimi K3	25.9%	measured
21	GPT-6 Luna	25.8%	estimated ± 14.6 pp, low confidence
22	Grok 4.6	25.3%	measured
23	Hy4 preview	25.3%	estimated ± 9.0 pp, medium confidence
24	Beam	24.9%	estimated ± 9.0 pp, medium confidence
25	GPT-6.1 Sol	24.6%	estimated ± 14.6 pp, low confidence
26	Mistral Large 4	24.6%	estimated ± 14.6 pp, low confidence
27	Ling 3.1 Flash	24.3%	estimated ± 14.6 pp, low confidence
28	Ornith-1.5-397B	24.2%	estimated ± 9.0 pp, medium confidence
29	Grok 4.5	23.5%	estimated ± 9.0 pp, medium confidence
30	Claude Opus 4.7 (Adaptive)	22.8%	estimated ± 9.0 pp, medium confidence
31	Qwen3.8-Omni-Flash	21.0%	estimated ± 9.0 pp, medium confidence
32	Gemini 3.7 Flash	20.3%	measured
33	Gemini 3.8 Flash	19.6%	measured
34	Qwen3.8-Flash-Next	19.6%	estimated ± 9.0 pp, medium confidence
35	Ornith-1.0-397B	19.1%	estimated ± 9.0 pp, medium confidence
36	GLM-5.2	18.9%	estimated ± 9.0 pp, medium confidence
37	Qwen3.8-27B	18.2%	estimated ± 9.0 pp, medium confidence
38	Qwen3.8 Max Preview	18.1%	estimated ± 14.6 pp, low confidence
39	GLM-5.3-Flash	18.1%	measured
40	Muse Spark 1.1	17.8%	estimated ± 9.0 pp, medium confidence
41	DeepSeek V4.1 Flash	17.5%	estimated ± 14.6 pp, low confidence
42	dots3-note Preview	16.9%	estimated ± 9.0 pp, medium confidence
43	Qwen3.7 Max	16.2%	estimated ± 9.0 pp, medium confidence
44	Qwen3.8 Max	15.8%	measured
45	MiMo-V2.6-Flash	15.6%	estimated ± 14.6 pp, low confidence
46	Atria Dawn Preview	14.4%	estimated ± 9.0 pp, medium confidence
47	Ornith-1.5-35B-A3B	14.4%	estimated ± 9.0 pp, medium confidence
48	GPT-5.6 Luna	14.4%	estimated ± 7.6 pp, low confidence
49	Laguna S 2.1	14.1%	estimated ± 9.0 pp, medium confidence
50	MiniMax M3	13.4%	estimated ± 9.0 pp, medium confidence
51	Sakana Fugu	13.4%	estimated ± 9.0 pp, medium confidence
52	GPT-5.5	12.6%	estimated ± 9.0 pp, medium confidence
53	Kimi K2.6	12.6%	estimated ± 9.0 pp, medium confidence
54	GLM-5.1	12.3%	estimated ± 9.0 pp, medium confidence
55	Muse Spark 1.2	12.0%	measured
56	Claude Opus 4.7	11.6%	estimated ± 9.5 pp, medium confidence
57	Claude Sonnet 5	11.1%	estimated ± 7.6 pp, low confidence
58	GPT-5.4	11.0%	estimated ± 9.0 pp, medium confidence
59	Qwen3.7 Plus	10.9%	estimated ± 9.0 pp, medium confidence
60	Qwen 3.6 Max (preview)	10.3%	estimated ± 9.0 pp, medium confidence
61	MiMo-V2.5-Pro	10.2%	estimated ± 9.0 pp, medium confidence
62	Gemini 3.6 Flash	10.1%	estimated ± 9.5 pp, medium confidence
63	Claude Opus 4.5	10.0%	estimated ± 9.0 pp, medium confidence
64	Gemini 3.1 Pro	9.7%	estimated ± 9.5 pp, medium confidence
65	GPT-5.3 Codex	9.4%	estimated ± 9.0 pp, medium confidence
66	Kimi K2.7 Code	9.4%	estimated ± 9.5 pp, medium confidence
67	Ling 3.0 Flash	9.1%	estimated ± 9.0 pp, medium confidence
68	Qwen3.6 Plus	9.1%	estimated ± 9.0 pp, medium confidence
69	Claude Sonnet 4.6	9.0%	estimated ± 9.5 pp, low confidence
70	Step 3.7 Flash	8.6%	estimated ± 9.0 pp, medium confidence
71	MiniMax M2.7	8.4%	estimated ± 9.0 pp, medium confidence
72	MiMo-V2.5	8.2%	estimated ± 9.0 pp, medium confidence
73	Gemini 3 Flash	8.0%	estimated ± 9.5 pp, low confidence
74	Inkling-Small	7.8%	estimated ± 9.0 pp, medium confidence
75	Hy3	7.3%	estimated ± 14.6 pp, low confidence
76	Hy3 Preview	7.3%	estimated ± 14.6 pp, low confidence
77	GPT-5.2	7.3%	estimated ± 9.0 pp, medium confidence
78	GPT-5.4 mini	7.3%	estimated ± 9.5 pp, low confidence
79	GPT-5.2-Codex	7.1%	estimated ± 9.5 pp, low confidence
80	DeepSeek V4 Pro 0813	6.9%	estimated ± 9.0 pp, medium confidence
81	Grok 4.3	6.8%	estimated ± 9.5 pp, low confidence
82	Gemini 3.5 Flash	6.4%	estimated ± 9.0 pp, medium confidence
83	GLM-5	6.4%	estimated ± 9.0 pp, medium confidence
84	GPT-5.4 nano	6.3%	estimated ± 9.5 pp, low confidence
85	GLM-4.7	6.2%	estimated ± 9.5 pp, low confidence
86	Nemotron 3 Ultra	6.1%	estimated ± 9.5 pp, low confidence
87	Quasar 438B	5.8%	estimated ± 14.6 pp, low confidence
88	Claude Haiku 4.5	5.5%	estimated ± 9.5 pp, low confidence
89	Mistral Medium 3.5 128B	5.5%	estimated ± 9.5 pp, low confidence
90	Qwen3.5 Flash	5.0%	estimated ± 9.5 pp, low confidence
91	Gemini 3.5 Flash-Lite	4.8%	estimated ± 9.0 pp, low confidence
92	Gemini 3.1 Flash-Lite	4.7%	estimated ± 9.5 pp, low confidence
93	Inkling	4.1%	measured
94	MiMo-V2-Flash	3.7%	estimated ± 14.8 pp, low confidence
95	Qwen3.6-27B	3.6%	estimated ± 9.0 pp, low confidence
96	Composer 2.5	3.5%	estimated ± 7.6 pp, low confidence
97	GPT-5.1	3.5%	estimated ± 14.8 pp, low confidence
98	Gemini 2.5 Pro	3.4%	estimated ± 9.5 pp, low confidence
99	Claude Opus 4.6	3.4%	estimated ± 9.0 pp, low confidence
100	Kimi K2.5 (Reasoning)	2.6%	estimated ± 14.8 pp, low confidence
101	MAI-Thinking-1	2.3%	estimated ± 9.0 pp, low confidence
102	DeepSeek V4 Flash 0731	2.0%	estimated ± 9.0 pp, low confidence
103	Muse Spark	1.6%	estimated ± 9.0 pp, low confidence
104	o1	1.0%	estimated ± 14.8 pp, low confidence
105	GPT-5 (high)	0.7%	estimated ± 14.8 pp, low confidence
106	Grok 4.20	0.5%	estimated ± 9.0 pp, low confidence
107	o1-preview	0.4%	estimated ± 14.8 pp, low confidence
108	K-Exaone	0.3%	estimated ± 14.8 pp, low confidence
109	Gemma 4 12B	0.2%	estimated ± 14.8 pp, low confidence
110	Ling 2.6 Flash	0.1%	estimated ± 14.8 pp, low confidence
111	Gemini 1.5 Pro	0.0%	estimated ± 14.8 pp, low confidence
112	GPT-4 Turbo	0.0%	estimated ± 14.8 pp, low confidence
113	GPT-4.1 mini	0.0%	estimated ± 14.8 pp, low confidence
114	Claude 3 Opus	0.0%	estimated ± 14.8 pp, low confidence
115	Nemotron 3 Nano Omni 30B A3B	0.0%	estimated ± 14.8 pp, low confidence
116	Ultravox v0.6 Llama 3.3 70B	0.0%	estimated ± 14.8 pp, low confidence
117	GPT-4.1 nano	0.0%	estimated ± 14.8 pp, low confidence
118	GPT-4o mini	0.0%	estimated ± 14.8 pp, low confidence
119	A.X K2	0.0%	estimated ± 14.6 pp, low confidence
120	Apodex 1.1	0.0%	estimated ± 14.6 pp, low confidence
121	Apodex 1.1 Mini	0.0%	estimated ± 14.6 pp, low confidence
122	Celeris-1	0.0%	estimated ± 14.6 pp, low confidence
123	Command A+	0.0%	estimated ± 14.6 pp, low confidence
124	DeepSeek V3	0.0%	estimated ± 14.6 pp, low confidence
125	DeepSeek V3 0324	0.0%	estimated ± 14.6 pp, low confidence
126	Gemma 3 27B	0.0%	estimated ± 14.6 pp, low confidence
127	Gemma 4 26B A4B	0.0%	estimated ± 14.6 pp, low confidence
128	Gemma 4 31B	0.0%	estimated ± 14.6 pp, low confidence
129	Gemma 4 E2B	0.0%	estimated ± 14.8 pp, low confidence
130	Gemma 4 E4B	0.0%	estimated ± 14.6 pp, low confidence
131	GPT-OSS 120B	0.0%	estimated ± 14.6 pp, low confidence
132	GPT-OSS 20B	0.0%	estimated ± 14.6 pp, low confidence
133	Granite 4.2 30B	0.0%	estimated ± 9.0 pp, low confidence
134	Granite 4.2 3B	0.0%	estimated ± 14.6 pp, low confidence
135	Granite 4.2 8B	0.0%	estimated ± 9.0 pp, low confidence
136	K-EXAONE 2.0	0.0%	estimated ± 14.6 pp, low confidence
137	Kimi K2.5	0.0%	estimated ± 9.0 pp, low confidence
138	Laguna M.1	0.0%	estimated ± 9.0 pp, low confidence
139	Laguna XS.2	0.0%	estimated ± 9.0 pp, low confidence
140	Laguna XS 2.1	0.0%	estimated ± 9.0 pp, low confidence
141	LFM2.5-2.6B	0.0%	estimated ± 14.6 pp, low confidence
142	Ling 3.0 Flash FP8	0.0%	estimated ± 14.6 pp, low confidence
143	Ling 3.0 Flash VL	0.0%	estimated ± 14.6 pp, low confidence
144	Ling 3.0 Tiny	0.0%	estimated ± 14.6 pp, low confidence
145	LLaDA2.2-flash	0.0%	estimated ± 9.0 pp, low confidence
146	Llama 4 Maverick	0.0%	estimated ± 14.6 pp, low confidence
147	Llama 4 Scout	0.0%	estimated ± 14.6 pp, low confidence
148	LongCat-Flash-Lite-Sparse	0.0%	estimated ± 9.0 pp, low confidence
149	Mercury 2.5	0.0%	estimated ± 14.6 pp, low confidence
150	MiniCPM5-2B	0.0%	estimated ± 9.0 pp, low confidence
151	Mistral Large 3	0.0%	estimated ± 14.6 pp, low confidence
152	Mistral Small 4	0.0%	estimated ± 14.6 pp, low confidence
153	Mistral Small 4 (Reasoning)	0.0%	estimated ± 14.6 pp, low confidence
154	Muse Glimmer 30B	0.0%	estimated ± 9.0 pp, low confidence
155	Nemotron 3.5 Lightning 30B A3B NVFP4	0.0%	estimated ± 14.6 pp, low confidence
156	Nemotron 3 Nano 30B	0.0%	estimated ± 14.6 pp, low confidence
157	Nemotron 3 Super 100B	0.0%	estimated ± 14.6 pp, low confidence
158	North Mini Code	0.0%	estimated ± 14.6 pp, low confidence
159	Ornith-1.0-35B	0.0%	estimated ± 9.0 pp, low confidence
160	Ornith-1.0-9B	0.0%	estimated ± 9.0 pp, low confidence
161	Ornith-1.5-9B	0.0%	estimated ± 9.0 pp, low confidence
162	Qwen3.5-122B-A10B	0.0%	estimated ± 14.6 pp, low confidence
163	Qwen3.5 397B	0.0%	estimated ± 9.0 pp, low confidence
164	Qwen3.6-35B-A3B	0.0%	estimated ± 9.0 pp, low confidence
165	Solar Pro 3	0.0%	estimated ± 14.6 pp, low confidence
166	Solar Pro 4	0.0%	estimated ± 14.6 pp, low confidence
167	Trinity-Large-Preview	0.0%	estimated ± 14.6 pp, low confidence
168	Trinity-Large-Thinking	0.0%	estimated ± 14.6 pp, low confidence

FrontierSWE v2 leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General