Coding

SWE-Atlas Codebase QnA leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap
License: https://creativecommons.org/licenses/by-nc/4.0/

As of 2026-10-29, the highest measured score on SWE-Atlas Codebase QnA is 64.5% by Ling 3.1 Flash. 48 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	Muse Wonder Full	64.6%	estimated ± 5.4 pp, low confidence
2	Muse Wonder	64.6%	estimated ± 5.4 pp, low confidence
3	Ling 3.1 Flash	64.5%	measured
4	Gemini 4 Argon	64.5%	estimated ± 5.4 pp, low confidence
5	DeepSeek V4.1 Flash 1017	64.3%	estimated ± 5.4 pp, low confidence
6	Kimi K3.1 Pro	64.1%	estimated ± 5.4 pp, low confidence
7	Ember-1	64.0%	estimated ± 5.4 pp, low confidence
8	Claude Opus 5.5	63.8%	estimated ± 5.4 pp, low confidence
9	GPT-6 Astra	63.8%	estimated ± 5.4 pp, low confidence
10	Pareto 26.9	63.8%	estimated ± 5.4 pp, low confidence
11	Gemini 3.8 Flash	63.8%	estimated ± 5.4 pp, low confidence
12	MiniMax M3.1 Flash	63.7%	estimated ± 5.4 pp, low confidence
13	SWE-2	63.6%	estimated ± 5.4 pp, low confidence
14	GPT-5.6 Sol	63.5%	estimated ± 5.4 pp, low confidence
15	Particle-1	63.5%	estimated ± 5.4 pp, low confidence
16	GPT-6.1 Sol	63.3%	estimated ± 5.4 pp, low confidence
17	MiMo-V2.6-Pro	63.3%	estimated ± 5.4 pp, low confidence
18	Claude Sonnet 5.5	63.1%	estimated ± 5.4 pp, low confidence
19	Grok 4.7	63.1%	estimated ± 5.4 pp, low confidence
20	Pareto 26.10 Preview	62.8%	estimated ± 5.4 pp, low confidence
21	GPT-5.6 Terra	62.7%	estimated ± 5.4 pp, low confidence
22	Claude Opus 5	62.5%	estimated ± 5.4 pp, low confidence
23	GPT-6 Sol	62.5%	estimated ± 5.4 pp, low confidence
24	MiMo-V2.6-Flash	62.2%	estimated ± 5.4 pp, low confidence
25	Step 5 Preview	62.1%	estimated ± 5.4 pp, low confidence
26	Kimi K3	62.0%	estimated ± 5.4 pp, low confidence
27	Claude Fable 5.1	62.0%	estimated ± 5.4 pp, low confidence
28	GPT-5.6 Luna	61.9%	estimated ± 5.4 pp, low confidence
29	GLM-5.3	61.8%	estimated ± 5.4 pp, low confidence
30	GPT-6 Luna	61.7%	estimated ± 5.4 pp, low confidence
31	Grok 4.6	61.4%	estimated ± 5.4 pp, low confidence
32	Gemini 3.7 Flash	61.2%	estimated ± 5.4 pp, low confidence
33	Hy4 preview	60.7%	estimated ± 5.4 pp, low confidence
34	GLM-5.3-Flash	60.3%	estimated ± 5.4 pp, low confidence
35	DeepSeek V4 Pro 0813	59.9%	estimated ± 5.4 pp, low confidence
36	Muse Spark 1.3	59.4%	measured
37	Ultramarine 1 Flash	59.2%	estimated ± 5.4 pp, low confidence
38	Muse Spark Open	58.0%	measured
39	Muse Spark 1.2	57.6%	estimated ± 5.4 pp, low confidence
40	Qwen3.8-Flash-Next	57.2%	estimated ± 5.4 pp, low confidence
41	Qwen3.8-Omni-Flash	56.4%	estimated ± 5.4 pp, low confidence
42	Qwen3.8 Max	55.3%	estimated ± 5.4 pp, low confidence
43	Ornith-1.5-397B	54.7%	estimated ± 5.4 pp, low confidence
44	DeepSeek V4 Flash 0731	52.9%	estimated ± 5.4 pp, low confidence
45	Muse Spark 1.1	51.6%	estimated ± 5.4 pp, low confidence
46	Grok 4.5	51.2%	estimated ± 5.4 pp, low confidence
47	Gemini 3.6 Flash	45.0%	estimated ± 5.4 pp, low confidence
48	Mercury 3	39.8%	estimated ± 5.4 pp, low confidence
49	Beam 1.0 501B-A23B FP8	32.2%	measured
50	Beam 1.0 501B-A23B	31.8%	measured
51	Beam 1.0 501B-A23B NVFP4	31.4%	measured
52	Qwen3.8-27B	30.7%	estimated ± 5.4 pp, low confidence
53	Laguna S 2.1	26.4%	estimated ± 5.4 pp, low confidence
54	Ornith-1.5-35B-A3B	1.1%	estimated ± 5.4 pp, low confidence

SWE-Atlas Codebase QnA leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General