Coding

NL2Repo leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

As of 2026-10-07, the highest measured score on NL2Repo is 65.4% by DeepSeek V4.1 Flash. 168 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	Claude Opus 5.5	81.4%	estimated ± 6.4 pp, low confidence
2	Claude Mythos 5	76.7%	estimated ± 6.1 pp, low confidence
3	Claude Sonnet 5.5	71.6%	estimated ± 6.4 pp, low confidence
4	Ember-1	71.2%	estimated ± 6.1 pp, low confidence
5	Gemini 4 Argon	68.6%	estimated ± 6.5 pp, low confidence
6	MiMo-V2.6-Pro	67.2%	estimated ± 6.5 pp, low confidence
7	DeepSeek V4.1 Flash	65.4%	measured
8	Step 5 Preview	64.0%	estimated ± 6.5 pp, medium confidence
9	GPT-6 Sol	61.9%	estimated ± 6.5 pp, medium confidence
10	Grok 4.7	61.6%	estimated ± 6.5 pp, medium confidence
11	DeepSeek V4 Pro 0813	61.5%	measured
12	Pareto 26.9	61.3%	estimated ± 6.6 pp, medium confidence
13	Claude Opus 5	60.9%	estimated ± 4.7 pp, medium confidence
14	SWE-2	60.9%	estimated ± 6.6 pp, medium confidence
15	Claude Fable 5.1	60.7%	estimated ± 5.1 pp, low confidence
16	GPT-5.6 Sol	60.1%	estimated ± 4.7 pp, high confidence
17	Pareto 26.10 Preview	59.7%	estimated ± 6.6 pp, medium confidence
18	Ornith-1.5-397B	59.5%	measured
19	Grok 4.6	59.4%	estimated ± 4.7 pp, high confidence
20	GPT-5.6 Terra	59.2%	estimated ± 4.7 pp, high confidence
21	Hy4 preview	58.9%	measured
22	Claude Fable 5	58.8%	estimated ± 4.7 pp, high confidence
23	GLM-5.3	58.0%	measured
24	Claude Haiku 5.5	57.8%	estimated ± 6.5 pp, medium confidence
25	Kimi K3	57.1%	estimated ± 4.7 pp, high confidence
26	GPT-6 Luna	57.1%	estimated ± 6.5 pp, medium confidence
27	GPT-6 Astra	57.1%	estimated ± 5.1 pp, low confidence
28	GPT-5.6 Luna	56.7%	estimated ± 4.7 pp, high confidence
29	GPT-6.1 Sol	56.5%	estimated ± 6.5 pp, medium confidence
30	Mistral Large 4	56.5%	estimated ± 6.5 pp, medium confidence
31	Ling 3.1 Flash	56.3%	estimated ± 6.5 pp, medium confidence
32	GLM-5.3-Flash	56.3%	measured
33	Muse Spark 1.3	56.2%	estimated ± 5.1 pp, low confidence
34	Qwen3.8 Max	55.9%	measured
35	Claude Opus 4.7 (Adaptive)	54.5%	estimated ± 5.1 pp, medium confidence
36	DeepSeek V4 Flash 0731	54.2%	measured
37	Qwen3.8 Max Preview	53.1%	estimated ± 5.1 pp, medium confidence
38	Claude Opus 4.8	52.5%	estimated ± 4.7 pp, high confidence
39	GPT-5.4	52.5%	estimated ± 5.1 pp, medium confidence
40	MiMo-V2.6-Flash	51.8%	estimated ± 6.5 pp, medium confidence
41	Grok 4.5	50.8%	estimated ± 4.7 pp, high confidence
42	Muse Spark 1.2	50.8%	estimated ± 4.7 pp, high confidence
43	dots3-note Preview	49.8%	measured
44	Beam	49.2%	estimated ± 6.1 pp, medium confidence
45	Claude Opus 4.6	49.0%	estimated ± 6.1 pp, medium confidence
46	GLM-5.2	48.9%	measured
47	Qwen3.8-Omni-Flash	48.9%	measured
48	Ornith-1.0-397B	48.2%	measured
49	Qwen3.8-Flash-Next	48.1%	measured
50	Sakana Fugu-Ultra	47.7%	estimated ± 3.1 pp, low confidence
51	GPT-5.5	47.7%	estimated ± 4.7 pp, high confidence
52	Sakana Fugu	47.5%	estimated ± 3.1 pp, low confidence
53	Inkling-Small	47.4%	estimated ± 4.7 pp, high confidence
54	Solar Open 2	47.3%	estimated ± 3.1 pp, low confidence
55	GPT-5.2	47.3%	estimated ± 6.1 pp, medium confidence
56	Claude Opus 4.7	47.2%	estimated ± 4.7 pp, high confidence
57	Muse Spark 1.1	47.2%	estimated ± 4.7 pp, high confidence
58	Qwen3.7 Max	47.2%	measured
59	Atria Dawn Preview	47.0%	estimated ± 6.4 pp, medium confidence
60	Seed 2.1 Pro	47.0%	measured
61	Laguna S 2.1	46.8%	estimated ± 6.4 pp, medium confidence
62	Ternary Bonsai 2 27B	46.4%	estimated ± 3.1 pp, low confidence
63	Gemini 3.7 Flash	46.4%	estimated ± 4.7 pp, high confidence
64	Kimi K2.6	46.2%	estimated ± 3.1 pp, low confidence
65	Ornith-1.5-35B-A3B	46.2%	measured
66	Nemotron 3 Ultra	46.0%	estimated ± 3.1 pp, low confidence
67	Gemini 3.8 Flash	45.9%	estimated ± 4.7 pp, high confidence
68	BTL-3	45.6%	estimated ± 3.1 pp, low confidence
69	Claude Sonnet 5	45.6%	estimated ± 4.7 pp, high confidence
70	Composer 2.5	45.6%	estimated ± 4.7 pp, high confidence
71	Gemini 3.6 Flash	45.6%	estimated ± 4.7 pp, high confidence
72	MAI-Thinking-1	45.5%	estimated ± 3.1 pp, low confidence
73	Qwen3.6 Plus	45.2%	estimated ± 3.1 pp, low confidence
74	Gemini 3.1 Pro	45.1%	estimated ± 4.7 pp, high confidence
75	Gemini 3.5 Flash	45.1%	estimated ± 4.7 pp, high confidence
76	Quasar 438B	44.8%	estimated ± 5.1 pp, medium confidence
77	Kimi K2.7 Code	44.7%	estimated ± 4.7 pp, high confidence
78	GPT-5.3 Codex	44.6%	estimated ± 4.7 pp, high confidence
79	Apodex 1.1	44.5%	estimated ± 5.1 pp, medium confidence
80	Apodex 1.1 Mini	44.5%	estimated ± 5.1 pp, medium confidence
81	Kimi K2.5	44.4%	estimated ± 3.1 pp, low confidence
82	Inkling	44.4%	estimated ± 4.7 pp, high confidence
83	Claude Sonnet 4.6	44.3%	estimated ± 4.7 pp, high confidence
84	A.X K2	44.0%	estimated ± 3.1 pp, low confidence
85	Qwen3.5 397B	43.8%	estimated ± 3.1 pp, low confidence
86	Seed 2.1 Turbo	43.7%	measured
87	MiMo-V2-Pro	43.2%	estimated ± 6.1 pp, medium confidence
88	Claude Opus 4.5	43.2%	measured
89	Gemini 3.5 Flash-Lite	43.0%	estimated ± 4.7 pp, high confidence
90	Gemini 3 Flash	43.0%	estimated ± 4.7 pp, high confidence
91	Hy3	43.0%	estimated ± 5.1 pp, medium confidence
92	Hy3 Preview	43.0%	estimated ± 5.1 pp, medium confidence
93	Qwen 3.6 Max (preview)	42.9%	measured
94	GLM-5	42.8%	estimated ± 6.1 pp, medium confidence
95	GLM-5.1	42.7%	measured
96	Muse Spark	42.7%	estimated ± 4.7 pp, high confidence
97	MiMo-V2.5-Pro	42.5%	estimated ± 4.7 pp, high confidence
98	Qwen3.8-27B	42.3%	measured
99	MiniMax M3	42.1%	measured
100	GPT-5.4 mini	42.0%	estimated ± 4.7 pp, high confidence
101	GPT-5.2-Codex	41.7%	estimated ± 4.7 pp, high confidence
102	Grok 4.20	41.6%	estimated ± 4.7 pp, high confidence
103	Claude Sonnet 4.5	41.6%	estimated ± 6.1 pp, medium confidence
104	Grok 4.3	41.3%	estimated ± 4.7 pp, high confidence
105	MiMo-V2.5	41.1%	estimated ± 4.7 pp, high confidence
106	Qwen3.7 Plus	41.1%	measured
107	GPT-5.4 nano	40.7%	estimated ± 4.7 pp, high confidence
108	Granite 4.2 30B	40.5%	estimated ± 3.1 pp, low confidence
109	GLM-4.7	40.5%	estimated ± 4.7 pp, high confidence
110	Ling 3.0 Flash VL	40.5%	estimated ± 6.5 pp, medium confidence
111	MiniMax M2.7	39.8%	measured
112	Claude Haiku 4.5	39.6%	estimated ± 4.7 pp, medium confidence
113	Mistral Medium 3.5 128B	39.5%	estimated ± 4.7 pp, medium confidence
114	Granite 4.2 8B	39.5%	estimated ± 3.1 pp, low confidence
115	Ling 3.0 Flash	39.2%	estimated ± 4.7 pp, medium confidence
116	Qwen3.5 Flash	38.9%	estimated ± 4.7 pp, medium confidence
117	Gemma 4 12B	38.9%	estimated ± 3.1 pp, low confidence
118	Gemini 3.1 Flash-Lite	38.5%	estimated ± 4.7 pp, medium confidence
119	Mellum2-12B-A2.5B-Thinking	38.0%	estimated ± 3.1 pp, low confidence
120	Granite 4.2 3B	37.9%	estimated ± 3.1 pp, low confidence
121	MiniCPM5-2B	37.7%	estimated ± 3.1 pp, low confidence
122	Laguna M.1	37.5%	estimated ± 4.7 pp, medium confidence
123	Laguna XS.2	37.2%	estimated ± 4.7 pp, medium confidence
124	Gemini 2.5 Pro	37.1%	estimated ± 4.7 pp, medium confidence
125	MiMo-V2-Omni	36.8%	estimated ± 6.1 pp, medium confidence
126	Ling 3.0 Flash FP8	36.6%	estimated ± 5.1 pp, medium confidence
127	BTL-4	36.3%	estimated ± 3.1 pp, low confidence
128	Claude 4.1 Opus	36.2%	estimated ± 6.1 pp, medium confidence
129	Qwen3.6-27B	36.2%	measured
130	ZAYA1-8B	36.2%	estimated ± 3.1 pp, low confidence
131	ZAYA1-74B-Preview	36.1%	estimated ± 3.1 pp, low confidence
132	MiMo-V2-Flash	36.0%	estimated ± 5.1 pp, medium confidence
133	GPT-5.1	35.6%	estimated ± 5.1 pp, medium confidence
134	Muse Glimmer 30B	35.3%	estimated ± 5.1 pp, medium confidence
135	Ornith-1.0-35B	34.6%	measured
136	Kimi K2.5 (Reasoning)	33.6%	estimated ± 5.1 pp, medium confidence
137	Agents-A1-4B	33.3%	estimated ± 3.1 pp, low confidence
138	LFM2.5-2.6B	33.2%	estimated ± 3.1 pp, low confidence
139	Claude 4 Sonnet	32.8%	estimated ± 6.1 pp, medium confidence
140	Qwen3.5-122B-A10B	32.8%	estimated ± 5.1 pp, medium confidence
141	MAI-Code-1.1-Flash	32.6%	estimated ± 6.1 pp, medium confidence
142	Ornith-1.5-9B	32.4%	measured
143	Qwen3.5-27B	32.2%	estimated ± 6.1 pp, medium confidence
144	DeepSeek V3 0324	32.2%	estimated ± 6.5 pp, medium confidence
145	North Mini Code	31.8%	estimated ± 6.5 pp, medium confidence
146	Mercury 2.5	31.4%	estimated ± 6.5 pp, medium confidence
147	Gemma 4 31B	31.0%	estimated ± 5.1 pp, medium confidence
148	Laguna XS 2.1	29.5%	estimated ± 6.1 pp, medium confidence
149	Qwen3.6-35B-A3B	29.4%	measured
150	Grok Code Fast 1	29.3%	estimated ± 6.1 pp, medium confidence
151	Solar Pro 4	28.9%	estimated ± 6.1 pp, medium confidence
152	o1	28.1%	estimated ± 5.1 pp, low confidence
153	Step 3.7 Flash	28.0%	estimated ± 5.1 pp, low confidence
154	Gemma 4 26B A4B	27.8%	estimated ± 5.1 pp, low confidence
155	Ornith-1.0-9B	27.2%	measured
156	GPT-5 (high)	26.6%	estimated ± 5.1 pp, low confidence
157	Nemotron 3 Super 100B	26.5%	estimated ± 5.1 pp, low confidence
158	Qwen3.5-35B-A3B	26.5%	estimated ± 6.1 pp, low confidence
159	K-EXAONE 2.0	24.8%	estimated ± 6.1 pp, low confidence
160	LongCat-Flash-Lite-Sparse	24.8%	estimated ± 6.1 pp, low confidence
161	o1-preview	23.7%	estimated ± 5.1 pp, low confidence
162	Mellum2-12B-A2.5B-Instruct	22.2%	estimated ± 3.1 pp, low confidence
163	K-Exaone	22.2%	estimated ± 5.1 pp, low confidence
164	GPT-OSS 120B	20.9%	estimated ± 5.1 pp, low confidence
165	MiniCPM5-1B	20.2%	estimated ± 3.1 pp, low confidence
166	Command A+	18.8%	estimated ± 5.1 pp, low confidence
167	Nemotron 3.5 Lightning 30B A3B NVFP4	18.0%	estimated ± 5.1 pp, low confidence
168	Mistral Small 4	17.9%	estimated ± 5.1 pp, low confidence
169	Mistral Small 4 (Reasoning)	17.9%	estimated ± 5.1 pp, low confidence
170	LLaDA2.2-mini	17.3%	estimated ± 3.1 pp, low confidence
171	Trinity-Large-Preview	17.2%	estimated ± 5.1 pp, low confidence
172	Trinity-Large-Thinking	17.2%	estimated ± 5.1 pp, low confidence
173	Ling 2.6 Flash	16.8%	estimated ± 5.1 pp, low confidence
174	Gemini 1.5 Pro	15.6%	estimated ± 5.1 pp, low confidence
175	DeepSeek V3	15.1%	estimated ± 5.1 pp, low confidence
176	GPT-4 Turbo	13.9%	estimated ± 5.1 pp, low confidence
177	GPT-OSS 20B	13.3%	estimated ± 5.1 pp, low confidence
178	GPT-4.1 mini	12.9%	estimated ± 5.1 pp, low confidence
179	Mistral Large 3	12.8%	estimated ± 5.1 pp, low confidence
180	Claude 3 Opus	12.4%	estimated ± 5.1 pp, low confidence
181	Solar Pro 3	10.6%	estimated ± 6.5 pp, low confidence
182	Llama 4 Maverick	9.8%	estimated ± 5.1 pp, low confidence
183	GPT-4.1	8.6%	estimated ± 6.1 pp, low confidence
184	Ling 3.0 Tiny	8.5%	estimated ± 6.5 pp, low confidence
185	Celeris-1	8.4%	estimated ± 5.1 pp, low confidence
186	Nemotron 3 Nano 30B	8.3%	estimated ± 5.1 pp, low confidence
187	Nemotron 3 Nano Omni 30B A3B	7.9%	estimated ± 5.1 pp, low confidence
188	Ultravox v0.6 Llama 3.3 70B	6.4%	estimated ± 5.1 pp, low confidence
189	GPT-4o mini	6.0%	estimated ± 5.1 pp, low confidence
190	GPT-4.1 nano	5.8%	estimated ± 5.1 pp, low confidence
191	o3-mini	5.4%	estimated ± 6.1 pp, low confidence
192	LLaDA2.2-flash	5.4%	estimated ± 6.1 pp, low confidence
193	Claude 3.5 Sonnet	5.2%	estimated ± 6.1 pp, low confidence
194	Gemma 3 27B	5.0%	estimated ± 5.1 pp, low confidence
195	Gemma 4 E4B	4.5%	estimated ± 5.1 pp, low confidence
196	Llama 4 Scout	3.5%	estimated ± 5.1 pp, low confidence
197	Gemma 4 E2B	2.8%	estimated ± 5.1 pp, low confidence

NL2Repo leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General