benchgap
Coding

NL2Repo leaderboard

As of 2026-11-21, the highest measured score on NL2Repo is 68.2% by DeepSeek V4.1 Flash 1017. 214 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#ModelScoreSource
1Gemini 4 Neon75.1%estimated ± 6.5 pp, low confidence
2GPT-6.2 Astra75.0%estimated ± 6.5 pp, low confidence
3Gemini 4 Carbon73.8%estimated ± 6.5 pp, low confidence
4GPT-6.2 Sol73.1%estimated ± 6.5 pp, low confidence
5Claude Mythos 571.7%estimated ± 6.1 pp, low confidence
6DeepSeek V4.1 Flash 101768.2%measured
7ERNIE 5.5 Flash Preview64.8%measured
8Muse Wonder Full64.7%estimated ± 6.0 pp, low confidence
9Muse Wonder64.6%estimated ± 6.0 pp, low confidence
10Gemini 4 Hydrogen64.6%estimated ± 6.5 pp, low confidence
11Gemini 4 Argon64.2%estimated ± 6.0 pp, low confidence
12Qwen4 Max63.3%estimated ± 6.0 pp, medium confidence
13Kimi K3.1 Pro63.2%estimated ± 6.0 pp, medium confidence
14Ember-163.0%estimated ± 6.0 pp, medium confidence
15Hy462.9%estimated ± 6.0 pp, medium confidence
16Claude Fable 5.262.7%estimated ± 6.0 pp, medium confidence
17Claude Opus 5.562.6%estimated ± 6.0 pp, medium confidence
18Pareto 26.962.5%estimated ± 6.0 pp, medium confidence
19MAI Code 262.2%estimated ± 6.0 pp, medium confidence
20Sakana Fugu-Ultra62.1%estimated ± 2.8 pp, medium confidence
21SWE-262.1%estimated ± 6.0 pp, medium confidence
22Particle-161.9%estimated ± 6.0 pp, medium confidence
23Inkling 2 Small61.7%estimated ± 6.0 pp, medium confidence
24GPT-6.1 Sol61.6%estimated ± 6.0 pp, medium confidence
25MiMo-V2.6-Pro61.6%estimated ± 6.0 pp, medium confidence
26DeepSeek V4 Pro61.5%measured
27Claude Sonnet 5.561.2%estimated ± 6.0 pp, medium confidence
28Grok 4.761.2%estimated ± 6.0 pp, medium confidence
29MiniMax M3.1 Flash61.2%measured
30Claude Opus 561.1%estimated ± 4.7 pp, medium confidence
31Claude Fable 5.161.0%estimated ± 4.7 pp, medium confidence
32Pareto 26.10 Preview60.8%estimated ± 6.0 pp, medium confidence
33LongCat-2.560.5%measured
34GPT-6 Sol60.3%estimated ± 6.0 pp, medium confidence
35GPT-5.6 Sol60.2%estimated ± 4.7 pp, high confidence
36MiMo-V2.6-Flash59.9%estimated ± 6.0 pp, medium confidence
37Step 5 Preview59.8%estimated ± 6.0 pp, medium confidence
38Grok 4.659.6%estimated ± 4.7 pp, high confidence
39Ornith-1.5-397B59.5%measured
40GPT-6 Luna59.4%estimated ± 6.0 pp, medium confidence
41GPT-5.6 Terra59.3%estimated ± 4.7 pp, high confidence
42Claude Fable 558.9%estimated ± 4.7 pp, high confidence
43Hy4 preview58.9%measured
44GPT-6.1 Terra58.7%estimated ± 7.4 pp, medium confidence
45Hy4 flash58.5%estimated ± 6.0 pp, medium confidence
46GLM-5.358.0%measured
47GPT-6 Astra57.2%estimated ± 4.7 pp, medium confidence
48Kimi K357.2%estimated ± 4.7 pp, high confidence
49Ultramarine 1 Flash57.2%estimated ± 6.0 pp, medium confidence
50GPT-5.6 Luna56.8%estimated ± 4.7 pp, high confidence
51SWE-2 Flash56.6%estimated ± 6.0 pp, medium confidence
52Muse Spark Open56.5%estimated ± 6.0 pp, medium confidence
53MAI Code 2 Flash56.5%estimated ± 6.0 pp, medium confidence
54Ling 3.1 Flash56.4%estimated ± 6.0 pp, medium confidence
55Muse Spark 1.356.3%estimated ± 4.7 pp, medium confidence
56GLM-5.3-Flash56.3%measured
57Claude Haiku 5.556.1%estimated ± 6.5 pp, low confidence
58Qwen3.8 Max55.9%measured
59DeepSeek V4 Flash 073154.2%measured
60Qwen3.8 Max Preview53.1%estimated ± 4.7 pp, high confidence
61Claude Opus 4.852.4%estimated ± 4.7 pp, high confidence
62Inkling 2 Mini51.5%estimated ± 6.0 pp, medium confidence
63Mercury 350.7%estimated ± 6.0 pp, medium confidence
64Grok 4.550.6%estimated ± 4.7 pp, high confidence
65Muse Spark 1.250.6%estimated ± 4.7 pp, high confidence
66Ripple-65B Preview49.8%estimated ± 6.0 pp, medium confidence
67dots3-note Preview49.8%measured
68Beam 1.0 501B-A23B49.4%estimated ± 6.0 pp, medium confidence
69Beam 1.0 501B-A23B FP849.3%estimated ± 6.0 pp, medium confidence
70Beam 1.0 501B-A23B NVFP449.1%estimated ± 6.0 pp, medium confidence
71GLM-5.248.9%measured
72Qwen3.8-Omni-Flash48.9%measured
73Sage-A1.548.4%estimated ± 6.1 pp, medium confidence
74Solar Open 248.3%estimated ± 3.4 pp, high confidence
75Ornith-1.0-397B48.2%measured
76Laguna S 2.148.2%estimated ± 6.0 pp, medium confidence
77Qwen3.8-Flash-Next48.1%measured
78GPT-5.547.4%estimated ± 4.7 pp, high confidence
79Qwen3.7 Max47.2%measured
80Inkling-Small47.1%estimated ± 4.7 pp, high confidence
81Seed 2.1 Pro47.0%measured
82Claude Opus 4.746.9%estimated ± 4.7 pp, high confidence
83Muse Spark 1.146.9%estimated ± 4.7 pp, high confidence
84Atria Dawn Preview46.4%estimated ± 6.1 pp, medium confidence
85Ornith-1.5-35B-A3B46.2%measured
86GPT-5.1-Codex46.1%estimated ± 7.9 pp, low confidence
87Gemini 3.7 Flash46.0%estimated ± 4.7 pp, high confidence
88o345.7%estimated ± 7.9 pp, low confidence
89GPT-5.1-Codex-Max45.7%estimated ± 7.9 pp, low confidence
90o4-mini45.7%estimated ± 7.9 pp, low confidence
91GLM-4.645.7%estimated ± 7.9 pp, low confidence
92Qwen3 Max45.7%estimated ± 7.9 pp, low confidence
93o3-mini45.7%estimated ± 7.9 pp, low confidence
94Claude 3.7 Sonnet Thinking45.7%estimated ± 7.9 pp, low confidence
95GLM-4.545.7%estimated ± 7.9 pp, low confidence
96Gemini 3.8 Flash45.5%estimated ± 4.7 pp, high confidence
97Claude Sonnet 545.2%estimated ± 4.7 pp, high confidence
98Composer 2.545.2%estimated ± 4.7 pp, high confidence
99Gemini 3.6 Flash45.2%estimated ± 4.7 pp, high confidence
100Gemini 3.1 Pro44.7%estimated ± 4.7 pp, high confidence
101Gemini 3.5 Flash44.7%estimated ± 4.7 pp, high confidence
102Quasar 438B44.6%estimated ± 4.7 pp, high confidence
103Ternary Bonsai 2 27B44.4%estimated ± 3.4 pp, high confidence
104Claude Opus 4.6 (Adaptive)44.3%estimated ± 4.7 pp, high confidence
105GPT-5.444.3%estimated ± 4.7 pp, high confidence
106Kimi K2.7 Code44.3%estimated ± 4.7 pp, high confidence
107Apodex 1.144.3%estimated ± 4.7 pp, high confidence
108Apodex 1.1 Mini44.3%estimated ± 4.7 pp, high confidence
109GPT-5.3 Codex44.2%estimated ± 4.7 pp, high confidence
110Inkling43.9%estimated ± 4.7 pp, high confidence
111Claude Sonnet 4.643.8%estimated ± 4.7 pp, high confidence
112Sakana Fugu43.8%estimated ± 2.8 pp, medium confidence
113Seed 2.1 Turbo43.7%measured
114Claude Opus 4.5 Thinking43.2%estimated ± 4.7 pp, high confidence
115Gemini 3 Pro43.2%estimated ± 4.7 pp, high confidence
116Claude Opus 4.543.2%measured
117Qwen 3.6 Max (preview)42.9%measured
118GPT-5.242.9%estimated ± 4.7 pp, high confidence
119Kimi K2.642.8%estimated ± 2.8 pp, high confidence
120GLM-5.142.7%measured
121Hy342.7%estimated ± 4.7 pp, high confidence
122Hy3 Preview42.7%estimated ± 4.7 pp, high confidence
123Nemotron 3 Ultra42.6%estimated ± 3.4 pp, high confidence
124Gemini 3.5 Flash-Lite42.5%estimated ± 4.7 pp, high confidence
125Gemini 3 Flash42.5%estimated ± 4.7 pp, high confidence
126BTL-442.4%estimated ± 6.2 pp, medium confidence
127Qwen3.8-27B42.3%measured
128Muse Spark42.1%estimated ± 4.7 pp, high confidence
129MiniMax M342.1%measured
130MiMo-V2.5-Pro41.9%estimated ± 4.7 pp, high confidence
131MiMo-V2-Pro41.6%estimated ± 6.2 pp, medium confidence
132GPT-5.4 mini41.5%estimated ± 4.7 pp, high confidence
133GPT-5.2-Codex41.2%estimated ± 4.7 pp, high confidence
134BTL-341.1%estimated ± 3.4 pp, high confidence
135Qwen3.7 Plus41.1%measured
136Grok 4.2041.1%estimated ± 4.7 pp, high confidence
137Grok 4.340.7%estimated ± 4.7 pp, high confidence
138MiMo-V2.540.5%estimated ± 4.7 pp, high confidence
139Claude Sonnet 4.5 Thinking40.1%estimated ± 4.7 pp, high confidence
140GPT-5.140.1%estimated ± 4.7 pp, high confidence
141GPT-5.4 nano40.1%estimated ± 4.7 pp, high confidence
142Spark-X2.6-4B40.0%estimated ± 6.1 pp, medium confidence
143Claude Sonnet 4.539.9%estimated ± 6.2 pp, medium confidence
144MiniMax M2.739.8%measured
145GPT-539.7%estimated ± 4.7 pp, high confidence
146Ling 3.0 Flash VL39.5%estimated ± 7.4 pp, medium confidence
147Claude Haiku 4.538.9%estimated ± 4.7 pp, medium confidence
148Claude Opus 4.638.9%estimated ± 6.1 pp, medium confidence
149Mistral Medium 3.5 128B38.8%estimated ± 4.7 pp, medium confidence
150Ling 3.0 Flash38.5%estimated ± 4.7 pp, medium confidence
151Step 3.5 Flash38.4%estimated ± 3.4 pp, high confidence
152Qwen3.5 Flash38.2%estimated ± 4.7 pp, medium confidence
153Qwen3.6 Plus37.9%measured
154Gemini 3.1 Flash-Lite37.8%estimated ± 4.7 pp, medium confidence
155Grok 436.8%estimated ± 4.7 pp, medium confidence
156Laguna M.136.8%estimated ± 4.7 pp, medium confidence
157Laguna XS.236.4%estimated ± 4.7 pp, medium confidence
158Gemini 2.5 Pro36.3%estimated ± 4.7 pp, medium confidence
159Qwen3.6-27B36.2%measured
160GLM-4.736.1%estimated ± 3.4 pp, high confidence
161GLM-535.9%measured
162Grok 4 Fast (Reasoning)35.6%estimated ± 4.7 pp, medium confidence
163Grok 4.1 Fast (Reasoning)35.4%estimated ± 4.7 pp, medium confidence
164Mistral Large 335.4%estimated ± 4.7 pp, medium confidence
165MAI-Thinking-135.4%estimated ± 2.8 pp, high confidence
166MiMo-V2-Omni35.0%estimated ± 6.2 pp, medium confidence
167Muse Glimmer 30B34.8%estimated ± 4.7 pp, high confidence
168A.X K234.8%estimated ± 3.4 pp, high confidence
169Ornith-1.0-35B34.6%measured
170Claude 4.1 Opus34.5%estimated ± 6.2 pp, medium confidence
171Nemotron 3.5 Super VL 120B A12B34.2%estimated ± 2.8 pp, high confidence
172Nemotron 3.5 Super VL 120B A12B NVFP433.5%estimated ± 2.8 pp, high confidence
173Ornith-1.5-9B32.4%measured
174Qwen3.5 397B32.2%measured
175Kimi K2.532.0%measured
176Mellum2.1-12B-A2.5B-Thinking31.8%estimated ± 3.4 pp, high confidence
177Laguna XS 2.131.8%estimated ± 6.1 pp, medium confidence
178Nemotron 3.5 Bolt 74B-A9B31.6%estimated ± 7.4 pp, medium confidence
179Jais-3-MoVA-36B-A4B31.2%estimated ± 6.2 pp, medium confidence
180Claude 4 Sonnet31.0%estimated ± 6.2 pp, medium confidence
181MAI-Code-1.1-Flash30.8%estimated ± 6.2 pp, medium confidence
182Claude 4 Opus30.6%estimated ± 6.2 pp, medium confidence
183Jais-3-32B30.4%estimated ± 6.2 pp, medium confidence
184DeepSeek V3 032430.4%estimated ± 7.4 pp, medium confidence
185Qwen3.5-27B30.0%estimated ± 3.4 pp, high confidence
186North Mini Code30.0%estimated ± 7.4 pp, medium confidence
187MiMo-V2-Flash29.9%estimated ± 3.4 pp, high confidence
188Mercury 2.529.5%estimated ± 7.4 pp, medium confidence
189Flint29.5%estimated ± 3.4 pp, medium confidence
190Qwen3.6-35B-A3B29.4%measured
191Gemma 4 31B29.1%estimated ± 3.4 pp, medium confidence
192Qwen3.5-122B-A10B27.7%estimated ± 3.4 pp, medium confidence
193Grok Code Fast 127.5%estimated ± 6.2 pp, medium confidence
194o127.4%estimated ± 4.7 pp, medium confidence
195Step 3.7 Flash27.3%estimated ± 4.7 pp, medium confidence
196Ornith-1.0-9B27.2%measured
197Solar Pro 427.1%estimated ± 6.2 pp, medium confidence
198Jais-3-375B-A23B26.4%estimated ± 6.2 pp, medium confidence
199Jais-3-7B26.4%estimated ± 6.2 pp, medium confidence
200Gemma 4 26B A4B25.4%estimated ± 3.4 pp, medium confidence
201Qwen3-Coder 480B-A35B25.4%estimated ± 6.2 pp, medium confidence
202Granite 4.2 30B23.9%estimated ± 3.4 pp, medium confidence
203LongCat-Flash-Lite-Sparse23.3%estimated ± 6.1 pp, low confidence
204K-EXAONE 2.023.1%estimated ± 6.2 pp, low confidence
205o1-preview22.8%estimated ± 4.7 pp, medium confidence
206Qwen3.5-35B-A3B22.6%estimated ± 3.4 pp, medium confidence
207LFM3-1.7B21.4%estimated ± 3.4 pp, medium confidence
208K-Exaone21.3%estimated ± 4.7 pp, medium confidence
209Granite 4.2 8B21.2%estimated ± 3.4 pp, medium confidence
210Gemma 4 12B20.0%estimated ± 3.4 pp, medium confidence
211GPT-OSS 120B19.9%estimated ± 4.7 pp, medium confidence
212DeepSeek V3.119.6%estimated ± 6.2 pp, low confidence
213Kimi K219.3%estimated ± 6.2 pp, low confidence
214Mellum2-12B-A2.5B-Thinking18.1%estimated ± 3.4 pp, medium confidence
215Granite 4.2 3B17.9%estimated ± 3.4 pp, medium confidence
216Command A+17.9%estimated ± 4.7 pp, medium confidence
217MiniCPM5-2B17.4%estimated ± 3.4 pp, medium confidence
218Nemotron 3.5 Lightning 30B A3B NVFP417.0%estimated ± 4.7 pp, medium confidence
219Mistral Small 416.9%estimated ± 4.7 pp, medium confidence
220Nemotron 3 Nano 30B16.8%estimated ± 3.4 pp, medium confidence
221Trinity-Large-Preview16.2%estimated ± 4.7 pp, medium confidence
222Trinity-Large-Thinking16.2%estimated ± 4.7 pp, medium confidence
223Ling 2.6 Flash15.8%estimated ± 4.7 pp, medium confidence
224Claude 3.7 Sonnet14.6%estimated ± 6.2 pp, low confidence
225Gemini 1.5 Pro14.5%estimated ± 4.7 pp, medium confidence
226DeepSeek V314.0%estimated ± 4.7 pp, medium confidence
227GPT-4 Turbo12.8%estimated ± 4.7 pp, medium confidence
228Nemotron 3 Super 100B12.5%estimated ± 6.2 pp, low confidence
229GPT-OSS 20B12.1%estimated ± 4.7 pp, medium confidence
230GPT-4.1 mini11.7%estimated ± 4.7 pp, medium confidence
231Claude 3 Opus11.2%estimated ± 4.7 pp, medium confidence
232GLM-4.7-Flash11.2%estimated ± 6.2 pp, low confidence
233Mistral Small 3.110.7%estimated ± 7.4 pp, low confidence
234LLaDA2.2-flash10.5%estimated ± 6.1 pp, low confidence
235DeepSeek-R1-05289.6%estimated ± 6.2 pp, low confidence
236Llama 4 Maverick8.6%estimated ± 4.7 pp, medium confidence
237MiniMax M1 80k8.3%estimated ± 6.2 pp, low confidence
238GPT-4.17.2%estimated ± 6.2 pp, low confidence
239Celeris-17.1%estimated ± 4.7 pp, medium confidence
240Solar Pro 36.7%estimated ± 7.4 pp, low confidence
241Nemotron 3 Nano Omni 30B A3B6.6%estimated ± 4.7 pp, medium confidence
242ZAYA1-74B-Preview6.2%estimated ± 6.2 pp, low confidence
243Ultravox v0.6 Llama 3.3 70B5.1%estimated ± 4.7 pp, medium confidence
244GPT-4o mini4.6%estimated ± 4.7 pp, medium confidence
245GPT-4.1 nano4.5%estimated ± 4.7 pp, medium confidence
246Ling 3.0 Tiny4.4%estimated ± 7.4 pp, low confidence
247Gemma 3 27B3.6%estimated ± 4.7 pp, medium confidence
248Gemma 4 E4B3.1%estimated ± 4.7 pp, medium confidence
249LFM2.5-2.6B0.0%estimated ± 7.4 pp, low confidence
250Llama 4 Scout0.0%estimated ± 7.4 pp, low confidence

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General