Coding

ScarfBench leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap
License: https://creativecommons.org/licenses/by-nc/4.0/

As of 2026-11-03, the highest measured score on ScarfBench is 75.2% by Gemini 4 Argon. 142 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	Gemini 4 Argon	75.2%	measured
2	Muse Wonder Full	71.7%	estimated ± 3.8 pp, low confidence
3	Claude Opus 5.5	70.5%	measured
4	Gemini 4 Carbon	70.4%	estimated ± 4.9 pp, low confidence
5	Claude Sonnet 5.5	69.9%	estimated ± 3.8 pp, medium confidence
6	GPT-6 Sol	69.9%	estimated ± 3.8 pp, medium confidence
7	Claude Fable 5.1	69.9%	estimated ± 3.8 pp, medium confidence
8	Grok 4.7	69.9%	estimated ± 3.8 pp, medium confidence
9	Claude Haiku 5.5	69.9%	estimated ± 3.8 pp, medium confidence
10	GPT-5.6 Sol	69.9%	estimated ± 3.8 pp, medium confidence
11	Muse Wonder	69.9%	estimated ± 3.8 pp, medium confidence
12	SWE-2	69.9%	estimated ± 3.8 pp, medium confidence
13	GPT-5.5	69.9%	estimated ± 3.8 pp, medium confidence
14	GPT-6 Luna	69.8%	estimated ± 3.8 pp, medium confidence
15	Claude Opus 5	69.8%	measured
16	Kimi K3.1 Pro	69.7%	estimated ± 3.8 pp, medium confidence
17	Muse Spark 1.3	69.6%	estimated ± 3.8 pp, medium confidence
18	Ember-1	69.5%	estimated ± 3.8 pp, medium confidence
19	Mistral Large 4	69.4%	estimated ± 3.8 pp, medium confidence
20	DeepSeek V4.1 Flash 1017	69.0%	measured
21	Pareto 26.9	68.8%	estimated ± 3.8 pp, medium confidence
22	MiniMax M3.1 Flash	68.6%	estimated ± 3.8 pp, medium confidence
23	GPT-6.1 Sol	68.4%	measured
24	Particle-1	67.9%	estimated ± 3.8 pp, medium confidence
25	MiMo-V2.6-Pro	67.5%	estimated ± 3.8 pp, medium confidence
26	Pareto 26.10 Preview	66.2%	estimated ± 3.8 pp, medium confidence
27	GPT-5.6 Terra	66.0%	estimated ± 3.8 pp, medium confidence
28	Gemini 3.8 Flash	66.0%	measured
29	GPT-6 Astra	65.4%	measured
30	MiMo-V2.6-Flash	64.9%	estimated ± 3.8 pp, medium confidence
31	Step 5 Preview	64.8%	estimated ± 3.8 pp, medium confidence
32	GPT-5.6 Luna	64.5%	estimated ± 3.8 pp, low confidence
33	GLM-5.3	64.3%	estimated ± 3.8 pp, low confidence
34	Claude Fable 5	63.7%	estimated ± 4.9 pp, low confidence
35	Inkling	63.7%	estimated ± 4.9 pp, low confidence
36	Qwen3.8 Max	63.7%	estimated ± 4.9 pp, low confidence
37	Grok 4.6	63.6%	estimated ± 3.8 pp, low confidence
38	Gemini 3.7 Flash	63.2%	estimated ± 3.8 pp, low confidence
39	GLM-5.2	62.9%	estimated ± 5.5 pp, low confidence
40	Hy4 preview	62.6%	estimated ± 3.8 pp, low confidence
41	GLM-5.3-Flash	62.0%	estimated ± 3.8 pp, low confidence
42	DeepSeek V4 Pro 0813	61.5%	estimated ± 3.8 pp, low confidence
43	Kimi K3	61.3%	measured
44	Ultramarine 1 Flash	60.7%	estimated ± 3.8 pp, low confidence
45	Muse Spark Open	59.6%	estimated ± 3.8 pp, low confidence
46	Ling 3.1 Flash	59.4%	estimated ± 3.8 pp, low confidence
47	Muse Spark 1.2	59.1%	estimated ± 3.8 pp, low confidence
48	Qwen3.8-Flash-Next	58.7%	estimated ± 3.8 pp, low confidence
49	GPT-6.1 Terra	58.5%	estimated ± 14.0 pp, low confidence
50	Qwen3.8-Omni-Flash	58.1%	estimated ± 3.8 pp, low confidence
51	Claude Opus 4.7	57.2%	estimated ± 5.5 pp, low confidence
52	Claude Opus 4.7 (Adaptive)	53.1%	estimated ± 9.1 pp, low confidence
53	SWE-1.7	50.4%	estimated ± 3.8 pp, low confidence
54	Beam 1.0 501B-A23B	49.0%	estimated ± 14.0 pp, low confidence
55	Solar Pro 4	48.3%	estimated ± 14.0 pp, low confidence
56	Claude Opus 4.8	48.3%	estimated ± 3.8 pp, low confidence
57	Nemotron 3.5 Super VL 120B A12B	48.0%	estimated ± 14.0 pp, low confidence
58	Nemotron 3.5 Super VL 120B A12B NVFP4	48.0%	estimated ± 14.0 pp, low confidence
59	Qwen3.8 Max Preview	46.5%	estimated ± 9.1 pp, low confidence
60	Claude Sonnet 5	45.6%	estimated ± 9.1 pp, low confidence
61	Muse Spark 1.1	44.9%	estimated ± 9.1 pp, low confidence
62	Qwen3.5 397B	44.0%	estimated ± 14.0 pp, low confidence
63	Qwen3.5 397B (Reasoning)	44.0%	estimated ± 14.0 pp, low confidence
64	Ling 3.0 Flash VL	43.2%	estimated ± 14.0 pp, low confidence
65	Grok 4.5	42.4%	estimated ± 5.5 pp, low confidence
66	Gemini 3.5 Flash	41.2%	estimated ± 9.1 pp, low confidence
67	K-EXAONE 2.0	40.4%	estimated ± 14.0 pp, low confidence
68	A.X K2	39.1%	estimated ± 14.0 pp, low confidence
69	Gemini 3.6 Flash	38.7%	estimated ± 9.1 pp, low confidence
70	DeepSeek V4 Flash 0731	38.2%	estimated ± 9.1 pp, low confidence
71	Gemini 3.1 Pro	37.6%	measured
72	Nemotron 3.5 Bolt 74B-A9B	37.5%	estimated ± 14.0 pp, low confidence
73	DeepSeek V3 0324	36.7%	estimated ± 14.0 pp, low confidence
74	North Mini Code	36.4%	estimated ± 14.0 pp, low confidence
75	Mercury 2.5	36.0%	estimated ± 14.0 pp, low confidence
76	Qwen3.8-27B	35.8%	estimated ± 9.1 pp, low confidence
77	Granite 4.2 30B	35.2%	estimated ± 14.0 pp, low confidence
78	Qwen3.7 Max	31.9%	estimated ± 9.1 pp, low confidence
79	GPT-5.4	28.7%	estimated ± 5.5 pp, low confidence
80	Kimi K2.6	27.4%	estimated ± 9.1 pp, low confidence
81	Quasar 438B	27.0%	estimated ± 9.1 pp, low confidence
82	Apodex 1.1	26.8%	estimated ± 9.1 pp, low confidence
83	Apodex 1.1 Mini	26.8%	estimated ± 9.1 pp, low confidence
84	Kimi K2.7 Code	26.8%	estimated ± 9.1 pp, low confidence
85	MiMo-V2.5-Pro	26.5%	estimated ± 9.1 pp, low confidence
86	Hy3	25.9%	estimated ± 9.1 pp, low confidence
87	Hy3 Preview	25.9%	estimated ± 9.1 pp, low confidence
88	Muse Spark	25.9%	estimated ± 9.1 pp, low confidence
89	MiniMax M3	25.8%	estimated ± 9.1 pp, low confidence
90	GPT-5.4 mini	25.2%	estimated ± 9.1 pp, low confidence
91	GPT-5.4 nano	25.2%	estimated ± 9.1 pp, low confidence
92	Qwen3.7 Plus	25.2%	estimated ± 9.1 pp, low confidence
93	GLM-5.1	25.2%	estimated ± 9.1 pp, low confidence
94	Qwen3.6 Plus	25.0%	estimated ± 9.1 pp, low confidence
95	Qwen3.6-27B	24.9%	estimated ± 9.1 pp, low confidence
96	Inkling-Small	24.8%	estimated ± 9.1 pp, low confidence
97	MiniMax M2.7	24.8%	estimated ± 9.1 pp, low confidence
98	Ling 3.0 Flash	24.6%	estimated ± 9.1 pp, low confidence
99	MiMo-V2-Flash	24.6%	estimated ± 9.1 pp, low confidence
100	GPT-5.1	24.6%	estimated ± 9.1 pp, low confidence
101	Gemini 3.5 Flash-Lite	24.6%	estimated ± 9.1 pp, low confidence
102	Nemotron 3 Ultra	24.6%	estimated ± 9.1 pp, low confidence
103	Muse Glimmer 30B	24.5%	estimated ± 9.1 pp, low confidence
104	Mistral Medium 3.5 128B	24.5%	estimated ± 9.1 pp, low confidence
105	Kimi K2.5	24.5%	estimated ± 9.1 pp, low confidence
106	Kimi K2.5 (Reasoning)	24.5%	estimated ± 9.1 pp, low confidence
107	Qwen3.5-122B-A10B	24.5%	estimated ± 9.1 pp, low confidence
108	GLM-4.7	24.4%	estimated ± 9.1 pp, low confidence
109	Gemma 4 31B	24.4%	estimated ± 9.1 pp, low confidence
110	Grok 4.3	24.4%	estimated ± 9.1 pp, low confidence
111	Qwen3.6-35B-A3B	24.4%	estimated ± 9.1 pp, low confidence
112	Gemma 4 26B A4B	24.4%	measured
113	o1	24.4%	estimated ± 9.1 pp, low confidence
114	Step 3.7 Flash	24.4%	estimated ± 9.1 pp, low confidence
115	GPT-5 (high)	24.4%	estimated ± 9.1 pp, low confidence
116	o1-preview	24.4%	estimated ± 9.1 pp, low confidence
117	Gemini 2.5 Pro	24.4%	estimated ± 9.1 pp, low confidence
118	K-Exaone	24.4%	estimated ± 9.1 pp, low confidence
119	Gemma 4 12B	24.4%	estimated ± 9.1 pp, low confidence
120	GPT-OSS 120B	24.4%	estimated ± 9.1 pp, low confidence
121	Command A+	24.4%	estimated ± 9.1 pp, low confidence
122	Nemotron 3.5 Lightning 30B A3B NVFP4	24.4%	estimated ± 9.1 pp, low confidence
123	Mistral Small 4	24.4%	estimated ± 9.1 pp, low confidence
124	Mistral Small 4 (Reasoning)	24.4%	estimated ± 9.1 pp, low confidence
125	Trinity-Large-Preview	24.4%	estimated ± 9.1 pp, low confidence
126	Trinity-Large-Thinking	24.4%	estimated ± 9.1 pp, low confidence
127	Ling 2.6 Flash	24.4%	estimated ± 9.1 pp, low confidence
128	Gemini 1.5 Pro	24.4%	estimated ± 9.1 pp, low confidence
129	DeepSeek V3	24.4%	estimated ± 9.1 pp, low confidence
130	Granite 4.2 8B	24.4%	estimated ± 9.1 pp, low confidence
131	GPT-4 Turbo	24.4%	estimated ± 9.1 pp, low confidence
132	GPT-OSS 20B	24.4%	estimated ± 9.1 pp, low confidence
133	GPT-4.1 mini	24.4%	estimated ± 9.1 pp, low confidence
134	Mistral Large 3	24.4%	estimated ± 9.1 pp, low confidence
135	Claude 3 Opus	24.4%	estimated ± 9.1 pp, low confidence
136	Llama 4 Maverick	24.4%	estimated ± 9.1 pp, low confidence
137	Celeris-1	24.4%	estimated ± 9.1 pp, low confidence
138	Nemotron 3 Nano 30B	24.4%	estimated ± 9.1 pp, low confidence
139	Nemotron 3 Nano Omni 30B A3B	24.4%	estimated ± 9.1 pp, low confidence
140	Ultravox v0.6 Llama 3.3 70B	24.4%	estimated ± 9.1 pp, low confidence
141	GPT-4o mini	24.4%	estimated ± 9.1 pp, low confidence
142	GPT-4.1 nano	24.4%	estimated ± 9.1 pp, low confidence
143	Gemma 3 27B	24.4%	estimated ± 9.1 pp, low confidence
144	Gemma 4 E4B	24.4%	estimated ± 9.1 pp, low confidence
145	Llama 4 Scout	24.4%	estimated ± 9.1 pp, low confidence
146	LFM2.5-2.6B	24.4%	estimated ± 9.1 pp, low confidence
147	Gemma 4 E2B	24.4%	estimated ± 9.1 pp, low confidence
148	Mistral Small 3.1	23.8%	estimated ± 14.0 pp, low confidence
149	MiniCPM5-2B	22.2%	estimated ± 14.0 pp, low confidence
150	Solar Pro 3	21.4%	estimated ± 14.0 pp, low confidence
151	Granite 4.2 3B	21.2%	estimated ± 14.0 pp, low confidence
152	Ling 3.0 Tiny	20.0%	estimated ± 14.0 pp, low confidence

ScarfBench leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General