Coding

Bug Hunt Bench leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

As of 2026-10-09, the highest measured score on Bug Hunt Bench is 51.3% by Claude Sonnet 5.5. 99 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	Claude Sonnet 5.5	51.3%	measured
2	GPT-6 Astra	45.0%	measured
3	GPT-6.1 Sol	44.3%	measured
4	Gemini 4 Argon	44.3%	estimated ± 7.2 pp, low confidence
5	Claude Fable 5.1	43.0%	measured
6	GPT-5.6 Sol	42.0%	measured
7	Claude Opus 5.5	41.7%	measured
8	Step 5 Preview	36.2%	estimated ± 10.9 pp, low confidence
9	Claude Fable 5	35.9%	estimated ± 7.4 pp, medium confidence
10	Muse Spark 1.3	32.2%	measured
11	DeepSeek V4 Flash 0731	31.8%	estimated ± 8.6 pp, low confidence
12	GPT-5.6 Terra	31.4%	estimated ± 8.6 pp, low confidence
13	Grok 4.7	28.8%	measured
14	GPT-5.6 Luna	27.9%	estimated ± 8.6 pp, low confidence
15	Claude Opus 4.8	27.2%	estimated ± 7.2 pp, low confidence
16	Claude Opus 5	27.0%	measured
17	Grok 4.6	27.0%	measured
18	Claude Sonnet 5	26.7%	estimated ± 8.6 pp, low confidence
19	Qwen3.8 Max	25.7%	measured
20	Claude Opus 4.7 (Adaptive)	24.6%	estimated ± 9.0 pp, low confidence
21	GLM-5.2	24.5%	estimated ± 7.2 pp, low confidence
22	Qwen3.8-Flash-Next	23.7%	estimated ± 9.0 pp, low confidence
23	MiMo-V2.6-Flash	23.4%	estimated ± 10.9 pp, low confidence
24	MiMo-V2.6-Pro	22.7%	measured
25	Composer 2.5	22.2%	estimated ± 8.6 pp, low confidence
26	Gemini 3.7 Flash	22.2%	estimated ± 7.4 pp, medium confidence
27	Qwen3.8 Max Preview	21.8%	estimated ± 9.0 pp, low confidence
28	DeepSeek V4.1 Flash	21.7%	measured
29	Claude Haiku 5.5	21.5%	measured
30	Muse Spark 1.1	21.1%	estimated ± 9.0 pp, low confidence
31	Kimi K3	21.0%	measured
32	Gemini 3.5 Flash	19.4%	estimated ± 9.0 pp, low confidence
33	Claude Haiku 4.5	19.0%	estimated ± 8.6 pp, low confidence
34	GLM-5.3	19.0%	measured
35	Hy4 preview	18.6%	estimated ± 10.9 pp, low confidence
36	GPT-6 Luna	18.3%	measured
37	Gemini 3.6 Flash	18.2%	estimated ± 9.0 pp, low confidence
38	Gemini 3.8 Flash	18.0%	measured
39	Muse Spark 1.2	17.9%	estimated ± 7.4 pp, low confidence
40	GLM-5.3-Flash	17.7%	measured
41	DeepSeek V4 Pro 0813	17.6%	estimated ± 9.0 pp, low confidence
42	Claude Opus 4.7	17.0%	estimated ± 7.2 pp, low confidence
43	Qwen3.8-27B	15.0%	measured
44	Qwen3.7 Max	14.1%	estimated ± 9.0 pp, low confidence
45	GPT-5.5	14.0%	estimated ± 7.2 pp, low confidence
46	Inkling	13.9%	estimated ± 7.4 pp, low confidence
47	Kimi K2.6	9.9%	estimated ± 9.0 pp, low confidence
48	Quasar 438B	9.4%	estimated ± 9.0 pp, low confidence
49	Apodex 1.1	9.1%	estimated ± 9.0 pp, low confidence
50	Apodex 1.1 Mini	9.1%	estimated ± 9.0 pp, low confidence
51	Kimi K2.7 Code	9.1%	estimated ± 9.0 pp, low confidence
52	MiMo-V2.5-Pro	8.6%	estimated ± 9.0 pp, low confidence
53	Hy3	7.5%	estimated ± 9.0 pp, low confidence
54	Hy3 Preview	7.5%	estimated ± 9.0 pp, low confidence
55	Muse Spark	7.4%	estimated ± 9.0 pp, low confidence
56	MiniMax M3	7.4%	estimated ± 9.0 pp, low confidence
57	Grok 4.5	6.8%	estimated ± 7.2 pp, low confidence
58	GPT-5.4 mini	5.8%	estimated ± 9.0 pp, low confidence
59	GPT-5.4 nano	5.8%	estimated ± 9.0 pp, low confidence
60	Qwen3.7 Plus	5.6%	estimated ± 9.0 pp, low confidence
61	GLM-5.1	5.6%	estimated ± 9.0 pp, low confidence
62	Qwen3.6 Plus	4.9%	estimated ± 9.0 pp, low confidence
63	Gemini 3.1 Pro	4.9%	estimated ± 7.2 pp, low confidence
64	Qwen3.6-27B	4.5%	estimated ± 9.0 pp, low confidence
65	Inkling-Small	4.1%	estimated ± 9.0 pp, low confidence
66	MiniMax M2.7	4.0%	estimated ± 9.0 pp, low confidence
67	Ling 3.0 Flash	3.2%	estimated ± 9.0 pp, low confidence
68	Ling 3.0 Flash FP8	3.2%	estimated ± 9.0 pp, low confidence
69	MiMo-V2-Flash	2.9%	estimated ± 9.0 pp, low confidence
70	GPT-5.1	2.8%	estimated ± 9.0 pp, low confidence
71	Gemini 3.5 Flash-Lite	2.7%	estimated ± 9.0 pp, low confidence
72	Nemotron 3 Ultra	2.7%	estimated ± 9.0 pp, low confidence
73	Muse Glimmer 30B	2.6%	estimated ± 9.0 pp, low confidence
74	GPT-5.4	2.1%	estimated ± 7.2 pp, low confidence
75	Mistral Medium 3.5 128B	2.0%	estimated ± 9.0 pp, low confidence
76	Kimi K2.5	2.0%	estimated ± 9.0 pp, low confidence
77	Kimi K2.5 (Reasoning)	2.0%	estimated ± 9.0 pp, low confidence
78	Qwen3.5-122B-A10B	1.7%	estimated ± 9.0 pp, low confidence
79	GLM-4.7	1.6%	estimated ± 9.0 pp, low confidence
80	Gemma 4 31B	1.3%	estimated ± 9.0 pp, low confidence
81	Grok 4.3	1.1%	estimated ± 9.0 pp, low confidence
82	Qwen3.6-35B-A3B	1.0%	estimated ± 9.0 pp, low confidence
83	o1	0.8%	estimated ± 9.0 pp, low confidence
84	Step 3.7 Flash	0.7%	estimated ± 9.0 pp, low confidence
85	Gemma 4 26B A4B	0.7%	estimated ± 9.0 pp, low confidence
86	GPT-5 (high)	0.6%	estimated ± 9.0 pp, low confidence
87	Nemotron 3 Super 100B	0.6%	estimated ± 9.0 pp, low confidence
88	o1-preview	0.3%	estimated ± 9.0 pp, low confidence
89	Gemini 2.5 Pro	0.3%	estimated ± 9.0 pp, low confidence
90	K-Exaone	0.2%	estimated ± 9.0 pp, low confidence
91	Gemma 4 12B	0.2%	estimated ± 9.0 pp, low confidence
92	GPT-OSS 120B	0.2%	estimated ± 9.0 pp, low confidence
93	Command A+	0.1%	estimated ± 9.0 pp, low confidence
94	Nemotron 3.5 Lightning 30B A3B NVFP4	0.1%	estimated ± 9.0 pp, low confidence
95	Mistral Small 4	0.1%	estimated ± 9.0 pp, low confidence
96	Mistral Small 4 (Reasoning)	0.1%	estimated ± 9.0 pp, low confidence
97	Trinity-Large-Preview	0.1%	estimated ± 9.0 pp, low confidence
98	Trinity-Large-Thinking	0.1%	estimated ± 9.0 pp, low confidence
99	Ling 2.6 Flash	0.1%	estimated ± 9.0 pp, low confidence
100	Gemini 1.5 Pro	0.0%	estimated ± 9.0 pp, low confidence
101	DeepSeek V3	0.0%	estimated ± 9.0 pp, low confidence
102	Granite 4.2 8B	0.0%	estimated ± 9.0 pp, low confidence
103	GPT-4 Turbo	0.0%	estimated ± 9.0 pp, low confidence
104	GPT-OSS 20B	0.0%	estimated ± 9.0 pp, low confidence
105	GPT-4.1 mini	0.0%	estimated ± 9.0 pp, low confidence
106	Mistral Large 3	0.0%	estimated ± 9.0 pp, low confidence
107	Claude 3 Opus	0.0%	estimated ± 9.0 pp, low confidence
108	Llama 4 Maverick	0.0%	estimated ± 9.0 pp, low confidence
109	Celeris-1	0.0%	estimated ± 9.0 pp, low confidence
110	Nemotron 3 Nano 30B	0.0%	estimated ± 9.0 pp, low confidence
111	Nemotron 3 Nano Omni 30B A3B	0.0%	estimated ± 9.0 pp, low confidence
112	Ultravox v0.6 Llama 3.3 70B	0.0%	estimated ± 9.0 pp, low confidence
113	GPT-4.1 nano	0.0%	estimated ± 9.0 pp, low confidence
114	GPT-4o mini	0.0%	estimated ± 9.0 pp, low confidence
115	Gemma 3 27B	0.0%	estimated ± 9.0 pp, low confidence
116	Gemma 4 E4B	0.0%	estimated ± 9.0 pp, low confidence
117	Llama 4 Scout	0.0%	estimated ± 9.0 pp, low confidence
118	Gemma 4 E2B	0.0%	estimated ± 9.0 pp, low confidence
119	LFM2.5-2.6B	0.0%	estimated ± 9.0 pp, low confidence

Bug Hunt Bench leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General