General

HLE-MultiChoice (Decision Index) leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap
License: https://creativecommons.org/licenses/by-nc/4.0/

As of 2026-10-29, the highest measured score on HLE-MultiChoice (Decision Index) is 18.9% by Jev 1.14. 88 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	Jev 1.14	18.9%	measured
2	elephant-1	18.3%	measured
3	mammoth-1	17.3%	estimated ± 2.0 pp, low confidence
4	Drex v2.0	17.3%	estimated ± 2.0 pp, medium confidence
5	Blink v0.3 26B-A4B (NVFP4)	17.3%	estimated ± 2.0 pp, medium confidence
6	Torchcast Decision 27B	17.3%	estimated ± 2.0 pp, medium confidence
7	Surogate Rune 26B-A4B v3	17.3%	estimated ± 2.0 pp, medium confidence
8	Drex v1.5	17.3%	estimated ± 2.0 pp, medium confidence
9	GEV-26B-Decide	17.3%	estimated ± 2.0 pp, medium confidence
10	Xor 1.2	17.2%	estimated ± 2.0 pp, medium confidence
11	Bespoke Nimble 9B v3	17.1%	estimated ± 2.0 pp, medium confidence
12	Eikos-27B	17.0%	estimated ± 2.0 pp, medium confidence
13	JEV-27B	16.9%	estimated ± 2.0 pp, medium confidence
14	d1-ultra	15.0%	measured
15	Perplexity Decider v1.2 27B	14.6%	measured
16	vLLM-SR Decision 2.0 Vega 27B	14.3%	estimated ± 2.0 pp, medium confidence
17	Celeris-1 Decision	13.3%	measured
18	simple-jev · Qwen3.8-27B	13.2%	estimated ± 2.0 pp, medium confidence
19	JADE	12.8%	estimated ± 2.0 pp, medium confidence
20	Kev 27B	12.4%	estimated ± 2.0 pp, medium confidence
21	Quyet-1.0-Large	12.4%	estimated ± 2.0 pp, medium confidence
22	Perplexity Decider Flash v1.2 4B	12.1%	measured
23	d1	11.5%	measured
24	Clef	11.1%	measured
25	Microsoft-Decision-1	10.9%	measured
26	GLiDE	10.5%	measured
27	deck31b (Gemma 4 31B)	10.4%	estimated ± 2.0 pp, medium confidence
28	Decider chat · Gemma-4-31B	10.4%	estimated ± 2.0 pp, medium confidence
29	Clef-Flash	10.3%	measured
30	Mercury Decide	10.1%	measured
31	Bosun v3.1 0.6B	10.1%	estimated ± 3.0 pp, low confidence
32	Bekko System One v0 17M	10.1%	estimated ± 3.0 pp, low confidence
33	Bekko System One v0 68M	10.1%	estimated ± 3.0 pp, low confidence
34	Bosun v3.1 1.7B	10.1%	estimated ± 3.0 pp, low confidence
35	EXAONE-4.0-1.2B-JEV v0.3	10.1%	estimated ± 3.0 pp, low confidence
36	Gevva E2B	10.1%	estimated ± 3.0 pp, low confidence
37	GLiNER2.5-Decide	10.1%	estimated ± 3.0 pp, low confidence
38	Jev-Style 0.8B v3	10.1%	estimated ± 3.0 pp, low confidence
39	JevK5 Lite	10.1%	estimated ± 3.0 pp, low confidence
40	JPT-0.8B	10.1%	estimated ± 3.0 pp, low confidence
41	Kev 0.8B r15	10.1%	estimated ± 3.0 pp, low confidence
42	Kodiak v0.2 1B (accuracy)	10.1%	estimated ± 3.0 pp, low confidence
43	Lavoir	10.1%	estimated ± 3.0 pp, low confidence
44	Sifr 0.8B v3.1	10.1%	estimated ± 3.0 pp, low confidence
45	vLLM-SR Decision 2.0 Eos 0.8B	10.1%	estimated ± 3.0 pp, low confidence
46	vLLM-SR Decision 2.0 Kai 0.6B	10.1%	estimated ± 3.0 pp, low confidence
47	vLLM-SR Decision 2.0 Sol 2B	10.1%	estimated ± 3.0 pp, low confidence
48	Von 1.3	10.1%	estimated ± 3.0 pp, low confidence
49	jeff	10.1%	estimated ± 3.0 pp, low confidence
50	decider-35b-a3b	9.7%	estimated ± 2.0 pp, medium confidence
51	Decider chat · Qwen3.6-27B	9.7%	estimated ± 2.0 pp, medium confidence
52	JPT-35B-A3B	9.6%	estimated ± 2.0 pp, medium confidence
53	JoshuaSP diffusiongemma (open-jev)	9.6%	estimated ± 2.0 pp, medium confidence
54	decisio 0.4.0 (Qwen3.6-35B-A3B)	9.6%	estimated ± 2.0 pp, medium confidence
55	lev	9.6%	estimated ± 2.0 pp, medium confidence
56	decisio 0.4.0 (Gemma 4 12B)	9.6%	estimated ± 2.0 pp, medium confidence
57	Lev1 3.7B	9.6%	estimated ± 2.0 pp, medium confidence
58	ezjev 4B s2	9.6%	estimated ± 2.0 pp, medium confidence
59	Jevfire	9.6%	estimated ± 2.0 pp, low confidence
60	vLLM-SR Decision 2.0 Lux 9B	9.6%	estimated ± 2.0 pp, low confidence
61	djev	9.6%	estimated ± 2.0 pp, low confidence
62	Hopper 12B (trained)	9.6%	estimated ± 2.0 pp, low confidence
63	razorback16 openjev diffusiongemma (NVFP4, vLLM)	9.6%	estimated ± 2.0 pp, low confidence
64	Ryotide Qwen v0.3.1	9.6%	estimated ± 2.0 pp, low confidence
65	JevK5 9B v0.3.3	9.6%	estimated ± 2.0 pp, low confidence
66	Jobe	9.6%	estimated ± 2.0 pp, low confidence
67	Winnow-12B Q8	9.6%	estimated ± 2.0 pp, low confidence
68	JEV-9B	9.6%	estimated ± 2.0 pp, low confidence
69	Decider 4B	9.6%	estimated ± 2.0 pp, low confidence
70	Bespoke Nimble 9B v2	9.6%	estimated ± 2.0 pp, low confidence
71	Hopper (G) 1.2	9.6%	estimated ± 2.0 pp, low confidence
72	JevK5 v0.3	9.6%	estimated ± 2.0 pp, low confidence
73	JPT-9B	9.6%	estimated ± 2.0 pp, low confidence
74	mmastrac diffusiongemma (vLLM PR 57250)	9.6%	estimated ± 2.0 pp, low confidence
75	SemIf	9.6%	estimated ± 2.0 pp, low confidence
76	Intern-Decision-4B	9.6%	estimated ± 2.0 pp, low confidence
77	openvons	9.6%	estimated ± 2.0 pp, low confidence
78	mini-jev	9.6%	estimated ± 2.0 pp, low confidence
79	vLLM-SR Decision 2.0 Nox 4B	9.6%	estimated ± 2.0 pp, low confidence
80	Jev-Omni	9.6%	estimated ± 2.0 pp, low confidence
81	Kas 4B	9.6%	estimated ± 2.0 pp, low confidence
82	Solomon v1.1	9.6%	estimated ± 2.0 pp, low confidence
83	JPT-4B	9.6%	estimated ± 2.0 pp, low confidence
84	Sieve 9B	9.6%	estimated ± 2.0 pp, low confidence
85	Gevva E4B	9.6%	estimated ± 2.0 pp, low confidence
86	Winnow-E4B	9.6%	estimated ± 2.0 pp, low confidence
87	Intelif Qwen3-4B	9.6%	estimated ± 2.0 pp, low confidence
88	Tev1-4B-experimental	9.6%	estimated ± 2.0 pp, low confidence
89	decider-2b	9.6%	estimated ± 2.0 pp, low confidence
90	Bekko System One v0 400M	9.6%	estimated ± 2.0 pp, low confidence
91	Intern-Decision-2B	9.6%	estimated ± 2.0 pp, low confidence
92	Kev 4B r10	9.6%	estimated ± 2.0 pp, low confidence
93	Kev 9B v2	9.6%	estimated ± 2.0 pp, low confidence
94	open-jev (pngwn)	9.6%	estimated ± 2.0 pp, low confidence
95	RSI-Jev v5.0 VL 3B	9.6%	estimated ± 2.0 pp, low confidence
96	Sieve 2B	9.6%	estimated ± 2.0 pp, low confidence
97	this-that 1.2	9.6%	estimated ± 2.0 pp, low confidence
98	metask-jev-4b	9.6%	estimated ± 2.0 pp, low confidence
99	NeoHorse-Jev-4B	9.5%	estimated ± 2.2 pp, low confidence
100	Jevstral 8B	9.4%	estimated ± 2.2 pp, low confidence
101	Clef-Omni	9.0%	measured

HLE-MultiChoice (Decision Index) leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General