Knowledge & reasoning

JevBench 1.4 leaderboard

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

As of 2026-10-07, the highest measured score on JevBench 1.4 is 70.8% by classifier.dev (fast). 18 more models have estimated scores, calibrated from the benchmarks they were measured on.

Measured scores: benchlm.ai.

#	Model	Score	Source
1	classifier.dev (fast)	70.8%	measured
2	Imajev-4B	67.4%	measured
3	Plumb-4B	65.8%	measured
4	decider-4b v2	64.1%	measured
5	Jev 1.13.0	63.3%	measured
6	JevK5 v0.2.0	62.0%	measured
7	Cygnet	61.8%	measured
8	Hopper	59.4%	measured
9	JevK5 v0.3 4B	56.3%	estimated ± 10.7 pp, low confidence
10	Winnow-12B Q8	55.6%	measured
11	Decision 4B v1.2	55.4%	estimated ± 10.7 pp, low confidence
12	Imajev-4B (RTX 5090)	55.0%	estimated ± 10.7 pp, low confidence
13	Decision 4B v1.1	55.0%	estimated ± 10.7 pp, low confidence
14	reflex 4B	54.0%	measured
15	Manchego v2.1	53.6%	estimated ± 10.7 pp, low confidence
16	djev	52.2%	measured
17	Surogate Rune 26B-A4B v3 (RTX PRO 6000)	51.7%	estimated ± 10.7 pp, low confidence
18	Jev-Omni	51.3%	measured
19	metask-jev-4b	47.8%	measured
20	SemIf Qwen3.5-4B	47.7%	measured
21	Jobe Qwen3.5-4B	46.9%	measured
22	local-jev Qwen3.5-4B	46.8%	measured
23	system-one-open	45.1%	measured
24	spark-s1-4b-v6	44.6%	measured
25	Malkuth-4B	44.5%	measured
26	jqv Qwen3-32B	44.4%	measured
27	Qwen3-Reranker-4B	43.5%	measured
28	decider-35b-a3b	41.2%	measured
29	Raw Qwen3 4B Instruct 2507 direct logits	41.0%	measured
30	OpenSourceJev Qwen3.5-4B Q4_K_M	40.9%	measured
31	ZeroEntropy zerank-2	40.2%	measured
32	decision-machine-1	39.9%	measured
33	Malkuth-2B	38.9%	measured
34	Raw Phi-4 mini direct logits	38.0%	measured
35	JEV Qwen3.5-9B Base NVFP4	37.7%	measured
36	Instinct Dual 4B	36.9%	estimated ± 10.7 pp, low confidence
37	OpenJev DiffusionGemma 26B-A4B NVFP4	36.9%	measured
38	kev 4B	36.1%	measured
39	Decision 2B v59	35.8%	measured
40	Qwen3.5-9B Jev-like data-mix v2	35.2%	measured
41	GPT-6 Luna (low)	35.1%	measured
42	SimpleJev Qwen3.8-27B	34.6%	measured
43	NInfer Qwen3.8-Flash-Next mixed	34.0%	measured
44	swanOne	33.6%	measured
45	GPT-6 Luna (medium)	33.3%	measured
46	open-alternative-jev Qwen3.5-4B	33.2%	measured
47	jev-local Qwen3.5-9B	32.5%	measured
48	Decision Fast v53a	32.5%	measured
49	decider-2b	30.7%	measured
50	jeff	30.6%	measured
51	Laya	30.3%	measured
52	Autoloops Gemma 4 31B IT	30.0%	measured
53	Standard One 8B	29.1%	measured
54	lev-350m	28.5%	measured
55	openjev-sglang Qwen3.6-35B-A3B	27.7%	measured
56	Von 395M	27.5%	measured
57	NInfer Qwen3.8-27B NVFP4 (T=1.5)	26.9%	measured
58	NInfer Qwen3.8-27B NVFP4	26.3%	measured
59	kev 8B	25.6%	measured
60	JevOne	25.5%	measured
61	typecastlm	25.3%	measured
62	Nemotron Diffusion 8B (optimized vLLM)	25.1%	estimated ± 10.7 pp, low confidence
63	SimpleJev Qwen3.6-35B-A3B	24.9%	measured
64	kev 0.6B	24.8%	measured
65	Raw Qwen3 8B direct logits	23.7%	measured
66	system-one Qwen3-8B	23.4%	measured
67	AutoJev-27B	22.6%	estimated ± 10.7 pp, low confidence
68	AutoJev-27B (RTX PRO 6000)	22.6%	estimated ± 10.7 pp, low confidence
69	Eikos-27B	22.2%	estimated ± 10.7 pp, low confidence
70	OpenDecision ModernBERT-large	21.6%	measured
71	Bev / Bonsai 27B	21.2%	estimated ± 10.7 pp, low confidence
72	LitJev Qwen3.8-27B	19.5%	measured
73	openJev Verdict 1.4	19.0%	measured
74	kev 0.5B	18.9%	measured
75	Bespoke Nimble 9B	18.7%	measured
76	GPT-5.6 Luna (low)	18.5%	measured
77	openJev Verdict	18.1%	measured
78	Raw Qwen3 1.7B direct logits	18.1%	measured
79	reflex-27b	17.8%	measured
80	Bosun v3.1 0.6B	17.3%	estimated ± 10.7 pp, low confidence
81	Deem 0.8B v1	17.2%	estimated ± 10.7 pp, low confidence
82	JevAct	16.9%	measured
83	Laya multilingual	16.7%	estimated ± 10.7 pp, low confidence
84	Laya typed-decisions	16.7%	estimated ± 10.7 pp, low confidence
85	Needle 3 (2-bit)	16.7%	estimated ± 10.7 pp, low confidence
86	Needle 3 (options as tools)	16.7%	estimated ± 10.7 pp, low confidence
87	djev (thinking)	15.2%	measured
88	GLiNER2 large	15.1%	measured
89	OpenJev (thinking, BF16)	14.8%	measured
90	Qwen3.5-0.8B Decision Model	14.5%	measured
91	Gemini 3.1 Flash-Lite	14.3%	measured
92	open-jev-deberta-v3-large	12.6%	measured
93	smalljev semantic-v9	12.3%	measured
94	GLiNER2.5 base	11.8%	measured
95	Instinct Qwen3.8-27B	11.4%	measured
96	Open-Jev 9B	11.2%	measured
97	Open-Jev 2B	10.0%	measured
98	GLiNER2.5 multi	9.8%	measured
99	CLM-8B	8.6%	measured
100	SimpleJev Qwen3.5-0.8B	7.5%	measured
101	GLiNER2.5 small	7.2%	measured
102	Raw Qwen3 0.6B direct logits	7.1%	measured
103	verdict-small	5.7%	measured
104	DeepSeek V4.1 Flash	4.8%	measured
105	Mirror	2.1%	measured
106	Mixedbread mxbai-rerank-base-v2	0.4%	measured
107	BAAI bge-reranker-v2-m3	0.2%	measured
108	Alibaba GTE Reranker ModernBERT-base	0.2%	measured
109	Certo v1	0.0%	measured
110	Open Jev JSON Canvas	0.0%	measured
111	Qwen3.8-27B (Chutes TEE)	0.0%	measured

JevBench 1.4 leaderboard

All leaderboards

Agentic · terminal

Agentic · tools

Coding

Math

Knowledge & reasoning

Instruction following

Multilingual

Vision & documents

Long context

General