benchgap
Calibration

Decision Index Retrieval → Decision Index Language

Decision Index Language is estimated from Decision Index Retrieval with a linear curve fitted on 110 models measured on both: y = 0.8571·x + 0.0242, R² = 0.78, cross-validated error 9.5 pp. It is used for 0 estimates.

Fitted on

ModelDecision Index RetrievalDecision Index Language
Bosun v3.1 0.6B28.1%5.1%
Clef59.8%47.8%
Clef-Flash40.8%40.5%
Clef-Omni56.5%40.7%
decider-2b36.7%27.4%
decider-35b-a3b52.1%48.4%
Bekko System One v0 17M14.5%0.6%
Bekko System One v0 400M37.7%19.6%
Bekko System One v0 68M28.5%6.5%
Bespoke Nimble 9B v236.0%44.4%
Bespoke Nimble 9B v352.9%54.6%
Blink v0.3 26B-A4B (NVFP4)63.1%59.3%
Bosun v3.1 1.7B37.9%14.7%
CLM-v0.1-8B4.6%5.7%
Decider 4B42.2%33.9%
Decider chat · Qwen3.6-27B53.8%52.6%
decisio 0.4.0 (Gemma 4 12B)56.5%44.0%
decisio 0.4.0 (Qwen3.6-35B-A3B)54.0%50.2%
deck31b (Gemma 4 31B)64.8%57.4%
DM-JEPA 1.14.3%2.6%
EXAONE-4.0-1.2B-JEV v0.331.2%22.2%
ezjev 4B s252.0%47.1%
GEV-26B-Decide63.3%55.9%
Gevva E2B38.5%21.2%
Gevva E4B32.9%28.3%
GLiNER2.5-Decide18.3%15.5%
Hopper 12B (trained)52.9%47.1%
Hopper (G) 1.246.6%39.8%
Intelif Qwen3-4B36.6%36.6%
Intern-Decision-0.8B8.5%8.4%
Intern-Decision-2B17.7%28.8%
Intern-Decision-4B39.5%41.6%
JADE56.0%56.5%
JEV-27B58.5%51.2%
JEV-9B48.4%40.7%
Jev-Style 0.8B v321.7%12.0%
Jevfire53.7%44.7%
JevK5 9B v0.3.352.0%53.7%
JevK5 Lite20.3%13.8%
JevK5 v0.345.6%46.3%
Jevstral 8B36.6%37.3%
Jobe37.5%36.0%
JoshuaSP diffusiongemma (open-jev)54.2%51.9%
JPT-0.8B14.6%19.3%
JPT-35B-A3B55.1%57.2%
JPT-4B40.4%41.3%
JPT-9B42.3%46.5%
Julia 13.5%1.5%
Kas 4B40.8%37.6%
Kev 0.8B r1516.7%12.4%
Kev 27B55.6%57.5%
Kev 4B r1041.7%35.6%
Kev 9B v247.4%44.3%
Kodiak v0.2 1B (accuracy)29.9%10.6%
Lavoir17.1%9.2%
lev42.3%29.5%
Lev1 3.7B44.0%41.8%
LFM2.5-2.6B-RLCD4.9%5.4%
LFM2.5-350M-RLCD0.0%2.0%
mini-jev10.0%29.1%
mmastrac diffusiongemma (vLLM PR 57250)23.0%47.7%
NeoHorse-Jev-4B44.7%34.2%
open-jev (pngwn)24.0%36.9%
openvons30.0%31.7%
Quyet-1.0-Large63.2%58.1%
Qwen-2.5-1B-RLCD1.0%7.7%
razorback16 openjev diffusiongemma (NVFP4, vLLM)37.6%46.3%
Ryotide Qwen v0.3.145.8%33.2%
SemIf17.8%35.9%
Sieve 2B28.0%21.7%
Sieve 9B45.8%45.3%
Sifr 0.8B v3.126.0%14.3%
Solomon v1.126.9%54.2%
system-one-gemma8.8%1.7%
Tev1-0.8B-experimental3.4%14.4%
Tev1-4B-experimental18.5%41.4%
this-that 1.237.9%23.4%
Torchcast Decision 27B63.2%57.3%
Verdict0.0%0.0%
vLLM-SR Decision 2.0 Eos 0.8B33.1%19.7%
vLLM-SR Decision 2.0 Kai 0.6B26.8%13.1%
vLLM-SR Decision 2.0 Lux 9B54.3%46.1%
vLLM-SR Decision 2.0 Nox 4B52.3%42.2%
vLLM-SR Decision 2.0 Sol 2B40.4%24.0%
vLLM-SR Decision 2.0 Vega 27B61.1%53.9%
Von 1.320.9%14.7%
Winnow-E4B40.8%37.5%
Xor 1.257.9%53.9%
Zev0.0%1.8%
Decider chat · Gemma-4-31B63.4%55.5%
Surogate Rune 26B-A4B v362.3%58.6%
simple-jev · Qwen3.8-27B63.5%56.1%
djev44.8%48.6%
Drex Max67.6%62.1%
Drex v1.559.2%40.8%
Drex v2.058.5%52.7%
Eikos-27B59.3%55.1%
elephant-165.1%62.1%
GLiDE v1.273.4%66.8%
jeff17.2%7.8%
Jev 1.1464.8%58.0%
Jev-Omni39.3%48.1%
Laya9.2%14.6%
mammoth-166.2%67.1%
metask-jev-4b14.1%37.0%
Perplexity Decider Flash v1.2 4B62.2%62.2%
Perplexity Decider Flash v1.3 4B66.4%62.1%
Perplexity Decider v1.2 27B69.4%65.5%
Perplexity Decider v1.3 27B70.7%67.2%
Winnow-12B Q854.6%50.6%