benchgap
Calibration

Decision Index Retrieval → Decision Index Tools

Decision Index Tools is estimated from Decision Index Retrieval with a linear curve fitted on 110 models measured on both: y = 1.1307·x + 0.0426, R² = 0.84, cross-validated error 9.1 pp. It is used for 0 estimates.

Fitted on

ModelDecision Index RetrievalDecision Index Tools
Bosun v3.1 0.6B28.1%26.5%
Clef59.8%75.5%
Clef-Flash40.8%66.0%
Clef-Omni56.5%60.0%
decider-2b36.7%35.7%
decider-35b-a3b52.1%48.3%
Bekko System One v0 17M14.5%6.8%
Bekko System One v0 400M37.7%18.2%
Bekko System One v0 68M28.5%11.3%
Bespoke Nimble 9B v236.0%53.0%
Bespoke Nimble 9B v352.9%73.8%
Blink v0.3 26B-A4B (NVFP4)63.1%69.4%
Bosun v3.1 1.7B37.9%31.9%
CLM-v0.1-8B4.6%12.3%
Decider 4B42.2%47.7%
Decider chat · Qwen3.6-27B53.8%69.8%
decisio 0.4.0 (Gemma 4 12B)56.5%70.7%
decisio 0.4.0 (Qwen3.6-35B-A3B)54.0%66.9%
deck31b (Gemma 4 31B)64.8%75.6%
DM-JEPA 1.14.3%5.9%
EXAONE-4.0-1.2B-JEV v0.331.2%44.3%
ezjev 4B s252.0%62.6%
GEV-26B-Decide63.3%66.3%
Gevva E2B38.5%45.0%
Gevva E4B32.9%48.4%
GLiNER2.5-Decide18.3%12.3%
Hopper 12B (trained)52.9%66.5%
Hopper (G) 1.246.6%62.2%
Intelif Qwen3-4B36.6%54.9%
Intern-Decision-0.8B8.5%29.2%
Intern-Decision-2B17.7%36.9%
Intern-Decision-4B39.5%55.8%
JADE56.0%73.6%
JEV-27B58.5%73.9%
JEV-9B48.4%56.8%
Jev-Style 0.8B v321.7%23.8%
Jevfire53.7%72.2%
JevK5 9B v0.3.352.0%60.8%
JevK5 Lite20.3%0.0%
JevK5 v0.345.6%53.0%
Jevstral 8B36.6%57.6%
Jobe37.5%48.2%
JoshuaSP diffusiongemma (open-jev)54.2%66.9%
JPT-0.8B14.6%34.0%
JPT-35B-A3B55.1%66.4%
JPT-4B40.4%56.1%
JPT-9B42.3%61.0%
Julia 13.5%6.1%
Kas 4B40.8%53.8%
Kev 0.8B r1516.7%30.6%
Kev 27B55.6%77.5%
Kev 4B r1041.7%50.1%
Kev 9B v247.4%58.7%
Kodiak v0.2 1B (accuracy)29.9%12.8%
Lavoir17.1%9.8%
lev42.3%58.0%
Lev1 3.7B44.0%63.7%
LFM2.5-2.6B-RLCD4.9%8.1%
LFM2.5-350M-RLCD0.0%3.5%
mini-jev10.0%29.6%
mmastrac diffusiongemma (vLLM PR 57250)23.0%36.6%
NeoHorse-Jev-4B44.7%55.7%
open-jev (pngwn)24.0%36.7%
openvons30.0%46.9%
Quyet-1.0-Large63.2%74.4%
Qwen-2.5-1B-RLCD1.0%7.7%
razorback16 openjev diffusiongemma (NVFP4, vLLM)37.6%53.9%
Ryotide Qwen v0.3.145.8%51.7%
SemIf17.8%39.6%
Sieve 2B28.0%31.7%
Sieve 9B45.8%62.1%
Sifr 0.8B v3.126.0%31.3%
Solomon v1.126.9%45.6%
system-one-gemma8.8%2.5%
Tev1-0.8B-experimental3.4%30.6%
Tev1-4B-experimental18.5%44.2%
this-that 1.237.9%38.4%
Torchcast Decision 27B63.2%78.9%
Verdict0.0%5.6%
vLLM-SR Decision 2.0 Eos 0.8B33.1%29.2%
vLLM-SR Decision 2.0 Kai 0.6B26.8%24.4%
vLLM-SR Decision 2.0 Lux 9B54.3%65.0%
vLLM-SR Decision 2.0 Nox 4B52.3%58.4%
vLLM-SR Decision 2.0 Sol 2B40.4%46.6%
vLLM-SR Decision 2.0 Vega 27B61.1%77.5%
Von 1.320.9%7.6%
Winnow-E4B40.8%53.5%
Xor 1.257.9%71.6%
Zev0.0%2.5%
Decider chat · Gemma-4-31B63.4%70.5%
Surogate Rune 26B-A4B v362.3%69.0%
simple-jev · Qwen3.8-27B63.5%75.8%
djev44.8%52.9%
Drex Max67.6%80.4%
Drex v1.559.2%71.2%
Drex v2.058.5%72.5%
Eikos-27B59.3%78.5%
elephant-165.1%82.1%
GLiDE v1.273.4%86.0%
jeff17.2%0.0%
Jev 1.1464.8%74.4%
Jev-Omni39.3%63.4%
Laya9.2%7.9%
mammoth-166.2%83.4%
metask-jev-4b14.1%38.4%
Perplexity Decider Flash v1.2 4B62.2%75.7%
Perplexity Decider Flash v1.3 4B66.4%77.1%
Perplexity Decider v1.2 27B69.4%82.4%
Perplexity Decider v1.3 27B70.7%82.6%
Winnow-12B Q854.6%62.9%