benchgap
Calibration

Decision Index Arts → Artificial Analysis Decision Index v1.0

Artificial Analysis Decision Index v1.0 is estimated from Decision Index Arts with a Michaelis–Menten curve fitted on 11 models measured on both: y = 2.0000·x / (1.31403 + x), R² = 0.84, cross-validated error 3.2 pp. It is used for 72 estimates.

Estimated modelDecision Index ArtsArtificial Analysis Decision Index v1.0Source
mammoth-162.7%64.6%estimated ± 3.2 pp, medium confidence
Perplexity Decider v1.3 27B55.7%59.5%estimated ± 3.2 pp, high confidence
Blink v0.3 26B-A4B (NVFP4)52.3%56.9%estimated ± 3.2 pp, high confidence
Torchcast Decision 27B52.2%56.9%estimated ± 3.2 pp, high confidence
Surogate Rune 26B-A4B v352.0%56.7%estimated ± 3.2 pp, high confidence
GEV-26B-Decide48.6%54.0%estimated ± 3.2 pp, high confidence
Xor 1.248.1%53.6%estimated ± 3.2 pp, high confidence
Bespoke Nimble 9B v347.2%52.9%estimated ± 3.2 pp, high confidence
Eikos-27B47.1%52.8%estimated ± 3.2 pp, high confidence
JEV-27B46.9%52.6%estimated ± 3.2 pp, high confidence
Perplexity Decider Flash v1.3 4B46.9%52.6%estimated ± 3.2 pp, high confidence
vLLM-SR Decision 2.0 Vega 27B45.6%51.5%estimated ± 3.2 pp, high confidence
simple-jev · Qwen3.8-27B45.3%51.3%estimated ± 3.2 pp, high confidence
JADE45.2%51.2%estimated ± 3.2 pp, high confidence
Kev 27B45.1%51.1%estimated ± 3.2 pp, high confidence
Quyet-1.0-Large45.1%51.1%estimated ± 3.2 pp, high confidence
deck31b (Gemma 4 31B)44.3%50.4%estimated ± 3.2 pp, high confidence
Decider chat · Gemma-4-31B44.3%50.4%estimated ± 3.2 pp, high confidence
decider-35b-a3b42.9%49.2%estimated ± 3.2 pp, high confidence
Decider chat · Qwen3.6-27B42.9%49.2%estimated ± 3.2 pp, high confidence
JPT-35B-A3B41.4%47.9%estimated ± 3.2 pp, high confidence
JoshuaSP diffusiongemma (open-jev)40.9%47.5%estimated ± 3.2 pp, high confidence
decisio 0.4.0 (Qwen3.6-35B-A3B)40.2%46.9%estimated ± 3.2 pp, high confidence
lev39.4%46.1%estimated ± 3.2 pp, high confidence
decisio 0.4.0 (Gemma 4 12B)38.7%45.5%estimated ± 3.2 pp, high confidence
Lev1 3.7B38.7%45.5%estimated ± 3.2 pp, high confidence
ezjev 4B s237.8%44.7%estimated ± 3.2 pp, high confidence
Jevfire37.6%44.5%estimated ± 3.2 pp, medium confidence
vLLM-SR Decision 2.0 Lux 9B37.6%44.5%estimated ± 3.2 pp, medium confidence
djev37.4%44.3%estimated ± 3.2 pp, medium confidence
Hopper 12B (trained)37.3%44.2%estimated ± 3.2 pp, medium confidence
razorback16 openjev diffusiongemma (NVFP4, vLLM)35.9%42.9%estimated ± 3.2 pp, medium confidence
Ryotide Qwen v0.3.135.7%42.7%estimated ± 3.2 pp, medium confidence
JevK5 9B v0.3.335.1%42.2%estimated ± 3.2 pp, medium confidence
Jobe35.1%42.2%estimated ± 3.2 pp, medium confidence
Winnow-12B Q835.1%42.2%estimated ± 3.2 pp, medium confidence
JEV-9B34.6%41.7%estimated ± 3.2 pp, medium confidence
Decider 4B34.5%41.6%estimated ± 3.2 pp, medium confidence
Bespoke Nimble 9B v234.2%41.3%estimated ± 3.2 pp, medium confidence
Hopper (G) 1.234.0%41.1%estimated ± 3.2 pp, medium confidence
JevK5 v0.333.5%40.6%estimated ± 3.2 pp, medium confidence
JPT-9B33.2%40.3%estimated ± 3.2 pp, medium confidence
mmastrac diffusiongemma (vLLM PR 57250)33.2%40.3%estimated ± 3.2 pp, medium confidence
SemIf33.0%40.1%estimated ± 3.2 pp, medium confidence
Intern-Decision-4B32.9%40.0%estimated ± 3.2 pp, medium confidence
openvons32.7%39.9%estimated ± 3.2 pp, medium confidence
mini-jev31.9%39.1%estimated ± 3.2 pp, medium confidence
vLLM-SR Decision 2.0 Nox 4B31.9%39.1%estimated ± 3.2 pp, medium confidence
Jev-Omni31.9%39.1%estimated ± 3.2 pp, medium confidence
Kas 4B31.6%38.8%estimated ± 3.2 pp, medium confidence
Solomon v1.130.7%37.9%estimated ± 3.2 pp, medium confidence
JPT-4B30.6%37.8%estimated ± 3.2 pp, medium confidence
Sieve 9B29.7%36.9%estimated ± 3.2 pp, medium confidence
Gevva E4B28.3%35.4%estimated ± 3.2 pp, medium confidence
Winnow-E4B28.2%35.3%estimated ± 3.2 pp, medium confidence
Tev1-4B-experimental27.3%34.4%estimated ± 3.2 pp, medium confidence
Intelif Qwen3-4B27.2%34.3%estimated ± 3.2 pp, medium confidence
Bekko System One v0 400M24.6%31.5%estimated ± 3.2 pp, medium confidence
decider-2b24.4%31.3%estimated ± 3.2 pp, medium confidence
Sieve 2B23.6%30.5%estimated ± 3.2 pp, medium confidence
Kev 4B r1023.0%29.8%estimated ± 3.2 pp, medium confidence
metask-jev-4b22.2%28.9%estimated ± 3.2 pp, medium confidence
Intern-Decision-2B21.4%28.0%estimated ± 3.2 pp, medium confidence
Kev 9B v221.3%27.9%estimated ± 3.2 pp, medium confidence
open-jev (pngwn)20.3%26.8%estimated ± 3.2 pp, medium confidence
this-that 1.219.1%25.4%estimated ± 3.2 pp, medium confidence
Jevstral 8B17.6%23.6%estimated ± 3.2 pp, medium confidence
Kodiak v0.2 1B (accuracy)17.3%23.3%estimated ± 3.2 pp, medium confidence
Bekko System One v0 68M16.9%22.8%estimated ± 3.2 pp, medium confidence
vLLM-SR Decision 2.0 Sol 2B16.3%22.1%estimated ± 3.2 pp, medium confidence
EXAONE-4.0-1.2B-JEV v0.316.2%22.0%estimated ± 3.2 pp, medium confidence
NeoHorse-Jev-4B15.5%21.1%estimated ± 3.2 pp, medium confidence

Fitted on

ModelDecision Index ArtsArtificial Analysis Decision Index v1.0
Clef44.7%48.6%
Clef-Flash39.3%45.8%
Clef-Omni37.8%44.3%
Drex Max56.0%63.7%
Drex v1.550.1%50.7%
Drex v2.055.2%57.9%
elephant-156.7%64.7%
GLiDE v1.262.3%65.0%
Jev 1.1447.7%57.1%
Perplexity Decider Flash v1.2 4B45.0%50.2%
Perplexity Decider v1.2 27B56.4%56.2%