benchgap
NVIDIA · model

Nemotron 3.5 Lightning 30B A3B NVFP4 benchmark scores

As of 2026-11-22, Nemotron 3.5 Lightning 30B A3B NVFP4 (NVIDIA) has measured scores on 22 benchmarks and estimated scores on 77 more.

BenchmarkScoreSource
AA-GPQA Diamond74.3%measured
AA-HLE10.6%measured
AA-SciCode32.1%measured
Artificial Analysis Intelligence Index5.5%measured
CritPt0.0%measured
GDPval-AA6.9%measured
GPQA Diamond75.6%measured
HLE15.5%estimated ± 6.5 pp, medium confidence
IFBench72.9%measured
AA-LCR49.2%measured
LiveCodeBench v670.6%estimated ± 4.4 pp, high confidence
AA-Omniscience Accuracy14.7%measured
SciCode31.4%measured
Terminal-Bench 2.123.5%measured
BrowseComp36.8%measured
HLE w/ tools37.1%estimated ± 5.3 pp, low confidence
IFEval93.3%estimated ± 3.6 pp, high confidence
LongBench v252.1%estimated ± 5.5 pp, medium confidence
VITA-Bench6.8%estimated ± 6.7 pp, low confidence
τ²-bench results44.1%estimated ± 11.8 pp, low confidence
AA Coding Index26.8%measured
APEX-Agents4.5%estimated ± 9.4 pp, low confidence
APEX-Agents-AA7.0%estimated ± 4.4 pp, medium confidence
BioMysteryBench (human-difficult)0.0%estimated ± 6.8 pp, low confidence
DeepSearchQA61.7%estimated ± 4.5 pp, medium confidence
SWE-bench Verified52.8%measured
AutomationBench30.1%estimated ± 9.1 pp, low confidence
BFCL v465.1%estimated ± 2.7 pp, high confidence
CyberGym50.3%estimated ± 5.3 pp, low confidence
JobBench0.0%estimated ± 8.7 pp, low confidence
SWE-bench Pro28.4%estimated ± 4.5 pp, high confidence
τ³-bench results9.5%measured
WideResearch49.4%estimated ± 3.5 pp, medium confidence
AA Agentic Index6.1%measured
MMLU-Pro81.6%measured
AA-IFBench58.1%estimated ± 7.4 pp, medium confidence
BBH96.9%estimated ± 2.8 pp, medium confidence
GPQA75.6%measured
MMLU80.3%estimated ± 0.9 pp, high confidence
Vals GPQA Diamond75.7%estimated ± 2.1 pp, medium confidence
Vals LiveCodeBench81.8%estimated ± 0.8 pp, low confidence
Vals MMLU-Pro78.2%estimated ± 1.6 pp, medium confidence
MMMLU83.3%estimated ± 0.7 pp, medium confidence
AA MMLU-Pro83.3%estimated ± 0.8 pp, medium confidence
Gert Labs27.3%estimated ± 10.0 pp, low confidence
AA Harvey LAB v1.079.4%estimated ± 1.2 pp, low confidence
AA Tau3 Banking19.5%estimated ± 3.7 pp, medium confidence
ARC-AGI-114.0%estimated ± 3.9 pp, medium confidence
ARC-AGI-20.0%estimated ± 10.9 pp, low confidence
FrontierSWE v21.5%estimated ± 14.2 pp, low confidence
MazeBench0.4%estimated ± 5.5 pp, low confidence
OSWorld-Verified49.6%estimated ± 5.7 pp, low confidence
Vals SWE-bench38.4%estimated ± 5.3 pp, low confidence
Terminal-Bench 2.1 (Vals AI)24.8%estimated ± 7.3 pp, low confidence
CWE-Bench-AA27.0%estimated ± 6.4 pp, low confidence
AA Cyber Index4.5%estimated ± 9.2 pp, low confidence
DeepsecBench-AA0.0%estimated ± 8.9 pp, low confidence
HealthBench Professional (raw, Artificial Analysis)4.3%estimated ± 2.5 pp, low confidence
AA ITBench34.8%estimated ± 4.0 pp, medium confidence
Bug Hunt Bench10.2%estimated ± 10.5 pp, low confidence
CWE-bench v111.5%estimated ± 6.4 pp, low confidence
DeepSWE12.0%estimated ± 7.5 pp, low confidence
DRACO42.1%estimated ± 2.0 pp, low confidence
HLE w/o tools10.5%measured
SWE Multilingual36.5%measured
Toolathlon-Verified20.8%estimated ± 2.3 pp, low confidence
VulcanBench Frontier v460.1%estimated ± 6.8 pp, low confidence
GDP.pdf17.0%estimated ± 4.6 pp, medium confidence
ARC-AGI-30.0%estimated ± 7.2 pp, low confidence
ARC-AGI-3 Provider Adapter2.7%estimated ± 7.3 pp, low confidence
Vibe Code Bench0.0%estimated ± 8.4 pp, low confidence
Hermes Bench26.9%estimated ± 2.9 pp, medium confidence
Hermes Index6.9%estimated ± 8.7 pp, low confidence
ExploitGym0.0%estimated ± 10.6 pp, low confidence
C-Eval87.2%estimated ± 1.0 pp, low confidence
Claw-Eval62.7%estimated ± 6.4 pp, low confidence
MCP Atlas64.6%estimated ± 7.8 pp, low confidence
MMLU-Redux91.3%estimated ± 0.6 pp, high confidence
NL2Repo17.0%estimated ± 4.7 pp, medium confidence
QwenClawBench55.9%estimated ± 6.0 pp, low confidence
SuperGPQA55.6%estimated ± 8.5 pp, low confidence
Toolathlon16.7%estimated ± 4.2 pp, medium confidence
AA LiveCodeBench84.0%estimated ± 4.7 pp, low confidence
HealthBench Hard0.0%estimated ± 8.6 pp, low confidence
MMLU-Pro (Arcee)81.4%estimated ± 3.4 pp, medium confidence
React Native Evals67.4%estimated ± 2.7 pp, low confidence
ResearchClawBench8.9%estimated ± 1.9 pp, medium confidence
BioMysteryBench (human-solvable)81.7%estimated ± 0.6 pp, low confidence
HealthBench (length-adjusted)54.1%estimated ± 3.7 pp, medium confidence
HealthBench Professional56.7%estimated ± 4.2 pp, low confidence
ScarfBench24.4%estimated ± 9.1 pp, low confidence
STEMBench2.3%estimated ± 0.2 pp, low confidence
SimpleQA10.8%estimated ± 4.3 pp, low confidence
LiveCodeBench60.5%estimated ± 14.6 pp, low confidence
OpenHarmony Bench v1.051.0%estimated ± 2.4 pp, medium confidence
MRCRv222.9%estimated ± 11.3 pp, low confidence
MMLongBench-Doc59.3%estimated ± 2.6 pp, low confidence
BioBasicsBench7.2%estimated ± 10.2 pp, low confidence
PinchBench83.4%measured