benchgap
Other · model

Gemma 4 31B benchmark scores

As of 2026-11-21, Gemma 4 31B (Other) has measured scores on 28 benchmarks and estimated scores on 128 more.

BenchmarkScoreSource
AA-GPQA Diamond85.7%measured
AA-HLE23.6%measured
AA-SciCode45.5%measured
Artificial Analysis Intelligence Index4.4%measured
CritPt1.4%measured
GDPval-AA5.9%measured
GPQA Diamond84.4%estimated ± 1.2 pp, high confidence
HLE26.5%measured
IFBench73.8%estimated ± 6.6 pp, medium confidence
AA-LCR69.7%measured
LiveCodeBench v680.0%measured
AA-Omniscience Accuracy19.9%measured
SciCode45.5%estimated ± 3.1 pp, high confidence
BrowseComp67.4%estimated ± 5.7 pp, medium confidence
HLE w/ tools36.6%estimated ± 5.3 pp, low confidence
IFEval93.0%estimated ± 2.8 pp, high confidence
LongBench v258.4%estimated ± 5.5 pp, medium confidence
τ²-bench results59.9%measured
AA Coding Index43.4%measured
AA-MMMU-Pro73.4%measured
APEX-Agents3.8%estimated ± 9.4 pp, low confidence
APEX-Agents-AA9.7%estimated ± 9.5 pp, medium confidence
BioMysteryBench (human-difficult)20.9%estimated ± 6.8 pp, low confidence
DeepSearchQA63.9%estimated ± 4.5 pp, medium confidence
SWE-bench Verified76.1%estimated ± 5.0 pp, medium confidence
BFCL v464.1%estimated ± 2.7 pp, high confidence
CyberGym50.7%estimated ± 5.3 pp, low confidence
JobBench0.0%estimated ± 8.7 pp, low confidence
SWE-bench Pro38.9%estimated ± 6.0 pp, medium confidence
WideResearch44.2%estimated ± 5.9 pp, low confidence
AA Agentic Index6.7%measured
MMLU-Pro85.2%measured
AA-IFBench75.6%measured
BBH98.9%estimated ± 2.8 pp, medium confidence
FailureSensorIQ66.6%estimated ± 1.7 pp, high confidence
GPQA84.3%measured
MMLU86.2%estimated ± 0.9 pp, high confidence
Vals GPQA Diamond77.1%estimated ± 2.9 pp, medium confidence
Vals LiveCodeBench0.1%estimated ± 3.4 pp, low confidence
Vals MMLU-Pro77.8%estimated ± 1.8 pp, medium confidence
MMMLU86.8%estimated ± 0.7 pp, medium confidence
MMMU82.9%estimated ± 0.8 pp, high confidence
Chess Puzzles30.8%estimated ± 10.8 pp, low confidence
AA MMLU-Pro84.1%estimated ± 0.8 pp, medium confidence
Gert Labs35.3%measured
AA Harvey LAB v1.079.4%estimated ± 1.2 pp, low confidence
AA Tau3 Banking19.9%estimated ± 3.7 pp, medium confidence
ARC-AGI-128.0%estimated ± 3.9 pp, medium confidence
ARC-AGI-216.9%estimated ± 10.9 pp, low confidence
CursorBench 3.132.9%estimated ± 3.4 pp, low confidence
CursorBench 3.228.9%estimated ± 4.4 pp, medium confidence
EBR-bench21.6%estimated ± 11.3 pp, low confidence
FrontierCode 1.1 Main15.9%estimated ± 4.6 pp, medium confidence
FrontierSWE v211.2%estimated ± 14.2 pp, low confidence
Furniture Assembly33.8%estimated ± 12.4 pp, low confidence
MaxxBench43.1%estimated ± 5.7 pp, low confidence
MazeBench0.4%estimated ± 5.5 pp, low confidence
Mystery Game Puzzles17.2%estimated ± 13.3 pp, low confidence
NYT Connections70.6%measured
OfficeQA Pro50.3%estimated ± 6.8 pp, low confidence
OSWorld-Verified54.4%estimated ± 5.5 pp, medium confidence
ParseBench-162.4%measured
Vals SWE-bench65.7%estimated ± 5.5 pp, medium confidence
CWE-Bench-AA9.2%estimated ± 10.1 pp, low confidence
AA Cyber Index4.9%estimated ± 9.2 pp, low confidence
DeepsecBench-AA0.0%estimated ± 8.9 pp, low confidence
HealthBench Professional (length-adjusted, Artificial Analysis)57.5%estimated ± 4.4 pp, low confidence
HealthBench Professional (raw, Artificial Analysis)12.7%estimated ± 2.1 pp, low confidence
AA ITBench37.3%measured
Bug Hunt Bench18.5%estimated ± 10.5 pp, low confidence
Chartography (tools)79.2%estimated ± 4.1 pp, low confidence
CursorBench 4.031.8%estimated ± 4.9 pp, medium confidence
CWE-bench v19.9%estimated ± 6.4 pp, low confidence
DeepSWE47.2%estimated ± 7.5 pp, low confidence
DRACO38.4%estimated ± 2.0 pp, low confidence
FrontierCode 2.026.1%estimated ± 9.3 pp, low confidence
FrontierPhysics1.2%estimated ± 4.7 pp, medium confidence
HLE w/o tools19.5%measured
PostTrainBench v1.127.0%estimated ± 7.4 pp, low confidence
SpireClimb5.8%estimated ± 7.2 pp, medium confidence
StealthMark29.5%measured
SWE-bench Multimodal38.9%estimated ± 5.6 pp, low confidence
Toolathlon-Verified22.4%estimated ± 2.3 pp, low confidence
VulcanBench Frontier v475.7%estimated ± 6.8 pp, low confidence
GDP.pdf17.0%estimated ± 4.6 pp, medium confidence
ARC-AGI-30.0%estimated ± 7.2 pp, low confidence
ARC-AGI-3 Provider Adapter2.7%estimated ± 7.3 pp, low confidence
cursorBench4144.8%estimated ± 3.7 pp, medium confidence
SkillsBench official leaderboard (BenchFlow)52.8%estimated ± 6.1 pp, medium confidence
BinaryAudit35.8%estimated ± 12.1 pp, low confidence
OTelBench22.5%estimated ± 6.6 pp, medium confidence
Vibe Code Bench27.7%estimated ± 6.5 pp, low confidence
FACTS Multimodal42.3%measured
Hermes Bench24.0%estimated ± 2.9 pp, medium confidence
Hermes Index5.9%estimated ± 8.7 pp, low confidence
ParseBench-267.0%estimated ± 3.5 pp, high confidence
CharXiv84.6%estimated ± 5.6 pp, low confidence
CharXiv w/o tools81.0%estimated ± 1.2 pp, high confidence
ExploitBench62.4%estimated ± 7.3 pp, low confidence
ExploitGym0.0%estimated ± 10.6 pp, low confidence
AI2D_TEST89.4%estimated ± 2.0 pp, medium confidence
C-Eval92.2%estimated ± 0.9 pp, medium confidence
Claw-Eval52.7%estimated ± 6.8 pp, low confidence
CountBench93.9%estimated ± 2.9 pp, medium confidence
DynaMath84.6%estimated ± 3.0 pp, medium confidence
ERQA56.4%estimated ± 3.1 pp, high confidence
MathVision86.1%estimated ± 3.0 pp, high confidence
MCP Atlas69.1%estimated ± 6.8 pp, medium confidence
MLVU (M-Avg)85.4%estimated ± 2.2 pp, high confidence
MMLU-Redux93.3%estimated ± 0.6 pp, high confidence
MMMU-Pro76.9%measured
NL2Repo29.1%estimated ± 3.4 pp, medium confidence
QwenClawBench55.9%estimated ± 6.0 pp, low confidence
RealWorldQA83.4%estimated ± 2.8 pp, high confidence
ScreenSpot Pro74.7%estimated ± 9.6 pp, low confidence
SimpleVQA65.0%estimated ± 7.5 pp, medium confidence
SuperGPQA67.2%estimated ± 7.2 pp, medium confidence
Toolathlon48.8%estimated ± 5.1 pp, low confidence
V*100.0%estimated ± 7.5 pp, medium confidence
Video-MME (w/o subtitle)83.9%estimated ± 1.3 pp, high confidence
Video-MME (with subtitle)86.3%estimated ± 3.0 pp, high confidence
VideoMMMU84.7%estimated ± 2.4 pp, high confidence
We-Math81.7%estimated ± 7.4 pp, low confidence
AA LiveCodeBench88.1%estimated ± 4.7 pp, low confidence
HealthBench Hard12.3%estimated ± 8.6 pp, low confidence
MedXpertQA (MM)67.7%estimated ± 2.4 pp, high confidence
MedXpertQA (Text)32.8%estimated ± 8.9 pp, low confidence
MMLU-Pro (Arcee)82.4%estimated ± 4.0 pp, medium confidence
React Native Evals75.2%measured
ResearchClawBench9.5%estimated ± 1.9 pp, medium confidence
SWE-Rebench41.6%measured
DAYJOB0.0%estimated ± 3.2 pp, medium confidence
DAYJOB Healthcare0.0%estimated ± 4.7 pp, medium confidence
BioMysteryBench (human-solvable)81.7%estimated ± 0.6 pp, low confidence
HealthBench (raw)31.9%estimated ± 6.0 pp, low confidence
HealthBench (length-adjusted)47.3%estimated ± 3.4 pp, low confidence
HealthBench Professional28.3%estimated ± 4.2 pp, low confidence
HealthBench Professional (raw)43.6%estimated ± 4.5 pp, medium confidence
HLE-Verified15.3%estimated ± 7.1 pp, low confidence
ScarfBench24.4%estimated ± 9.1 pp, low confidence
BlenderWorld (text-only)64.8%estimated ± 4.3 pp, low confidence
Cookie-Bench78.0%estimated ± 4.8 pp, low confidence
STEMBench2.3%estimated ± 0.2 pp, low confidence
SimpleQA18.0%estimated ± 4.3 pp, medium confidence
BabyVision w/ Python88.8%estimated ± 3.8 pp, low confidence
OpenHarmony Bench v1.051.0%estimated ± 2.4 pp, high confidence
SEC-Bench Pro56.5%estimated ± 5.6 pp, medium confidence
BabyVision58.2%estimated ± 9.1 pp, low confidence
MMVU76.5%estimated ± 2.7 pp, medium confidence
PerceptionBench59.1%estimated ± 4.9 pp, low confidence
MRCRv266.4%measured
MMLongBench-Doc60.9%estimated ± 2.6 pp, low confidence
OCRBench V270.8%estimated ± 3.3 pp, high confidence
BioBasicsBench33.9%estimated ± 2.8 pp, low confidence
MMMU-Pro w/ Python76.8%estimated ± 2.0 pp, high confidence
ExploitBench Internal Port0.0%estimated ± 6.8 pp, low confidence