benchgap
Other · model

Gemma 4 12B benchmark scores

As of 2026-11-22, Gemma 4 12B (Other) has measured scores on 21 benchmarks and estimated scores on 100 more.

BenchmarkScoreSource
AA-GPQA Diamond75.3%measured
AA-HLE15.7%measured
AA-SciCode36.2%estimated ± 3.4 pp, high confidence
AIME2677.5%measured
Artificial Analysis Intelligence Index10.7%estimated ± 5.5 pp, low confidence
CritPt0.0%measured
GDPval-AA0.0%measured
GPQA Diamond78.8%measured
HLE8.0%estimated ± 6.5 pp, low confidence
IFBench71.5%estimated ± 6.6 pp, medium confidence
AA-LCR63.7%measured
LiveCodeBench v672.0%measured
AA-Omniscience Accuracy15.7%measured
SciCode36.8%estimated ± 4.6 pp, high confidence
BrowseComp0.0%estimated ± 9.2 pp, low confidence
HLE w/ tools33.8%estimated ± 5.3 pp, low confidence
IFEval93.0%estimated ± 2.8 pp, high confidence
LongBench v255.6%estimated ± 5.5 pp, medium confidence
τ²-bench results36.3%measured
AA Coding Index31.0%measured
AA-MMMU-Pro69.7%measured
APEX-Agents0.0%estimated ± 9.4 pp, low confidence
APEX-Agents-AA6.0%estimated ± 9.5 pp, medium confidence
BioMysteryBench (human-difficult)0.0%estimated ± 6.8 pp, low confidence
DeepSearchQA34.7%estimated ± 9.6 pp, medium confidence
SWE-bench Verified57.5%estimated ± 6.5 pp, medium confidence
BFCL v453.9%estimated ± 2.7 pp, high confidence
CyberGym0.0%estimated ± 6.0 pp, low confidence
JobBench0.0%estimated ± 8.7 pp, low confidence
SWE-bench Pro22.3%estimated ± 6.0 pp, medium confidence
WideResearch0.0%estimated ± 7.8 pp, low confidence
AA Agentic Index0.0%estimated ± 3.1 pp, high confidence
MMLU-Pro77.2%measured
AA-IFBench73.5%measured
BBH94.2%estimated ± 2.8 pp, medium confidence
GPQA78.8%measured
MMLU80.4%estimated ± 0.9 pp, high confidence
Vals GPQA Diamond78.5%estimated ± 2.1 pp, medium confidence
Vals LiveCodeBench66.1%estimated ± 7.1 pp, medium confidence
Vals MMLU-Pro80.2%estimated ± 1.6 pp, medium confidence
MATH-50095.2%estimated ± 3.1 pp, medium confidence
MMMLU83.4%measured
MMMU79.3%estimated ± 0.8 pp, high confidence
AIME 202573.9%estimated ± 1.8 pp, medium confidence
AA MMLU-Pro83.4%estimated ± 0.8 pp, medium confidence
Gert Labs26.9%estimated ± 10.0 pp, low confidence
AA Tau3 Banking0.0%estimated ± 4.1 pp, medium confidence
ARC-AGI-113.6%estimated ± 3.9 pp, medium confidence
ARC-AGI-20.0%estimated ± 10.9 pp, low confidence
FrontierSWE v20.2%estimated ± 14.3 pp, low confidence
MazeBench0.4%estimated ± 5.5 pp, low confidence
OfficeQA Pro50.3%estimated ± 6.8 pp, low confidence
OSWorld-Verified55.4%estimated ± 8.4 pp, low confidence
ParseBench-153.5%estimated ± 6.2 pp, medium confidence
Vals SWE-bench54.8%estimated ± 5.5 pp, medium confidence
CWE-Bench-AA0.0%estimated ± 10.1 pp, low confidence
DeepsecBench-AA0.0%estimated ± 8.9 pp, low confidence
HealthBench Professional (raw, Artificial Analysis)1.3%estimated ± 2.1 pp, low confidence
AA ITBench35.7%estimated ± 5.1 pp, low confidence
Chartography (tools)76.8%estimated ± 4.1 pp, low confidence
CWE-bench v10.0%estimated ± 6.4 pp, low confidence
DRACO0.0%estimated ± 2.0 pp, low confidence
HLE w/o tools5.2%measured
Toolathlon-Verified0.0%estimated ± 3.1 pp, medium confidence
GDP.pdf17.0%estimated ± 4.6 pp, medium confidence
ARC-AGI-30.0%estimated ± 7.2 pp, low confidence
ARC-AGI-3 Provider Adapter2.7%estimated ± 7.3 pp, low confidence
Vibe Code Bench9.8%estimated ± 10.0 pp, low confidence
Hermes Bench0.0%estimated ± 2.9 pp, medium confidence
Hermes Index0.0%estimated ± 8.7 pp, low confidence
ParseBench-257.1%estimated ± 5.0 pp, medium confidence
CharXiv76.4%estimated ± 3.8 pp, high confidence
CharXiv w/o tools75.3%estimated ± 1.2 pp, high confidence
AI2D_TEST90.1%estimated ± 1.0 pp, medium confidence
C-Eval89.3%estimated ± 1.0 pp, medium confidence
CountBench91.7%estimated ± 1.3 pp, medium confidence
DynaMath66.9%estimated ± 3.0 pp, low confidence
ERQA43.3%estimated ± 3.1 pp, medium confidence
HMMT Feb 202655.9%estimated ± 4.0 pp, high confidence
MathVision79.7%measured
MCP Atlas56.5%estimated ± 10.7 pp, low confidence
MLVU (M-Avg)83.1%estimated ± 1.5 pp, high confidence
MMLU-Redux92.2%estimated ± 0.5 pp, medium confidence
MMMU-Pro69.1%measured
NL2Repo20.0%estimated ± 3.4 pp, medium confidence
OmniDocBench 1.587.6%estimated ± 2.2 pp, high confidence
QwenClawBench55.9%estimated ± 6.0 pp, low confidence
RealWorldQA79.6%estimated ± 2.3 pp, high confidence
ScreenSpot Pro53.6%estimated ± 9.3 pp, medium confidence
SimpleVQA64.0%estimated ± 7.3 pp, medium confidence
SuperGPQA59.7%estimated ± 8.5 pp, low confidence
Toolathlon27.0%estimated ± 6.3 pp, low confidence
V*45.6%estimated ± 7.5 pp, low confidence
Video-MME (w/o subtitle)79.5%estimated ± 1.3 pp, high confidence
Video-MME (with subtitle)83.2%estimated ± 1.9 pp, high confidence
VideoMMMU78.6%estimated ± 2.4 pp, high confidence
We-Math69.6%estimated ± 2.1 pp, medium confidence
AA LiveCodeBench84.0%estimated ± 4.7 pp, low confidence
HealthBench Hard0.0%estimated ± 8.6 pp, low confidence
MedXpertQA (MM)48.7%measured
MMLU-Pro (Arcee)82.8%estimated ± 3.4 pp, medium confidence
React Native Evals71.8%estimated ± 3.2 pp, high confidence
ResearchClawBench11.1%estimated ± 2.2 pp, low confidence
BioMysteryBench (human-solvable)81.7%estimated ± 0.6 pp, low confidence
HealthBench (raw)22.4%estimated ± 6.0 pp, low confidence
HealthBench (length-adjusted)44.3%estimated ± 3.4 pp, low confidence
HealthBench Professional58.1%estimated ± 4.5 pp, low confidence
HealthBench Professional (raw)34.9%estimated ± 4.5 pp, medium confidence
ScarfBench24.4%estimated ± 9.1 pp, low confidence
STEMBench2.3%estimated ± 0.2 pp, low confidence
SimpleQA12.2%estimated ± 4.3 pp, medium confidence
BabyVision w/ Python88.8%estimated ± 3.8 pp, low confidence
IMOAnswerBench64.9%estimated ± 10.7 pp, low confidence
BabyVision32.7%estimated ± 3.7 pp, high confidence
MMVU71.5%estimated ± 2.7 pp, medium confidence
PerceptionBench54.7%estimated ± 4.9 pp, low confidence
MRCRv243.4%measured
MMLongBench-Doc60.6%estimated ± 2.6 pp, low confidence
OCRBench V268.6%estimated ± 3.3 pp, high confidence
BioBasicsBench33.9%estimated ± 10.7 pp, low confidence
MMMU-Pro w/ Python73.5%estimated ± 2.0 pp, high confidence