benchgap
Other · model

Mellum2-12B-A2.5B-Instruct benchmark scores

As of 2026-11-22, Mellum2-12B-A2.5B-Instruct (Other) has measured scores on 6 benchmarks and estimated scores on 25 more.

BenchmarkScoreSource
AA-GPQA Diamond54.9%estimated ± 4.4 pp, medium confidence
AA-HLE0.0%estimated ± 3.9 pp, medium confidence
AA-SciCode9.1%estimated ± 4.7 pp, medium confidence
Artificial Analysis Intelligence Index9.4%estimated ± 7.4 pp, low confidence
CritPt0.3%estimated ± 4.0 pp, high confidence
GDPval-AA0.0%estimated ± 5.5 pp, low confidence
GPQA Diamond40.9%measured
HLE0.6%estimated ± 6.5 pp, low confidence
IFBench35.7%estimated ± 9.2 pp, medium confidence
LiveCodeBench v637.2%measured
AA-Omniscience Accuracy2.3%estimated ± 8.4 pp, low confidence
SciCode11.0%estimated ± 4.9 pp, medium confidence
IFEval75.8%measured
τ²-bench results68.5%estimated ± 12.4 pp, low confidence
AA Coding Index3.1%estimated ± 5.7 pp, low confidence
SWE-bench Verified57.5%estimated ± 9.7 pp, low confidence
BFCL v444.2%measured
MMLU-Pro62.4%estimated ± 1.1 pp, high confidence
AA-IFBench20.2%estimated ± 8.8 pp, medium confidence
BBH82.7%estimated ± 4.5 pp, medium confidence
GPQA40.9%measured
MMLU82.7%estimated ± 4.2 pp, high confidence
Vals GPQA Diamond39.1%estimated ± 4.5 pp, medium confidence
Vals MMLU-Pro61.6%estimated ± 2.3 pp, medium confidence
MMMLU75.4%estimated ± 1.4 pp, low confidence
C-Eval77.1%estimated ± 1.2 pp, medium confidence
MMLU-Redux78.1%measured
SuperGPQA36.0%estimated ± 4.1 pp, high confidence
MMLU-Pro (Arcee)61.2%estimated ± 3.4 pp, low confidence
SimpleQA14.6%estimated ± 13.4 pp, low confidence
PinchBench50.6%estimated ± 5.0 pp, low confidence