benchgap
Other · model

ZAYA1-74B-Preview benchmark scores

As of 2026-11-22, ZAYA1-74B-Preview (Other) has measured scores on 6 benchmarks and estimated scores on 32 more.

BenchmarkScoreSource
AA-GPQA Diamond59.0%estimated ± 4.4 pp, medium confidence
AA-HLE3.1%estimated ± 3.9 pp, medium confidence
AA-SciCode35.0%estimated ± 3.8 pp, high confidence
AIME2676.4%measured
Artificial Analysis Intelligence Index9.5%estimated ± 7.4 pp, medium confidence
CritPt0.3%estimated ± 4.0 pp, high confidence
GPQA Diamond57.3%measured
HLE4.0%estimated ± 6.5 pp, low confidence
LiveCodeBench v665.7%measured
AA-Omniscience Accuracy5.5%estimated ± 8.4 pp, low confidence
SciCode34.4%estimated ± 3.6 pp, high confidence
AA Coding Index25.2%estimated ± 5.3 pp, medium confidence
SWE-bench Verified53.2%measured
SWE-bench Pro28.8%estimated ± 4.5 pp, high confidence
MMLU-Pro68.1%measured
BBH88.4%estimated ± 2.8 pp, medium confidence
GPQA57.3%measured
MMLU83.3%estimated ± 2.1 pp, high confidence
Vals GPQA Diamond55.3%estimated ± 4.5 pp, high confidence
Vals LiveCodeBench54.1%estimated ± 10.0 pp, low confidence
Vals MMLU-Pro72.6%estimated ± 2.3 pp, high confidence
MATH-50095.1%estimated ± 3.1 pp, medium confidence
MMMLU76.8%estimated ± 1.7 pp, high confidence
AIME 202572.3%estimated ± 1.8 pp, medium confidence
AA MMLU-Pro84.1%estimated ± 1.2 pp, low confidence
Vals SWE-bench40.1%estimated ± 6.6 pp, low confidence
HLE w/o tools8.7%estimated ± 4.3 pp, medium confidence
Vibe Code Bench0.0%estimated ± 8.4 pp, low confidence
C-Eval84.6%estimated ± 1.2 pp, high confidence
HMMT Feb 202654.4%estimated ± 4.0 pp, high confidence
MMLU-Redux82.9%estimated ± 0.6 pp, high confidence
NL2Repo6.2%estimated ± 6.2 pp, low confidence
SuperGPQA47.1%estimated ± 12.4 pp, low confidence
MMLU-Pro (Arcee)72.4%estimated ± 3.4 pp, low confidence
React Native Evals67.6%estimated ± 2.7 pp, low confidence
SimpleQA14.6%estimated ± 11.7 pp, low confidence
LiveCodeBench60.8%estimated ± 14.6 pp, low confidence
IMOAnswerBench64.3%estimated ± 10.7 pp, low confidence