benchgap
Alibaba (Qwen) · model

Qwen3.5 397B benchmark scores

As of 2026-10-07, Qwen3.5 397B (Alibaba (Qwen)) has measured scores on 46 benchmarks and estimated scores on 81 more.

BenchmarkScoreSource
AA-GPQA Diamond86.1%measured
AA-HLE19.8%measured
AA-SciCode42.3%estimated ± 3.7 pp, high confidence
AIME2693.3%measured
Artificial Analysis Intelligence Index21.4%measured
CritPt0.9%measured
GDPval-AA22.7%estimated ± 3.7 pp, low confidence
GPQA Diamond88.4%estimated ± 0.4 pp, high confidence
HLE28.7%measured
IFBench57.9%estimated ± 6.9 pp, medium confidence
AA-LCR64.3%measured
LiveCodeBench v683.6%measured
AA-Omniscience Accuracy24.5%measured
SciCode46.7%estimated ± 3.8 pp, high confidence
BrowseComp62.0%measured
HLE w/ tools44.3%estimated ± 2.4 pp, medium confidence
IFEval92.6%measured
LongBench v263.2%measured
VITA-Bench43.7%measured
τ²-bench results83.9%measured
AA Coding Index52.8%estimated ± 5.1 pp, medium confidence
AA-MMMU-Pro52.7%measured
APEX-Agents6.7%estimated ± 0.6 pp, low confidence
APEX-Agents-AA10.9%estimated ± 4.6 pp, high confidence
BioMysteryBench (human-difficult)30.9%estimated ± 5.9 pp, low confidence
DeepSearchQA77.5%estimated ± 8.1 pp, low confidence
SWE-bench Verified76.2%measured
AutomationBench17.8%estimated ± 10.3 pp, low confidence
BFCL v446.7%estimated ± 8.6 pp, low confidence
CyberGym65.5%estimated ± 4.6 pp, medium confidence
JobBench17.2%estimated ± 5.1 pp, low confidence
SkillsBench45.7%estimated ± 5.6 pp, low confidence
SWE-bench Pro50.9%measured
τ³-bench results68.4%measured
WideResearch74.0%measured
AA Agentic Index17.3%estimated ± 5.1 pp, medium confidence
MMLU-Pro87.8%measured
FrontierMath v2 (Tier 4)0.1%estimated ± 3.7 pp, low confidence
FrontierMath v2 (Tiers 1-3)26.8%estimated ± 8.4 pp, low confidence
GPQA88.4%measured
AA-IFBench51.6%measured
Gert Labs46.8%measured
AA-AnalystAgent9.5%estimated ± 12.5 pp, low confidence
AA EnterpriseOps-Gym28.1%estimated ± 3.6 pp, medium confidence
AA Harvey LAB86.0%estimated ± 9.8 pp, low confidence
ApprenticeBench0.0%estimated ± 3.1 pp, low confidence
ARC-AGI-158.3%estimated ± 5.4 pp, medium confidence
ARC-AGI-242.7%estimated ± 11.6 pp, low confidence
CursorBench 3.148.4%estimated ± 5.6 pp, low confidence
CursorBench 3.242.2%estimated ± 3.6 pp, medium confidence
FrontierCode 1.1 Main13.6%estimated ± 2.7 pp, low confidence
FrontierSWE v20.0%estimated ± 9.0 pp, low confidence
OfficeQA Pro50.3%estimated ± 7.2 pp, low confidence
OSWorld-Verified53.8%estimated ± 5.4 pp, medium confidence
Vals GPQA Diamond84.6%estimated ± 4.8 pp, high confidence
Vals LiveCodeBench78.3%estimated ± 4.7 pp, medium confidence
Vals MMLU-Pro85.8%estimated ± 1.2 pp, high confidence
Vals SWE-bench71.1%estimated ± 6.8 pp, medium confidence
Terminal-Bench 2.1 (Vals AI)45.9%estimated ± 6.3 pp, medium confidence
AA AutomationBench2.6%estimated ± 5.7 pp, low confidence
GDP.pdf9.3%estimated ± 6.6 pp, low confidence
AA ITBench21.4%estimated ± 3.6 pp, low confidence
AA Tau3 Banking17.9%estimated ± 10.2 pp, low confidence
CursorBench 4.028.5%estimated ± 3.6 pp, medium confidence
HLE w/o tools29.3%estimated ± 4.9 pp, high confidence
OSWorld 2.00.0%estimated ± 4.5 pp, low confidence
PostTrainBench v1.120.4%estimated ± 4.7 pp, medium confidence
SWE Multilingual76.8%estimated ± 2.1 pp, low confidence
Toolathlon-Verified57.0%estimated ± 5.5 pp, low confidence
Vibe Code Bench7.7%estimated ± 8.1 pp, medium confidence
CharXiv80.8%measured
CharXiv w/o tools83.7%estimated ± 1.7 pp, medium confidence
ExploitGym0.0%estimated ± 6.1 pp, low confidence
C-Eval93.0%measured
Claw-Eval56.8%measured
DeepPlanning37.6%measured
HMMT Feb 202594.8%measured
HMMT Feb 202687.9%measured
HMMT Nov 202592.7%measured
MathVision88.6%measured
MCP Atlas46.1%measured
MCP-Tasks74.2%measured
MMAnswerBench80.9%measured
MMLU-ProX84.7%measured
MMLU-Redux94.9%measured
MMMU-Pro79.0%measured
NL2Repo43.8%estimated ± 3.1 pp, low confidence
NOVA-6359.1%measured
QwenClawBench51.8%measured
ScreenSpot Pro65.6%measured
SuperGPQA70.4%measured
Terminal-Bench 2.052.5%measured
Toolathlon36.3%measured
V*95.8%measured
VideoMMMU84.7%measured
ERQA66.9%estimated ± 2.4 pp, low confidence
HealthBench Hard0.1%estimated ± 5.1 pp, low confidence
LiveCodeBench Pro76.2%estimated ± 6.0 pp, low confidence
MedXpertQA (MM)72.4%estimated ± 2.4 pp, high confidence
MMLU-Pro (Arcee)80.5%estimated ± 4.0 pp, medium confidence
ResearchClawBench14.2%measured
SWE-Rebench58.0%estimated ± 3.6 pp, medium confidence
ARC-AGI-30.0%estimated ± 2.1 pp, medium confidence
BioMysteryBench (human-solvable)81.7%estimated ± 0.6 pp, low confidence
FrontierCode 1.1 Extended52.7%estimated ± 3.0 pp, low confidence
HealthBench (raw)27.7%estimated ± 6.4 pp, low confidence
HealthBench (length-adjusted)53.9%estimated ± 3.7 pp, low confidence
HealthBench Professional47.0%estimated ± 4.8 pp, low confidence
HealthBench Professional (raw)39.8%estimated ± 5.0 pp, low confidence
HLE-Verified15.3%estimated ± 2.0 pp, low confidence
LABBench271.2%estimated ± 1.8 pp, low confidence
AIME 202594.1%estimated ± 1.5 pp, medium confidence
MMMU70.8%estimated ± 0.9 pp, medium confidence
LiveCodeBench84.1%estimated ± 10.2 pp, low confidence
OpenHarmony Bench53.3%estimated ± 4.9 pp, low confidence
IMOAnswerBench82.5%estimated ± 5.9 pp, medium confidence
BabyVision56.5%estimated ± 4.3 pp, medium confidence
MMVU77.0%estimated ± 2.9 pp, medium confidence
SimpleVQA91.7%estimated ± 5.7 pp, low confidence
ZeroBench15.6%estimated ± 4.7 pp, low confidence
MMLU88.7%estimated ± 1.1 pp, medium confidence
MMMU-Pro w/ Python80.3%estimated ± 0.6 pp, high confidence
MATH-50097.4%estimated ± 3.1 pp, medium confidence
MathVision w/ Python94.3%estimated ± 0.5 pp, low confidence
RealWorldQA63.2%estimated ± 0.8 pp, low confidence
Video-MME (with subtitle)87.6%estimated ± 0.9 pp, medium confidence
OmniDocBench 1.591.7%estimated ± 5.9 pp, low confidence