benchgap
Alibaba (Qwen) · model

Qwen3.5 397B benchmark scores

As of 2026-11-22, Qwen3.5 397B (Alibaba (Qwen)) has measured scores on 69 benchmarks and estimated scores on 128 more.

BenchmarkScoreSource
AA-GPQA Diamond89.3%measured
AA-HLE29.0%measured
AA-SciCode44.8%measured
AIME2693.3%measured
Artificial Analysis Intelligence Index26.1%estimated ± 3.0 pp, high confidence
CritPt1.7%measured
GDPval-AA27.7%estimated ± 7.8 pp, medium confidence
GPQA Diamond88.5%estimated ± 1.2 pp, high confidence
HLE28.7%measured
IFBench76.5%measured
AA-LCR77.3%measured
LiveCodeBench v683.6%measured
AA-Omniscience Accuracy31.2%estimated ± 5.4 pp, low confidence
SciCode44.8%estimated ± 3.1 pp, high confidence
BrowseComp69.0%measured
HLE w/ tools50.4%estimated ± 4.1 pp, medium confidence
IFEval92.6%measured
LongBench v263.2%measured
VITA-Bench43.7%measured
τ²-bench results95.6%measured
AA Coding Index45.8%estimated ± 4.3 pp, high confidence
AA-MMMU-Pro77.3%measured
APEX-Agents19.8%estimated ± 4.9 pp, low confidence
APEX-Agents-AA15.3%measured
BioMysteryBench (human-difficult)23.5%estimated ± 6.8 pp, low confidence
DeepSearchQA79.6%estimated ± 6.8 pp, low confidence
SWE-bench Verified76.2%measured
AutomationBench48.5%estimated ± 9.1 pp, low confidence
BFCL v467.8%estimated ± 4.7 pp, low confidence
CyberGym56.6%estimated ± 7.3 pp, medium confidence
JobBench17.2%estimated ± 5.1 pp, low confidence
SkillsBench40.6%estimated ± 4.1 pp, low confidence
SWE-bench Pro50.9%measured
τ³-bench results68.4%measured
WideResearch74.0%measured
AA Agentic Index19.1%estimated ± 5.9 pp, medium confidence
MMLU-Pro87.8%measured
AA-IFBench78.8%measured
BBH100.0%estimated ± 2.8 pp, medium confidence
FailureSensorIQ66.4%estimated ± 5.0 pp, medium confidence
GPQA88.4%measured
MMLU91.7%estimated ± 2.1 pp, medium confidence
Vals GPQA Diamond88.4%estimated ± 3.2 pp, high confidence
Vals LiveCodeBench39.2%estimated ± 3.4 pp, low confidence
Vals MMLU-Pro86.0%estimated ± 2.3 pp, high confidence
MATH-50097.4%estimated ± 3.1 pp, medium confidence
MMMLU87.0%estimated ± 0.7 pp, medium confidence
MMMU85.0%measured
Chess Puzzles35.0%estimated ± 4.7 pp, medium confidence
AIME 202594.6%estimated ± 1.8 pp, medium confidence
AA MMLU-Pro86.8%estimated ± 1.2 pp, low confidence
Gert Labs29.2%measured
AA Harvey LAB v1.087.8%estimated ± 9.8 pp, low confidence
AA Tau3 Banking39.2%estimated ± 2.7 pp, low confidence
ApprenticeBench0.0%estimated ± 6.3 pp, low confidence
ARC-AGI-177.6%estimated ± 6.7 pp, medium confidence
ARC-AGI-243.0%estimated ± 10.9 pp, low confidence
CursorBench 3.131.2%estimated ± 3.4 pp, low confidence
CursorBench 3.242.2%estimated ± 3.6 pp, medium confidence
EBR-bench6.6%estimated ± 10.4 pp, low confidence
FrontierCode 1.1 Main20.8%estimated ± 4.2 pp, low confidence
FrontierMath v2 (Tier 4)12.6%estimated ± 4.1 pp, medium confidence
FrontierMath v2 (Tiers 1-3)30.1%estimated ± 7.8 pp, low confidence
FrontierSWE v20.0%estimated ± 8.3 pp, low confidence
Furniture Assembly32.9%estimated ± 9.3 pp, medium confidence
MaxxBench47.2%estimated ± 8.8 pp, medium confidence
MazeBench0.0%estimated ± 2.7 pp, high confidence
Mystery Game Puzzles18.0%measured
NYT Connections60.7%estimated ± 13.8 pp, low confidence
OfficeQA Pro50.4%estimated ± 6.8 pp, medium confidence
OSWorld-Verified62.2%measured
ParseBench-152.1%estimated ± 5.2 pp, low confidence
Terminal-Bench Hard42.7%estimated ± 4.0 pp, medium confidence
Vals SWE-bench65.9%estimated ± 5.2 pp, low confidence
Terminal-Bench 2.1 (Vals AI)46.1%estimated ± 6.4 pp, medium confidence
CWE-Bench-AA45.3%estimated ± 6.4 pp, medium confidence
AA Cyber Index18.8%estimated ± 12.4 pp, low confidence
DeepsecBench-AA8.9%estimated ± 8.2 pp, low confidence
HealthBench Professional (raw, Artificial Analysis)30.6%estimated ± 2.1 pp, low confidence
AA ITBench40.7%estimated ± 4.1 pp, high confidence
Bug Hunt Bench7.4%estimated ± 7.2 pp, low confidence
Chartography (tools)81.7%estimated ± 4.1 pp, low confidence
CursorBench 4.028.7%estimated ± 3.3 pp, medium confidence
CWE-bench v138.8%estimated ± 5.7 pp, medium confidence
DeepSWE45.4%estimated ± 7.5 pp, low confidence
DRACO67.8%estimated ± 2.9 pp, low confidence
FrontierCode 2.019.1%estimated ± 6.7 pp, low confidence
FrontierPhysics2.5%estimated ± 5.0 pp, medium confidence
HLE w/o tools29.1%estimated ± 5.3 pp, medium confidence
OSWorld 2.00.0%estimated ± 3.6 pp, medium confidence
OSWorld 2.110.1%estimated ± 3.6 pp, medium confidence
OSWorld 2.1 Offline39.2%estimated ± 10.1 pp, low confidence
PostTrainBench v1.120.4%estimated ± 4.4 pp, medium confidence
SpireClimb0.0%estimated ± 9.3 pp, medium confidence
StealthMark37.8%estimated ± 9.2 pp, medium confidence
SWE Multilingual69.3%measured
Toolathlon-Verified62.8%estimated ± 3.0 pp, high confidence
VulcanBench Frontier v475.0%estimated ± 6.8 pp, low confidence
AA AutomationBench42.8%estimated ± 9.9 pp, low confidence
GDP.pdf13.2%estimated ± 4.2 pp, medium confidence
ARC-AGI-30.0%estimated ± 9.3 pp, low confidence
ARC-AGI-3 Provider Adapter2.7%estimated ± 7.3 pp, low confidence
cursorBench4144.8%estimated ± 3.7 pp, medium confidence
CWE-bench32.7%estimated ± 6.2 pp, low confidence
MKA32.9%estimated ± 4.6 pp, medium confidence
SkillsBench official leaderboard (BenchFlow)49.2%estimated ± 6.3 pp, low confidence
CompileBench80.0%estimated ± 10.0 pp, low confidence
OTelBench20.6%estimated ± 4.1 pp, medium confidence
Vibe Code Bench6.3%estimated ± 8.4 pp, medium confidence
FACTS Multimodal41.5%estimated ± 5.8 pp, low confidence
Hermes Index20.0%estimated ± 5.7 pp, low confidence
ParseBench-270.6%estimated ± 5.0 pp, medium confidence
CharXiv80.8%measured
CharXiv w/o tools83.1%estimated ± 1.2 pp, high confidence
ExploitBench33.6%estimated ± 7.2 pp, low confidence
ExploitGym0.0%estimated ± 8.3 pp, low confidence
AI2D_TEST93.9%measured
C-Eval93.0%measured
Claw-Eval56.8%measured
CountBench97.2%measured
DeepPlanning37.6%measured
DynaMath86.3%measured
ERQA67.5%measured
HMMT Feb 202594.8%measured
HMMT Feb 202687.9%measured
HMMT Nov 202592.7%measured
INCLUDE85.6%measured
MathVision88.6%measured
MAXIFE88.2%measured
MCP Atlas46.1%measured
MCP-Tasks74.2%measured
MLVU (M-Avg)86.7%measured
MMAnswerBench80.9%measured
MMLU-ProX84.7%measured
MMLU-Redux94.9%measured
MMMU-Pro79.0%measured
NL2Repo32.2%measured
NOVA-6359.1%measured
OmniDocBench 1.590.8%measured
PolyMath73.3%measured
PostTrainBench v115.7%estimated ± 4.6 pp, medium confidence
QwenClawBench51.8%measured
RealWorldQA83.9%measured
ScreenSpot Pro65.6%measured
SimpleVQA67.1%measured
SuperGPQA70.4%measured
Terminal-Bench 2.052.5%measured
Toolathlon36.3%measured
V*95.8%measured
Video-MME (w/o subtitle)83.7%measured
Video-MME (with subtitle)87.5%measured
VideoMMMU84.7%measured
We-Math87.9%measured
HealthBench Hard0.1%estimated ± 5.1 pp, low confidence
LiveCodeBench Pro76.2%estimated ± 6.0 pp, low confidence
MedXpertQA (MM)72.4%estimated ± 2.4 pp, high confidence
MMLU-Pro (Arcee)83.5%estimated ± 4.0 pp, medium confidence
React Native Evals77.9%estimated ± 2.7 pp, medium confidence
ResearchClawBench14.2%measured
SWE-Rebench58.0%estimated ± 3.6 pp, medium confidence
PostTrainBench18.9%estimated ± 3.7 pp, medium confidence
DAYJOB1.3%estimated ± 3.2 pp, high confidence
DAYJOB Healthcare0.7%estimated ± 4.2 pp, high confidence
Finance Agent v255.1%estimated ± 2.4 pp, low confidence
BioMysteryBench (human-solvable)81.7%estimated ± 0.6 pp, low confidence
FrontierCode 1.1 Extended52.7%estimated ± 3.0 pp, low confidence
HealthBench (raw)37.8%estimated ± 6.0 pp, low confidence
HealthBench (length-adjusted)49.4%estimated ± 3.4 pp, low confidence
HealthBench Professional48.0%estimated ± 4.2 pp, low confidence
HealthBench Professional (raw)49.1%estimated ± 4.5 pp, medium confidence
HLE-Verified16.0%estimated ± 2.0 pp, low confidence
LABBench276.0%estimated ± 1.8 pp, low confidence
ScarfBench44.0%estimated ± 14.0 pp, low confidence
BlenderWorld (text-only)70.1%estimated ± 2.8 pp, medium confidence
Cookie-Bench77.0%estimated ± 1.3 pp, medium confidence
SimpleQA33.3%estimated ± 7.8 pp, low confidence
LiveCodeBench77.0%estimated ± 14.6 pp, low confidence
Agents' Last Exam29.6%estimated ± 9.7 pp, low confidence
BabyVision w/ Python88.8%estimated ± 3.8 pp, low confidence
OpenHarmony Bench v1.051.0%estimated ± 2.4 pp, high confidence
SEC-Bench Pro41.1%estimated ± 9.3 pp, medium confidence
IMOAnswerBench82.1%estimated ± 5.6 pp, medium confidence
BabyVision55.9%estimated ± 3.7 pp, high confidence
MMVU77.8%estimated ± 2.7 pp, medium confidence
PerceptionBench55.7%estimated ± 3.6 pp, medium confidence
ZeroBench15.7%estimated ± 3.6 pp, medium confidence
MRCRv285.3%estimated ± 11.3 pp, low confidence
CC-OCR82.0%measured
MMLongBench-Doc61.5%measured
RefCOCO (avg)92.3%measured
OCRBench V270.8%estimated ± 2.1 pp, medium confidence
BioBasicsBench33.9%estimated ± 2.8 pp, low confidence
LifeVal55.5%estimated ± 1.6 pp, medium confidence
MMMU-Pro w/ Python80.3%estimated ± 2.0 pp, high confidence
ExploitBench Internal Port0.1%estimated ± 5.2 pp, low confidence
MathVision w/ Python94.3%estimated ± 0.5 pp, low confidence
PinchBench89.5%estimated ± 3.6 pp, medium confidence