benchgap
Alibaba (Qwen) · model

Qwen3.5-27B benchmark scores

As of 2026-11-22, Qwen3.5-27B (Alibaba (Qwen)) has measured scores on 30 benchmarks and estimated scores on 125 more.

BenchmarkScoreSource
AA-GPQA Diamond85.8%measured
AA-HLE23.9%measured
AA-SciCode44.9%estimated ± 3.8 pp, high confidence
Artificial Analysis Intelligence Index22.9%measured
CritPt0.9%measured
GDPval-AA22.9%estimated ± 7.8 pp, medium confidence
GPQA Diamond85.6%estimated ± 1.2 pp, high confidence
HLE22.1%estimated ± 5.1 pp, medium confidence
IFBench73.8%estimated ± 6.6 pp, medium confidence
AA-LCR77.7%measured
LiveCodeBench v680.7%measured
AA-Omniscience Accuracy20.7%measured
SciCode44.1%estimated ± 3.6 pp, high confidence
BrowseComp61.0%measured
HLE w/ tools49.1%estimated ± 5.5 pp, medium confidence
IFEval95.0%measured
LongBench v260.6%measured
VITA-Bench32.6%estimated ± 6.7 pp, low confidence
τ²-bench results93.9%measured
AA Coding Index44.9%estimated ± 5.3 pp, medium confidence
AA-MMMU-Pro75.0%measured
APEX-Agents-AA11.3%estimated ± 4.4 pp, high confidence
BioMysteryBench (human-difficult)15.7%estimated ± 6.8 pp, low confidence
DeepSearchQA77.0%estimated ± 6.8 pp, low confidence
SWE-bench Verified72.4%measured
BFCL v468.4%estimated ± 8.5 pp, low confidence
CyberGym71.3%estimated ± 5.8 pp, low confidence
JobBench17.5%estimated ± 10.7 pp, low confidence
SkillsBench33.8%estimated ± 4.1 pp, low confidence
SWE-bench Pro50.1%estimated ± 4.5 pp, high confidence
WideResearch73.3%estimated ± 2.1 pp, low confidence
AA Agentic Index11.0%estimated ± 8.0 pp, medium confidence
MMLU-Pro86.1%measured
AA-IFBench75.6%measured
BBH99.4%estimated ± 2.8 pp, medium confidence
FailureSensorIQ65.0%estimated ± 1.7 pp, high confidence
GPQA85.5%measured
MMLU88.6%estimated ± 0.9 pp, high confidence
Vals GPQA Diamond84.9%estimated ± 3.2 pp, high confidence
Vals LiveCodeBench0.2%estimated ± 3.4 pp, low confidence
Vals MMLU-Pro85.0%estimated ± 2.3 pp, high confidence
MMMLU86.3%estimated ± 0.7 pp, medium confidence
MMMU82.3%measured
Chess Puzzles20.2%estimated ± 12.3 pp, low confidence
AA MMLU-Pro84.2%estimated ± 0.8 pp, medium confidence
Gert Labs28.9%measured
AA Harvey LAB v1.085.7%estimated ± 9.8 pp, low confidence
AA Tau3 Banking23.5%estimated ± 3.6 pp, low confidence
ApprenticeBench3.2%estimated ± 10.4 pp, low confidence
ARC-AGI-168.5%estimated ± 6.7 pp, medium confidence
ARC-AGI-224.5%estimated ± 10.9 pp, low confidence
FrontierCode 1.1 Main14.1%estimated ± 4.2 pp, low confidence
MazeBench0.0%estimated ± 2.7 pp, high confidence
Mystery Game Puzzles14.5%estimated ± 14.6 pp, low confidence
NYT Connections47.9%measured
OfficeQA Pro50.3%estimated ± 6.8 pp, low confidence
OSWorld-Verified56.2%measured
ParseBench-159.1%estimated ± 6.2 pp, medium confidence
Terminal-Bench Hard41.5%estimated ± 4.0 pp, medium confidence
Vals SWE-bench65.9%estimated ± 6.6 pp, medium confidence
Terminal-Bench 2.1 (Vals AI)36.1%estimated ± 6.4 pp, medium confidence
CWE-Bench-AA41.1%estimated ± 6.4 pp, medium confidence
AA Cyber Index18.1%estimated ± 12.4 pp, low confidence
DeepsecBench-AA8.8%estimated ± 8.2 pp, low confidence
HealthBench Professional (length-adjusted, Artificial Analysis)52.8%estimated ± 4.3 pp, low confidence
HealthBench Professional (raw, Artificial Analysis)57.8%estimated ± 1.1 pp, low confidence
AA ITBench35.5%measured
Chartography (tools)80.3%estimated ± 4.1 pp, low confidence
CWE-bench v138.6%estimated ± 5.7 pp, medium confidence
DeepSWE33.5%estimated ± 12.7 pp, low confidence
DRACO60.9%estimated ± 2.9 pp, low confidence
HLE w/o tools34.2%estimated ± 4.9 pp, high confidence
OSWorld 2.00.0%estimated ± 3.6 pp, medium confidence
OSWorld 2.19.6%estimated ± 3.6 pp, medium confidence
SpireClimb5.8%estimated ± 7.2 pp, low confidence
StealthMark41.3%estimated ± 10.7 pp, low confidence
SWE Multilingual69.9%estimated ± 3.2 pp, low confidence
Toolathlon-Verified62.6%estimated ± 3.0 pp, high confidence
AA AutomationBench42.7%estimated ± 9.9 pp, low confidence
GDP.pdf13.1%estimated ± 4.2 pp, medium confidence
ARC-AGI-31.3%estimated ± 9.8 pp, medium confidence
ARC-AGI-3 Provider Adapter2.7%estimated ± 7.3 pp, low confidence
CWE-bench32.5%estimated ± 6.2 pp, low confidence
MKA32.9%estimated ± 4.6 pp, medium confidence
SkillsBench official leaderboard (BenchFlow)47.6%estimated ± 6.1 pp, medium confidence
BinaryAudit35.8%estimated ± 12.1 pp, low confidence
OTelBench22.5%estimated ± 6.6 pp, medium confidence
Vibe Code Bench0.3%estimated ± 8.4 pp, medium confidence
FACTS Multimodal28.2%estimated ± 4.0 pp, medium confidence
Hermes Index19.8%estimated ± 5.7 pp, low confidence
ParseBench-264.9%estimated ± 5.0 pp, medium confidence
CharXiv79.5%measured
CharXiv w/o tools79.3%estimated ± 1.2 pp, high confidence
ExploitGym0.0%estimated ± 8.3 pp, low confidence
AI2D_TEST93.4%estimated ± 0.9 pp, medium confidence
C-Eval92.4%estimated ± 1.0 pp, medium confidence
Claw-Eval54.1%estimated ± 6.2 pp, low confidence
CountBench97.1%estimated ± 1.3 pp, medium confidence
DynaMath85.5%estimated ± 0.8 pp, medium confidence
ERQA63.3%estimated ± 3.1 pp, high confidence
INCLUDE83.2%estimated ± 1.4 pp, low confidence
MathVision86.0%measured
MAXIFE74.8%estimated ± 6.5 pp, low confidence
MCP Atlas66.9%estimated ± 6.8 pp, medium confidence
MLVU (M-Avg)85.7%estimated ± 1.5 pp, high confidence
MMLU-ProX82.2%measured
MMLU-Redux93.8%estimated ± 0.6 pp, high confidence
MMMU-Pro75.0%measured
NL2Repo30.0%estimated ± 3.4 pp, high confidence
NOVA-6356.3%estimated ± 1.3 pp, low confidence
OmniDocBench 1.589.4%estimated ± 2.2 pp, high confidence
PolyMath42.5%estimated ± 11.4 pp, low confidence
PostTrainBench v111.8%estimated ± 5.5 pp, low confidence
RealWorldQA83.2%estimated ± 1.5 pp, high confidence
ScreenSpot Pro70.7%estimated ± 6.8 pp, medium confidence
SimpleVQA62.1%estimated ± 7.1 pp, medium confidence
SuperGPQA65.6%measured
Terminal-Bench 2.041.6%measured
Toolathlon32.6%estimated ± 4.2 pp, high confidence
V*93.7%measured
Video-MME (w/o subtitle)82.3%estimated ± 1.2 pp, high confidence
Video-MME (with subtitle)86.5%estimated ± 1.9 pp, high confidence
VideoMMMU85.2%estimated ± 1.5 pp, medium confidence
We-Math87.2%estimated ± 2.1 pp, medium confidence
HealthBench Hard8.9%estimated ± 8.6 pp, low confidence
MedXpertQA (MM)63.2%estimated ± 2.4 pp, high confidence
MMLU-Pro (Arcee)80.5%estimated ± 4.0 pp, medium confidence
React Native Evals76.2%estimated ± 2.7 pp, medium confidence
ResearchClawBench13.9%estimated ± 2.1 pp, low confidence
SWE-Rebench58.9%measured
Finance Agent v255.1%estimated ± 2.4 pp, low confidence
BioMysteryBench (human-solvable)81.7%estimated ± 0.6 pp, low confidence
HealthBench (raw)32.3%estimated ± 6.0 pp, low confidence
HealthBench (length-adjusted)47.5%estimated ± 3.4 pp, low confidence
HealthBench Professional56.7%estimated ± 4.2 pp, low confidence
HealthBench Professional (raw)43.9%estimated ± 4.5 pp, medium confidence
HLE-Verified16.0%estimated ± 7.1 pp, low confidence
STEMBench2.3%estimated ± 0.2 pp, low confidence
SimpleQA19.1%estimated ± 4.3 pp, medium confidence
LiveCodeBench74.6%estimated ± 14.6 pp, low confidence
Agents' Last Exam29.6%estimated ± 9.7 pp, low confidence
BabyVision w/ Python88.8%estimated ± 3.8 pp, low confidence
BabyVision47.9%estimated ± 3.7 pp, high confidence
MMVU73.3%measured
PerceptionBench52.8%estimated ± 3.6 pp, low confidence
ZeroBench15.7%estimated ± 3.6 pp, low confidence
MRCRv285.4%estimated ± 11.3 pp, low confidence
MMLongBench-Doc61.3%estimated ± 2.6 pp, low confidence
OCRBench V270.3%estimated ± 3.3 pp, high confidence
BioBasicsBench33.9%estimated ± 2.8 pp, low confidence
LifeVal39.3%estimated ± 1.6 pp, low confidence
MMMU-Pro w/ Python78.2%estimated ± 2.0 pp, high confidence
ExploitBench Internal Port0.4%estimated ± 13.8 pp, low confidence
MathVision w/ Python92.8%estimated ± 0.5 pp, low confidence
PinchBench88.9%estimated ± 3.6 pp, medium confidence