benchgap
Alibaba (Qwen) · model

Qwen3.8-27B benchmark scores

As of 2026-11-22, Qwen3.8-27B (Alibaba (Qwen)) has measured scores on 56 benchmarks and estimated scores on 138 more.

BenchmarkScoreSource
AA-GPQA Diamond90.5%measured
AA-HLE33.9%measured
AA-SciCode46.6%measured
Artificial Analysis Intelligence Index20.1%measured
CritPt5.4%measured
GDPval-AA45.9%measured
GPQA Diamond89.2%measured
HLE30.8%measured
IFBench79.5%measured
AA-LCR82.0%measured
LiveCodeBench v690.3%measured
AA-Omniscience Accuracy17.3%measured
SciCode44.7%estimated ± 2.9 pp, medium confidence
Terminal-Bench 2.173.0%measured
BrowseComp84.8%estimated ± 4.0 pp, high confidence
HLE w/ tools51.2%estimated ± 5.1 pp, medium confidence
IFEval94.6%estimated ± 3.6 pp, high confidence
LongBench v267.4%estimated ± 5.5 pp, medium confidence
AA Coding Index68.1%measured
AA-MMMU-Pro76.3%measured
APEX-Agents38.2%estimated ± 9.4 pp, low confidence
APEX-Agents-AA31.5%estimated ± 3.6 pp, medium confidence
BioMysteryBench (human-difficult)38.0%estimated ± 6.8 pp, low confidence
DeepSearchQA92.9%estimated ± 4.5 pp, high confidence
SWE-bench Verified79.4%estimated ± 2.3 pp, high confidence
AutomationBench47.1%estimated ± 6.1 pp, low confidence
BFCL v476.3%estimated ± 2.7 pp, medium confidence
CyberGym84.4%estimated ± 3.1 pp, low confidence
JobBench33.4%measured
SWE-bench Pro61.7%measured
WideResearch77.6%estimated ± 1.2 pp, medium confidence
AA Agentic Index46.5%measured
MMLU-Pro85.2%estimated ± 2.5 pp, medium confidence
AA-IFBench80.3%estimated ± 7.4 pp, medium confidence
FailureSensorIQ65.4%estimated ± 1.7 pp, high confidence
GPQA89.2%measured
MMLU80.6%estimated ± 0.9 pp, high confidence
Vals GPQA Diamond88.9%measured
Vals LiveCodeBench84.0%measured
Vals MMLU-Pro84.3%measured
MMMLU88.5%estimated ± 0.7 pp, medium confidence
MMMU84.2%estimated ± 2.7 pp, high confidence
Chess Puzzles48.2%estimated ± 10.8 pp, low confidence
AA MMLU-Pro83.9%estimated ± 0.7 pp, medium confidence
Gert Labs34.5%measured
AA-AnalystAgent44.8%estimated ± 5.9 pp, low confidence
AA EnterpriseOps-Gym44.2%measured
AA Harvey LAB v1.093.2%estimated ± 1.2 pp, medium confidence
AA Tau3 Banking48.0%measured
AA Terminal-Bench 2.179.8%measured
ApprenticeBench4.5%estimated ± 6.0 pp, low confidence
ARC-AGI-188.1%estimated ± 2.5 pp, high confidence
ARC-AGI-252.6%estimated ± 9.7 pp, medium confidence
BabaIsBench21.0%measured
CursorBench 3.135.6%estimated ± 3.4 pp, low confidence
CursorBench 3.264.9%estimated ± 3.3 pp, medium confidence
EBR-bench43.4%estimated ± 11.3 pp, low confidence
Terminal-Bench 3.04.9%estimated ± 6.9 pp, low confidence
FrontierCode 1.1 Main36.6%estimated ± 4.6 pp, high confidence
FrontierSWE v218.5%estimated ± 8.3 pp, medium confidence
Furniture Assembly46.5%estimated ± 12.4 pp, low confidence
MaxxBench57.6%estimated ± 5.7 pp, medium confidence
MazeBench0.0%measured
Mystery Game Puzzles25.9%estimated ± 12.4 pp, low confidence
NYT Connections54.5%measured
OfficeQA Pro50.3%estimated ± 6.8 pp, low confidence
OSWorld-Verified84.3%measured
ParseBench-169.0%measured
Terminal-Bench 4.19.6%estimated ± 5.8 pp, medium confidence
Terminal-Bench Hard47.2%estimated ± 3.9 pp, high confidence
Vals SWE-bench86.0%measured
Terminal-Bench 2.1 (Vals AI)58.4%measured
VulcanBench v382.6%measured
CWE-Bench-AA26.7%measured
AA Cyber Index12.7%measured
DeepsecBench-AA11.5%measured
Harvey LAB-AA v1.10.0%estimated ± 3.5 pp, low confidence
HealthBench Professional (length-adjusted, Artificial Analysis)51.9%estimated ± 4.3 pp, low confidence
HealthBench Professional (raw, Artificial Analysis)43.4%estimated ± 0.8 pp, low confidence
AA ITBench44.2%estimated ± 3.2 pp, high confidence
Bug Hunt Bench15.0%measured
Chartography (tools)81.1%estimated ± 4.1 pp, low confidence
CursorBench 4.027.2%estimated ± 2.0 pp, medium confidence
CWE-bench v157.4%estimated ± 5.1 pp, medium confidence
DeepSWE42.2%measured
DRACO86.9%estimated ± 0.7 pp, low confidence
FrontierCode 2.021.8%estimated ± 6.7 pp, medium confidence
FrontierPhysics4.7%estimated ± 4.7 pp, high confidence
HLE w/o tools30.8%measured
OSWorld 2.019.2%estimated ± 3.6 pp, high confidence
OSWorld 2.122.5%estimated ± 3.6 pp, medium confidence
OSWorld 2.1 Offline47.5%estimated ± 1.9 pp, low confidence
PostTrainBench v1.128.9%estimated ± 3.7 pp, high confidence
SpireClimb5.8%estimated ± 7.2 pp, low confidence
StealthMark51.5%measured
SWE-bench Multimodal56.5%estimated ± 5.6 pp, low confidence
Terminal-Bench 4.035.7%estimated ± 6.1 pp, low confidence
Toolathlon-Verified72.2%estimated ± 2.3 pp, medium confidence
VulcanBench Frontier v481.0%estimated ± 4.0 pp, low confidence
AA AutomationBench48.2%measured
GDP.pdf16.4%measured
AA Terminal-Bench 4.05.6%measured
ARC-AGI-30.7%estimated ± 7.2 pp, low confidence
ARC-AGI-3 Provider Adapter2.7%estimated ± 7.3 pp, low confidence
cursorBench4144.8%estimated ± 3.7 pp, medium confidence
CWE-bench36.6%estimated ± 6.2 pp, low confidence
MKA33.1%estimated ± 4.6 pp, medium confidence
SkillsBench official leaderboard (BenchFlow)49.4%estimated ± 6.1 pp, medium confidence
BinaryAudit35.8%estimated ± 12.1 pp, low confidence
OTelBench22.5%estimated ± 6.6 pp, medium confidence
Vibe Code Bench48.9%estimated ± 10.0 pp, low confidence
FACTS Multimodal31.7%estimated ± 4.0 pp, medium confidence
Hermes Bench70.6%estimated ± 1.5 pp, medium confidence
Hermes Index37.4%estimated ± 2.7 pp, medium confidence
ParseBench-274.5%measured
CharXiv90.2%measured
CharXiv w/o tools83.7%measured
ExploitBench52.3%estimated ± 12.6 pp, low confidence
ExploitGym11.7%estimated ± 2.7 pp, high confidence
AI2D_TEST93.5%estimated ± 0.9 pp, low confidence
C-Eval93.4%estimated ± 0.9 pp, medium confidence
Claw-Eval62.8%estimated ± 4.5 pp, medium confidence
CountBench97.3%estimated ± 1.3 pp, low confidence
DynaMath88.0%estimated ± 0.8 pp, low confidence
ERQA65.5%measured
MathVision90.0%measured
MCP Atlas83.1%estimated ± 6.0 pp, low confidence
MLVU (M-Avg)87.4%estimated ± 1.5 pp, high confidence
MMLU-Redux93.2%estimated ± 0.8 pp, low confidence
MMMU-Pro77.4%estimated ± 1.5 pp, high confidence
NL2Repo42.3%measured
OmniDocBench 1.591.1%measured
PostTrainBench v131.8%estimated ± 3.6 pp, high confidence
QwenClawBench93.2%estimated ± 6.0 pp, low confidence
RealWorldQA85.9%measured
ScreenSpot Pro77.0%estimated ± 4.8 pp, medium confidence
SimpleVQA72.0%estimated ± 7.1 pp, low confidence
SuperGPQA71.2%estimated ± 7.2 pp, medium confidence
Terminal-Bench 2.065.1%estimated ± 5.8 pp, medium confidence
Toolathlon55.9%estimated ± 6.3 pp, medium confidence
V*100.0%estimated ± 7.5 pp, medium confidence
Video-MME (w/o subtitle)84.0%estimated ± 1.7 pp, medium confidence
Video-MME (with subtitle)88.6%estimated ± 1.9 pp, high confidence
VideoMMMU87.3%estimated ± 1.5 pp, medium confidence
We-Math87.5%estimated ± 2.1 pp, low confidence
AA LiveCodeBench95.2%estimated ± 2.6 pp, medium confidence
HealthBench Hard0.4%estimated ± 5.1 pp, low confidence
LiveCodeBench Pro85.0%estimated ± 6.0 pp, low confidence
MedXpertQA (MM)76.3%estimated ± 4.2 pp, medium confidence
MedXpertQA (Text)51.5%estimated ± 4.5 pp, medium confidence
MMLU-Pro (Arcee)86.6%estimated ± 3.4 pp, medium confidence
React Native Evals81.2%estimated ± 3.2 pp, high confidence
ResearchClawBench20.9%estimated ± 1.9 pp, medium confidence
SWE-Rebench58.1%estimated ± 8.9 pp, low confidence
PostTrainBench27.0%estimated ± 3.7 pp, high confidence
DAYJOB7.3%estimated ± 3.2 pp, high confidence
DAYJOB Healthcare5.2%estimated ± 4.7 pp, high confidence
Finance Agent v255.2%estimated ± 2.4 pp, low confidence
BioMysteryBench (human-solvable)81.8%estimated ± 0.6 pp, low confidence
FrontierCode 1.1 Extended56.5%estimated ± 3.0 pp, low confidence
HealthBench (raw)42.9%estimated ± 6.0 pp, low confidence
HealthBench (length-adjusted)51.3%estimated ± 3.4 pp, low confidence
HealthBench Professional40.9%estimated ± 4.2 pp, low confidence
HealthBench Professional (raw)53.8%estimated ± 4.5 pp, medium confidence
HLE-Verified21.9%estimated ± 2.0 pp, low confidence
LABBench279.9%estimated ± 1.8 pp, low confidence
ScarfBench60.8%estimated ± 3.8 pp, low confidence
BlenderWorld (text-only)32.2%estimated ± 4.3 pp, low confidence
Cookie-Bench48.4%estimated ± 4.8 pp, low confidence
STEMBench2.3%estimated ± 0.2 pp, low confidence
SimpleQA14.4%estimated ± 4.3 pp, medium confidence
Agents' Last Exam42.9%measured
BabyVision w/ Python85.6%measured
OpenHarmony Bench v1.051.0%estimated ± 2.4 pp, high confidence
OpenHarmony Bench v1.245.7%estimated ± 11.0 pp, low confidence
SEC-Bench Pro61.2%estimated ± 9.3 pp, medium confidence
ZeroBench w/ Python38.0%estimated ± 7.0 pp, low confidence
BabyVision65.7%measured
MMVU79.1%estimated ± 3.3 pp, medium confidence
PerceptionBench55.9%estimated ± 2.5 pp, low confidence
ZeroBench15.7%estimated ± 3.6 pp, medium confidence
MRCRv285.9%estimated ± 11.3 pp, low confidence
LVBench77.3%estimated ± 8.2 pp, low confidence
CC-OCR81.7%estimated ± 3.7 pp, medium confidence
MMLongBench-Doc61.5%estimated ± 2.6 pp, low confidence
RefCOCO (avg)90.7%estimated ± 3.1 pp, low confidence
OCRBench V270.9%estimated ± 5.4 pp, low confidence
SWE-Marathon39.3%estimated ± 2.1 pp, low confidence
BioBasicsBench19.8%estimated ± 2.2 pp, low confidence
LifeVal75.2%estimated ± 3.8 pp, medium confidence
MMMU-Pro w/ Python79.4%estimated ± 2.0 pp, high confidence
ExploitBench Internal Port5.5%estimated ± 6.8 pp, low confidence
MathVision w/ Python94.6%measured
SWE-Atlas Codebase QnA30.4%estimated ± 4.8 pp, medium confidence