benchgap
Anthropic · model

Claude 3.5 Sonnet benchmark scores

As of 2026-11-21, Claude 3.5 Sonnet (Anthropic) has measured scores on 6 benchmarks and estimated scores on 30 more.

BenchmarkScoreSource
AA-GPQA Diamond63.3%estimated ± 3.6 pp, high confidence
AA-HLE4.4%estimated ± 5.7 pp, medium confidence
AA-SciCode32.7%estimated ± 3.8 pp, high confidence
Artificial Analysis Intelligence Index7.5%estimated ± 5.8 pp, medium confidence
CritPt0.3%estimated ± 4.0 pp, high confidence
GPQA Diamond59.7%estimated ± 1.2 pp, high confidence
HLE4.7%estimated ± 6.5 pp, low confidence
AA-Omniscience Accuracy21.8%estimated ± 6.5 pp, medium confidence
SciCode32.1%estimated ± 3.6 pp, high confidence
AA Coding Index24.3%estimated ± 5.3 pp, medium confidence
SWE-bench Verified49.0%measured
SWE-bench Pro24.2%estimated ± 4.5 pp, high confidence
MMLU-Pro75.9%estimated ± 4.4 pp, high confidence
BBH82.7%estimated ± 4.5 pp, low confidence
FailureSensorIQ58.6%measured
GPQA59.4%measured
MMLU88.2%estimated ± 1.2 pp, high confidence
Vals GPQA Diamond59.3%measured
Vals LiveCodeBench49.6%measured
Vals MMLU-Pro78.4%measured
MMMLU78.0%estimated ± 1.7 pp, high confidence
AA MMLU-Pro83.3%estimated ± 1.1 pp, medium confidence
ARC-AGI-158.1%estimated ± 8.4 pp, medium confidence
CursorBench 3.259.1%estimated ± 7.2 pp, low confidence
Vals SWE-bench34.5%estimated ± 6.6 pp, low confidence
VulcanBench v376.3%estimated ± 5.6 pp, low confidence
HLE w/o tools24.5%estimated ± 4.9 pp, high confidence
BinaryAudit0.0%estimated ± 8.9 pp, low confidence
OTelBench3.1%estimated ± 7.7 pp, low confidence
Vibe Code Bench0.0%estimated ± 8.4 pp, low confidence
C-Eval85.3%estimated ± 1.2 pp, high confidence
MMLU-Redux87.3%estimated ± 1.7 pp, high confidence
SuperGPQA41.3%estimated ± 8.5 pp, low confidence
React Native Evals65.7%estimated ± 2.7 pp, low confidence
SimpleQA14.6%estimated ± 13.4 pp, low confidence
LiveCodeBench57.4%estimated ± 14.6 pp, low confidence