benchgap
Other · model

Mercury 2.5 benchmark scores

As of 2026-11-22, Mercury 2.5 (Other) has measured scores on 13 benchmarks and estimated scores on 94 more.

BenchmarkScoreSource
AA-GPQA Diamond77.0%estimated ± 4.4 pp, high confidence
AA-HLE11.8%measured
AA-SciCode38.5%measured
Artificial Analysis Intelligence Index12.6%estimated ± 7.2 pp, medium confidence
CritPt0.0%measured
GDPval-AA0.0%measured
GPQA Diamond79.0%measured
HLE14.0%estimated ± 6.8 pp, medium confidence
IFBench77.0%measured
AA-LCR68.0%measured
LiveCodeBench v677.1%estimated ± 4.4 pp, high confidence
AA-Omniscience Accuracy22.0%measured
SciCode38.0%measured
Terminal-Bench 2.117.8%estimated ± 6.7 pp, low confidence
BrowseComp64.2%estimated ± 8.8 pp, low confidence
HLE w/ tools33.8%estimated ± 5.3 pp, low confidence
IFEval94.2%estimated ± 3.6 pp, high confidence
LongBench v257.5%estimated ± 5.5 pp, medium confidence
AA Coding Index33.8%estimated ± 6.1 pp, medium confidence
APEX-Agents0.0%estimated ± 9.4 pp, low confidence
APEX-Agents-AA9.0%estimated ± 9.3 pp, low confidence
BioMysteryBench (human-difficult)0.0%estimated ± 6.8 pp, low confidence
DeepSearchQA34.0%measured
SWE-bench Verified64.0%estimated ± 5.6 pp, medium confidence
AutomationBench49.9%estimated ± 9.1 pp, low confidence
BFCL v453.9%estimated ± 2.7 pp, high confidence
CyberGym0.0%estimated ± 6.0 pp, low confidence
JobBench0.0%estimated ± 8.7 pp, low confidence
SWE-bench Pro41.6%estimated ± 6.3 pp, medium confidence
τ³-bench results96.0%measured
WideResearch61.7%estimated ± 3.3 pp, medium confidence
AA Agentic Index0.0%estimated ± 3.1 pp, high confidence
MMLU-Pro84.4%estimated ± 5.0 pp, medium confidence
AA-IFBench79.1%estimated ± 7.4 pp, medium confidence
GPQA78.9%estimated ± 1.2 pp, high confidence
MMLU90.1%estimated ± 0.9 pp, high confidence
Vals GPQA Diamond78.7%estimated ± 2.1 pp, medium confidence
Vals LiveCodeBench83.4%estimated ± 0.8 pp, low confidence
Vals MMLU-Pro80.3%estimated ± 1.6 pp, medium confidence
MMMLU83.3%estimated ± 0.7 pp, medium confidence
AA MMLU-Pro84.4%estimated ± 0.8 pp, medium confidence
Gert Labs20.0%measured
AA Tau3 Banking0.0%estimated ± 4.1 pp, medium confidence
AA Terminal-Bench 2.179.2%estimated ± 2.5 pp, medium confidence
ApprenticeBench3.1%estimated ± 10.4 pp, low confidence
ARC-AGI-174.7%estimated ± 7.8 pp, low confidence
ARC-AGI-230.5%estimated ± 14.4 pp, low confidence
CursorBench 3.235.3%estimated ± 5.3 pp, low confidence
FrontierCode 1.1 Main0.0%estimated ± 6.6 pp, low confidence
FrontierSWE v24.4%estimated ± 14.2 pp, low confidence
MazeBench0.0%estimated ± 2.7 pp, medium confidence
OSWorld-Verified55.4%estimated ± 8.4 pp, low confidence
Terminal-Bench 4.14.1%estimated ± 14.9 pp, low confidence
Terminal-Bench Hard43.5%estimated ± 3.9 pp, medium confidence
Vals SWE-bench59.4%estimated ± 5.3 pp, low confidence
Terminal-Bench 2.1 (Vals AI)34.5%measured
CWE-Bench-AA31.1%estimated ± 8.1 pp, low confidence
AA Cyber Index3.2%estimated ± 12.4 pp, low confidence
DeepsecBench-AA5.8%estimated ± 8.2 pp, low confidence
HealthBench Professional (raw, Artificial Analysis)29.7%estimated ± 2.5 pp, low confidence
AA ITBench35.6%estimated ± 4.6 pp, low confidence
Bug Hunt Bench13.7%estimated ± 10.5 pp, low confidence
CursorBench 4.025.2%estimated ± 4.9 pp, medium confidence
CWE-bench v132.9%estimated ± 5.7 pp, low confidence
DeepSWE27.6%estimated ± 7.5 pp, low confidence
DRACO0.0%estimated ± 2.0 pp, low confidence
FrontierCode 2.026.1%estimated ± 9.3 pp, low confidence
HLE w/o tools12.9%estimated ± 4.3 pp, high confidence
OSWorld 2.11.2%estimated ± 3.6 pp, medium confidence
PostTrainBench v1.125.2%estimated ± 7.4 pp, low confidence
Toolathlon-Verified54.6%estimated ± 3.0 pp, high confidence
VulcanBench Frontier v468.0%estimated ± 6.8 pp, low confidence
AA AutomationBench37.8%estimated ± 9.9 pp, low confidence
GDP.pdf9.6%estimated ± 4.2 pp, medium confidence
AA Terminal-Bench 4.00.0%estimated ± 13.7 pp, low confidence
ARC-AGI-31.3%estimated ± 9.8 pp, low confidence
ARC-AGI-3 Provider Adapter2.7%estimated ± 7.3 pp, low confidence
CWE-bench25.6%estimated ± 6.2 pp, low confidence
MKA32.7%estimated ± 4.6 pp, medium confidence
Hermes Bench0.0%estimated ± 2.9 pp, medium confidence
Hermes Index14.4%estimated ± 5.7 pp, low confidence
ExploitGym0.0%estimated ± 8.3 pp, low confidence
C-Eval90.8%estimated ± 1.1 pp, low confidence
MCP Atlas56.5%estimated ± 10.7 pp, low confidence
MMLU-Redux93.8%estimated ± 0.8 pp, medium confidence
NL2Repo29.5%estimated ± 7.4 pp, medium confidence
QwenClawBench55.9%estimated ± 6.0 pp, low confidence
SuperGPQA65.0%estimated ± 9.3 pp, medium confidence
Terminal-Bench 2.044.8%estimated ± 5.8 pp, medium confidence
Toolathlon27.0%estimated ± 6.3 pp, low confidence
HealthBench Hard0.0%estimated ± 8.6 pp, low confidence
MMLU-Pro (Arcee)82.8%estimated ± 3.4 pp, medium confidence
React Native Evals71.8%estimated ± 4.7 pp, medium confidence
ResearchClawBench11.1%estimated ± 2.2 pp, low confidence
Finance Agent v255.1%estimated ± 2.4 pp, low confidence
BioMysteryBench (human-solvable)81.7%estimated ± 0.6 pp, low confidence
HLE-Verified17.2%estimated ± 7.1 pp, low confidence
ScarfBench36.0%estimated ± 14.0 pp, low confidence
STEMBench2.3%estimated ± 0.2 pp, low confidence
SimpleQA20.9%estimated ± 4.3 pp, medium confidence
Agents' Last Exam29.6%estimated ± 9.7 pp, low confidence
OpenHarmony Bench v1.051.0%estimated ± 2.4 pp, medium confidence
MRCRv264.2%estimated ± 11.3 pp, low confidence
MMLongBench-Doc60.8%estimated ± 2.6 pp, low confidence
BioBasicsBench34.1%estimated ± 3.8 pp, low confidence
LifeVal0.7%estimated ± 5.0 pp, low confidence
PinchBench95.1%estimated ± 5.5 pp, low confidence