benchgap
Other · model

Hy4 preview benchmark scores

As of 2026-11-21, Hy4 preview (Other) has measured scores on 32 benchmarks and estimated scores on 114 more.

BenchmarkScoreSource
AA-GPQA Diamond90.9%estimated ± 2.7 pp, high confidence
AA-HLE41.5%estimated ± 3.9 pp, high confidence
AA-SciCode6.9%estimated ± 3.0 pp, low confidence
Artificial Analysis Intelligence Index34.0%estimated ± 5.6 pp, medium confidence
CritPt16.9%measured
GDPval-AA41.0%estimated ± 4.6 pp, medium confidence
GPQA Diamond92.3%measured
HLE55.4%measured
LiveCodeBench v694.6%estimated ± 1.7 pp, medium confidence
AA-Omniscience Accuracy41.5%estimated ± 7.7 pp, medium confidence
SciCode49.7%estimated ± 5.9 pp, medium confidence
Terminal-Bench 2.185.4%measured
BrowseComp90.8%estimated ± 1.9 pp, medium confidence
HLE w/ tools55.4%measured
VITA-Bench49.6%estimated ± 9.9 pp, low confidence
τ²-bench results97.9%estimated ± 3.1 pp, high confidence
AA Coding Index74.4%estimated ± 2.5 pp, high confidence
AA-MMMU-Pro82.6%estimated ± 2.3 pp, high confidence
APEX-Agents37.1%measured
APEX-Agents-AA41.7%estimated ± 3.6 pp, low confidence
BioMysteryBench (human-difficult)47.2%estimated ± 6.8 pp, low confidence
DeepSearchQA91.0%estimated ± 7.3 pp, low confidence
SWE-bench Verified81.8%estimated ± 2.3 pp, high confidence
AutomationBench32.1%measured
BFCL v477.0%estimated ± 4.7 pp, low confidence
CyberGym78.4%measured
JobBench61.7%measured
SkillsBench62.9%measured
SWE-bench Pro65.7%measured
WideResearch83.9%measured
AA Agentic Index50.6%estimated ± 3.3 pp, medium confidence
MMLU-Pro85.7%estimated ± 2.5 pp, low confidence
FailureSensorIQ66.4%estimated ± 1.7 pp, high confidence
GPQA92.3%measured
MMLU88.1%estimated ± 4.2 pp, medium confidence
Vals GPQA Diamond91.4%estimated ± 2.1 pp, high confidence
Vals LiveCodeBench86.5%estimated ± 4.7 pp, medium confidence
Vals MMLU-Pro88.3%estimated ± 0.9 pp, high confidence
MMMLU90.5%estimated ± 0.7 pp, low confidence
Chess Puzzles42.6%estimated ± 10.8 pp, low confidence
AA MMLU-Pro87.2%estimated ± 1.2 pp, low confidence
Gert Labs67.4%estimated ± 5.5 pp, medium confidence
AA-AnalystAgent47.1%estimated ± 3.7 pp, medium confidence
AA EnterpriseOps-Gym46.7%estimated ± 1.2 pp, medium confidence
AA Harvey LAB v1.089.6%estimated ± 1.7 pp, medium confidence
AA Tau3 Banking39.4%estimated ± 3.3 pp, medium confidence
AA Terminal-Bench 2.180.1%estimated ± 2.5 pp, medium confidence
ApprenticeBench9.3%estimated ± 4.0 pp, high confidence
ARC-AGI-192.6%estimated ± 2.5 pp, high confidence
ARC-AGI-269.6%estimated ± 9.7 pp, medium confidence
CursorBench 3.158.3%estimated ± 5.6 pp, low confidence
CursorBench 3.264.2%estimated ± 3.6 pp, high confidence
EBR-bench35.2%estimated ± 11.3 pp, low confidence
Terminal-Bench 3.03.5%estimated ± 6.9 pp, low confidence
FrontierCode 1.1 Main46.7%estimated ± 3.5 pp, low confidence
FrontierSWE v225.4%estimated ± 8.3 pp, medium confidence
Furniture Assembly38.9%estimated ± 12.4 pp, low confidence
MaxxBench52.6%estimated ± 5.7 pp, medium confidence
MazeBench2.7%estimated ± 4.0 pp, high confidence
Mystery Game Puzzles30.2%estimated ± 13.3 pp, low confidence
NYT Connections68.2%measured
OfficeQA Pro66.2%measured
OSWorld-Verified78.6%estimated ± 4.7 pp, high confidence
ParseBench-171.9%estimated ± 5.4 pp, low confidence
Terminal-Bench 4.122.3%estimated ± 10.9 pp, low confidence
Terminal-Bench Hard46.1%estimated ± 3.9 pp, high confidence
Vals SWE-bench92.5%estimated ± 5.2 pp, medium confidence
Terminal-Bench 2.1 (Vals AI)55.1%measured
CWE-Bench-AA60.8%estimated ± 4.0 pp, medium confidence
AA Cyber Index32.8%estimated ± 9.4 pp, medium confidence
DeepsecBench-AA19.2%estimated ± 5.2 pp, low confidence
HealthBench Professional (length-adjusted, Artificial Analysis)57.9%estimated ± 4.4 pp, low confidence
HealthBench Professional (raw, Artificial Analysis)63.3%estimated ± 0.8 pp, low confidence
AA ITBench47.4%estimated ± 4.4 pp, high confidence
Bug Hunt Bench5.7%estimated ± 3.2 pp, low confidence
Chartography (tools)86.0%estimated ± 4.3 pp, low confidence
CursorBench 4.039.9%estimated ± 3.3 pp, high confidence
CWE-bench v153.0%measured
DeepSWE64.3%measured
DRACO77.2%measured
FrontierCode 2.023.4%estimated ± 6.7 pp, medium confidence
FrontierPhysics2.5%estimated ± 4.7 pp, high confidence
HLE w/o tools43.4%measured
OfficeQA77.4%estimated ± 0.2 pp, medium confidence
OSWorld 2.019.4%estimated ± 5.5 pp, low confidence
OSWorld 2.169.4%estimated ± 7.2 pp, low confidence
OSWorld 2.1 Offline65.7%estimated ± 5.5 pp, low confidence
PostTrainBench v1.129.3%estimated ± 4.4 pp, high confidence
ProgramBench17.5%measured
SpireClimb5.8%estimated ± 7.2 pp, low confidence
StealthMark43.9%measured
SWE Multilingual82.9%measured
SWE-bench Multimodal52.2%estimated ± 8.8 pp, low confidence
Toolathlon-Verified74.1%measured
VulcanBench Frontier v488.5%estimated ± 1.2 pp, low confidence
AA AutomationBench41.9%estimated ± 7.9 pp, low confidence
GDP.pdf19.0%estimated ± 4.1 pp, medium confidence
AA Terminal-Bench 4.017.7%estimated ± 11.0 pp, low confidence
ARC-AGI-35.1%estimated ± 7.2 pp, medium confidence
CWE-bench47.3%measured
MKA39.8%estimated ± 3.2 pp, high confidence
SkillsBench official leaderboard (BenchFlow)52.3%estimated ± 6.1 pp, medium confidence
BinaryAudit35.8%estimated ± 12.1 pp, low confidence
OTelBench22.5%estimated ± 6.6 pp, medium confidence
FACTS Multimodal38.9%estimated ± 4.0 pp, high confidence
Hermes Bench70.8%measured
Hermes Index41.0%measured
CharXiv91.7%estimated ± 5.1 pp, low confidence
ExploitBench33.9%estimated ± 12.6 pp, low confidence
ExploitGym16.4%estimated ± 5.5 pp, low confidence
C-Eval93.2%estimated ± 1.2 pp, medium confidence
Claw-Eval77.0%estimated ± 4.5 pp, medium confidence
DeepPlanning51.8%estimated ± 13.1 pp, low confidence
INCLUDE87.2%estimated ± 2.0 pp, medium confidence
MCP Atlas83.7%measured
MMLU-ProX88.0%estimated ± 1.3 pp, medium confidence
MMLU-Redux95.1%estimated ± 1.7 pp, high confidence
NL2Repo58.9%measured
PostTrainBench v131.2%estimated ± 4.6 pp, high confidence
QwenClawBench69.0%estimated ± 2.6 pp, medium confidence
SuperGPQA94.3%estimated ± 7.2 pp, low confidence
Terminal-Bench 2.062.6%estimated ± 5.8 pp, medium confidence
Toolathlon52.5%estimated ± 5.2 pp, low confidence
HealthBench Hard31.2%estimated ± 5.1 pp, low confidence
LiveCodeBench Pro88.0%estimated ± 6.0 pp, low confidence
MedXpertQA (Text)61.3%estimated ± 4.5 pp, medium confidence
MMLU-Pro (Arcee)87.6%estimated ± 3.4 pp, low confidence
ResearchClawBench19.7%estimated ± 1.8 pp, medium confidence
PostTrainBench27.0%estimated ± 3.7 pp, high confidence
DAYJOB2.9%estimated ± 3.2 pp, high confidence
DAYJOB Healthcare1.6%estimated ± 4.7 pp, high confidence
BioMysteryBench (human-solvable)88.4%estimated ± 0.6 pp, medium confidence
FrontierCode 1.1 Extended57.8%estimated ± 3.0 pp, medium confidence
HealthBench Professional52.6%estimated ± 4.2 pp, medium confidence
ScarfBench64.3%estimated ± 3.8 pp, medium confidence
BlenderWorld (text-only)61.0%estimated ± 4.3 pp, low confidence
Cookie-Bench75.2%estimated ± 4.8 pp, low confidence
STEMBench2.6%measured
SimpleQA38.6%estimated ± 13.4 pp, low confidence
Agents' Last Exam22.8%measured
OpenHarmony Bench v1.059.1%estimated ± 2.7 pp, high confidence
SEC-Bench Pro51.6%estimated ± 9.3 pp, medium confidence
SWE-Marathon31.9%measured
BioBasicsBench29.8%estimated ± 2.2 pp, low confidence
ExploitBench Internal Port0.0%estimated ± 6.8 pp, low confidence
SWE-Atlas Codebase QnA58.6%estimated ± 4.8 pp, high confidence