benchgap
Other · model

Ornith-1.5-35B-A3B benchmark scores

As of 2026-11-22, Ornith-1.5-35B-A3B (Other) has measured scores on 17 benchmarks and estimated scores on 89 more.

BenchmarkScoreSource
AA-GPQA Diamond86.1%estimated ± 2.7 pp, high confidence
AA-HLE33.1%estimated ± 3.9 pp, high confidence
AA-SciCode48.8%estimated ± 3.1 pp, high confidence
Artificial Analysis Intelligence Index14.6%estimated ± 5.6 pp, medium confidence
CritPt6.7%estimated ± 4.0 pp, high confidence
GDPval-AA25.2%estimated ± 5.6 pp, low confidence
GPQA Diamond89.2%measured
HLE25.6%measured
LiveCodeBench v689.7%estimated ± 1.7 pp, high confidence
AA-Omniscience Accuracy23.4%estimated ± 7.7 pp, medium confidence
SciCode47.2%estimated ± 3.6 pp, high confidence
Terminal-Bench 2.167.8%measured
BrowseComp67.6%measured
HLE w/ tools33.4%measured
VITA-Bench38.6%estimated ± 6.7 pp, low confidence
τ²-bench results76.2%estimated ± 3.1 pp, medium confidence
AA Coding Index61.4%estimated ± 2.5 pp, medium confidence
APEX-Agents32.1%estimated ± 12.4 pp, low confidence
APEX-Agents-AA14.9%estimated ± 4.4 pp, high confidence
DeepSearchQA79.1%estimated ± 6.8 pp, low confidence
SWE-bench Verified79.0%measured
BFCL v472.9%estimated ± 4.7 pp, low confidence
CyberGym54.9%estimated ± 7.3 pp, medium confidence
JobBench0.0%estimated ± 5.1 pp, low confidence
SkillsBench39.3%estimated ± 4.1 pp, low confidence
SWE-bench Pro59.6%measured
WideResearch67.8%measured
AA Agentic Index17.5%estimated ± 3.3 pp, low confidence
MMLU-Pro84.9%estimated ± 2.5 pp, medium confidence
GPQA89.2%measured
MMLU88.1%estimated ± 4.2 pp, medium confidence
Vals GPQA Diamond88.4%estimated ± 2.1 pp, high confidence
Vals LiveCodeBench84.5%estimated ± 4.7 pp, medium confidence
Vals MMLU-Pro84.6%estimated ± 0.9 pp, medium confidence
MMMLU89.5%estimated ± 1.1 pp, medium confidence
AA MMLU-Pro86.9%estimated ± 1.2 pp, low confidence
Gert Labs26.1%estimated ± 5.3 pp, medium confidence
AA-AnalystAgent46.5%estimated ± 3.7 pp, low confidence
AA Harvey LAB v1.079.2%estimated ± 1.7 pp, low confidence
AA Tau3 Banking27.3%estimated ± 3.6 pp, low confidence
AA Terminal-Bench 2.181.7%estimated ± 2.6 pp, medium confidence
ApprenticeBench9.7%estimated ± 11.0 pp, low confidence
ARC-AGI-188.1%estimated ± 2.5 pp, high confidence
ARC-AGI-252.6%estimated ± 9.7 pp, medium confidence
CursorBench 3.154.4%estimated ± 5.6 pp, low confidence
CursorBench 3.257.6%estimated ± 3.6 pp, high confidence
Terminal-Bench 3.05.1%measured
FrontierCode 1.1 Main25.7%estimated ± 4.2 pp, low confidence
FrontierSWE v214.9%estimated ± 8.3 pp, medium confidence
MazeBench0.0%estimated ± 6.4 pp, low confidence
OSWorld-Verified73.2%estimated ± 3.8 pp, medium confidence
Terminal-Bench 4.10.0%estimated ± 4.8 pp, medium confidence
Vals SWE-bench79.9%estimated ± 5.2 pp, medium confidence
Terminal-Bench 2.1 (Vals AI)61.7%estimated ± 3.4 pp, medium confidence
CWE-Bench-AA19.3%estimated ± 5.2 pp, low confidence
AA Cyber Index23.9%estimated ± 12.8 pp, low confidence
DeepsecBench-AA5.3%estimated ± 9.0 pp, low confidence
HealthBench Professional (length-adjusted, Artificial Analysis)57.5%estimated ± 4.4 pp, low confidence
HealthBench Professional (raw, Artificial Analysis)55.3%estimated ± 2.6 pp, low confidence
AA ITBench40.1%estimated ± 4.1 pp, high confidence
Bug Hunt Bench13.8%estimated ± 7.2 pp, low confidence
CursorBench 4.035.3%estimated ± 3.3 pp, medium confidence
DeepSWE22.0%measured
DRACO66.6%estimated ± 2.9 pp, low confidence
FrontierCode 2.021.0%estimated ± 6.7 pp, medium confidence
HLE w/o tools25.6%measured
PostTrainBench v1.125.4%estimated ± 4.4 pp, high confidence
SWE Multilingual71.4%measured
Toolathlon-Verified48.7%measured
VulcanBench Frontier v439.7%estimated ± 5.0 pp, medium confidence
AA AutomationBench41.9%estimated ± 7.9 pp, low confidence
GDP.pdf0.0%estimated ± 6.1 pp, low confidence
AA Terminal-Bench 4.04.3%estimated ± 11.0 pp, low confidence
ARC-AGI-30.2%estimated ± 7.2 pp, low confidence
Vibe Code Bench34.1%estimated ± 8.4 pp, medium confidence
C-Eval91.9%estimated ± 0.9 pp, medium confidence
Claw-Eval72.5%measured
DeepPlanning45.3%estimated ± 13.1 pp, low confidence
INCLUDE84.0%estimated ± 2.0 pp, medium confidence
MCP Atlas70.2%measured
MCP-Tasks54.8%estimated ± 6.0 pp, low confidence
MMLU-ProX83.8%estimated ± 1.3 pp, high confidence
MMLU-Redux94.6%estimated ± 1.7 pp, high confidence
NL2Repo46.2%measured
PostTrainBench v124.8%estimated ± 4.6 pp, high confidence
QwenClawBench59.6%estimated ± 2.6 pp, high confidence
SuperGPQA66.3%estimated ± 7.2 pp, medium confidence
Toolathlon37.9%estimated ± 4.2 pp, high confidence
HealthBench Hard0.4%estimated ± 5.1 pp, low confidence
LiveCodeBench Pro83.4%estimated ± 6.0 pp, low confidence
MedXpertQA (Text)51.5%estimated ± 4.5 pp, medium confidence
MMLU-Pro (Arcee)86.6%estimated ± 3.4 pp, medium confidence
React Native Evals79.2%estimated ± 2.7 pp, medium confidence
ResearchClawBench19.5%estimated ± 1.3 pp, high confidence
SWE-Rebench61.7%estimated ± 3.6 pp, medium confidence
PostTrainBench23.5%estimated ± 3.7 pp, high confidence
FrontierCode 1.1 Extended55.9%estimated ± 3.0 pp, low confidence
HealthBench Professional35.4%estimated ± 4.2 pp, low confidence
SimpleQA38.6%estimated ± 13.4 pp, low confidence
LiveCodeBench78.7%estimated ± 14.6 pp, low confidence
Agents' Last Exam17.3%estimated ± 11.6 pp, low confidence
OpenHarmony Bench v1.054.0%estimated ± 2.7 pp, high confidence
BioBasicsBench19.8%estimated ± 2.2 pp, low confidence
LifeVal52.9%estimated ± 1.6 pp, medium confidence
PinchBench89.4%estimated ± 3.6 pp, medium confidence
SWE-Atlas Codebase QnA1.1%estimated ± 4.8 pp, medium confidence