Other · model
Ultramarine 1 Flash benchmark scores
As of 2026-10-29, Ultramarine 1 Flash (Other) has measured scores on 4 benchmarks and estimated scores on 31 more.
| Benchmark | Score | Source |
|---|---|---|
| AA-SciCode | 55.5% | estimated ± 3.5 pp, medium confidence |
| GDPval-AA | 56.0% | estimated ± 6.2 pp, medium confidence |
| Terminal-Bench 2.1 | 83.2% | measured |
| BrowseComp | 95.3% | estimated ± 6.2 pp, low confidence |
| AA Coding Index | 73.5% | estimated ± 2.5 pp, high confidence |
| APEX-Agents-AA | 42.0% | estimated ± 3.6 pp, low confidence |
| DeepSearchQA | 90.6% | estimated ± 10.9 pp, low confidence |
| SWE-bench Verified | 85.5% | estimated ± 4.6 pp, high confidence |
| AutomationBench | 50.2% | measured |
| CyberGym | 87.1% | estimated ± 4.6 pp, medium confidence |
| JobBench | 62.9% | measured |
| SWE-bench Pro | 66.8% | estimated ± 7.5 pp, low confidence |
| WideResearch | 84.1% | estimated ± 1.2 pp, low confidence |
| Gert Labs | 100.0% | estimated ± 7.3 pp, low confidence |
| AA Tau3 Banking | 49.0% | estimated ± 3.8 pp, low confidence |
| AA Terminal-Bench 2.1 | 82.5% | estimated ± 6.8 pp, medium confidence |
| ApprenticeBench | 20.4% | estimated ± 5.6 pp, low confidence |
| CursorBench 3.2 | 65.7% | estimated ± 6.4 pp, low confidence |
| Terminal-Bench 3.0 | 16.3% | estimated ± 9.1 pp, low confidence |
| FrontierCode 1.1 Main | 40.3% | estimated ± 3.7 pp, low confidence |
| MazeBench | 0.0% | estimated ± 5.1 pp, medium confidence |
| OSWorld-Verified | 83.5% | estimated ± 8.4 pp, low confidence |
| Terminal-Bench 2.1 (Vals AI) | 70.2% | estimated ± 7.4 pp, medium confidence |
| CWE-Bench-AA | 59.9% | estimated ± 6.0 pp, low confidence |
| AA Cyber Index | 46.3% | estimated ± 9.4 pp, medium confidence |
| GDP.pdf | 19.9% | estimated ± 3.2 pp, medium confidence |
| AA Terminal-Bench 4.0 | 21.7% | estimated ± 11.6 pp, low confidence |
| Bug Hunt Bench | 23.4% | estimated ± 9.4 pp, low confidence |
| DeepSWE | 61.5% | measured |
| PostTrainBench v1.1 | 30.0% | estimated ± 4.5 pp, high confidence |
| Claw-Eval | 66.2% | estimated ± 5.6 pp, low confidence |
| MCP Atlas | 92.6% | estimated ± 7.7 pp, low confidence |
| NL2Repo | 56.7% | estimated ± 6.2 pp, medium confidence |
| Toolathlon | 84.7% | estimated ± 7.4 pp, low confidence |
| SWE-Atlas Codebase QnA | 59.2% | estimated ± 5.4 pp, low confidence |