Calibration
AA-SciCode → FrontierCode 1.1 Main
FrontierCode 1.1 Main is estimated from AA-SciCode with a offset logistic curve fitted on 11 models measured on both: y = 0.0000 + (0.5079 − 0.0000) / (1 + exp(−75.55·(x − 0.5191))), R² = 0.76, cross-validated error 6.6 pp. It is used for 12 estimates.
| Estimated model | AA-SciCode | FrontierCode 1.1 Main | Source |
|---|---|---|---|
| GPT-6.1 Terra | 55.1% | 46.6% | estimated ± 6.6 pp, medium confidence |
| Ling 3.0 Flash VL | 44.2% | 0.1% | estimated ± 6.6 pp, low confidence |
| A.X K2 | 41.0% | 0.0% | estimated ± 6.6 pp, low confidence |
| Trinity-Large-Preview | 40.6% | 0.0% | estimated ± 6.6 pp, low confidence |
| Trinity-Large-Thinking | 40.6% | 0.0% | estimated ± 6.6 pp, low confidence |
| Nemotron 3.5 Bolt 74B-A9B | 39.7% | 0.0% | estimated ± 6.6 pp, low confidence |
| DeepSeek V3 0324 | 39.0% | 0.0% | estimated ± 6.6 pp, low confidence |
| GPT-OSS 20B | 38.9% | 0.0% | estimated ± 6.6 pp, low confidence |
| Mistral Small 4 | 38.8% | 0.0% | estimated ± 6.6 pp, low confidence |
| North Mini Code | 38.8% | 0.0% | estimated ± 6.6 pp, low confidence |
| Command A+ | 38.5% | 0.0% | estimated ± 6.6 pp, low confidence |
| Mercury 2.5 | 38.5% | 0.0% | estimated ± 6.6 pp, low confidence |
Fitted on
| Model | AA-SciCode | FrontierCode 1.1 Main |
|---|---|---|
| Claude Fable 5 | 61.0% | 53.5% |
| Claude Haiku 5.5 | 55.0% | 46.4% |
| Claude Opus 4.8 | 54.4% | 46.5% |
| Claude Opus 5 | 56.4% | 53.4% |
| Claude Opus 5.5 | 66.9% | 54.4% |
| Claude Sonnet 5 | 54.3% | 42.7% |
| Claude Sonnet 5.5 | 61.0% | 46.2% |
| Gemini 3.7 Flash | 57.2% | 43.6% |
| GPT-5.4 mini | 52.1% | 27.0% |
| GPT-5.5 | 55.8% | 43.0% |
| GPT-6 Astra | 56.5% | 53.3% |