Calibration
SciCode → AA-SciCode
AA-SciCode is estimated from SciCode with a linear curve fitted on 20 models measured on both: y = 0.9231·x + 0.0332, R² = 0.89, cross-validated error 3.1 pp. It is used for 7 estimates.
| Estimated model | SciCode | AA-SciCode | Source |
|---|---|---|---|
| Agents-A1-4B | 29.6% | 30.6% | estimated ± 3.1 pp, high confidence |
| Kimi K2.5 | 48.7% | 48.3% | estimated ± 3.1 pp, high confidence |
| Ling 2.6 Flash | 27.0% | 28.2% | estimated ± 3.1 pp, high confidence |
| Nemotron 3 Nano Omni 30B A3B | 32.0% | 32.9% | estimated ± 3.1 pp, high confidence |
| Beam | 49.7% | 49.2% | estimated ± 3.1 pp, high confidence |
| Sakana Fugu | 60.1% | 58.8% | estimated ± 3.1 pp, medium confidence |
| Sakana Fugu-Ultra | 58.7% | 57.5% | estimated ± 3.1 pp, high confidence |