Calibration
AA-SciCode → AA Coding Index
AA Coding Index is estimated from AA-SciCode with a offset logistic curve fitted on 78 models measured on both: y = 0.0597 + (0.8474 − 0.0597) / (1 + exp(−13.45·(x − 0.4292))), R² = 0.89, cross-validated error 6.4 pp. It is used for 10 estimates.
| Estimated model | AA-SciCode | AA Coding Index | Source |
|---|---|---|---|
| A.X K2 | 41.0% | 40.3% | estimated ± 6.4 pp, medium confidence |
| DeepSeek V3 0324 | 39.0% | 35.2% | estimated ± 6.4 pp, medium confidence |
| Granite 4.2 3B | 25.3% | 12.7% | estimated ± 6.4 pp, medium confidence |
| Ling 3.0 Flash VL | 44.2% | 48.7% | estimated ± 6.4 pp, medium confidence |
| Ling 3.0 Tiny | 24.2% | 11.8% | estimated ± 6.4 pp, medium confidence |
| Ling 3.1 Flash | 54.1% | 70.4% | estimated ± 6.4 pp, medium confidence |
| Mercury 2.5 | 38.5% | 34.0% | estimated ± 6.4 pp, medium confidence |
| Mistral Large 4 | 54.2% | 70.6% | estimated ± 6.4 pp, medium confidence |
| North Mini Code | 38.8% | 34.7% | estimated ± 6.4 pp, medium confidence |
| Solar Pro 3 | 25.5% | 12.9% | estimated ± 6.4 pp, medium confidence |