Calibration
LiveCodeBench v6 → AA Coding Index
AA Coding Index is estimated from LiveCodeBench v6 with a Hill curve fitted on 9 models measured on both: y = 0.0000 + (1.2000 − 0.0000)·x^6.00 / (0.89402^6.00 + x^6.00), R² = 0.93, cross-validated error 6.4 pp. It is used for 9 estimates.
| Estimated model | LiveCodeBench v6 | AA Coding Index | Source |
|---|---|---|---|
| Agents-A1-4B | 59.6% | 9.7% | estimated ± 6.4 pp, medium confidence |
| BTL-3 | 88.1% | 57.4% | estimated ± 6.4 pp, medium confidence |
| LLaDA2.2-mini | 28.1% | 0.1% | estimated ± 6.4 pp, low confidence |
| Mellum2-12B-A2.5B-Instruct | 37.2% | 0.6% | estimated ± 6.4 pp, low confidence |
| Mellum2-12B-A2.5B-Thinking | 69.9% | 22.3% | estimated ± 6.4 pp, medium confidence |
| MiniCPM5-1B | 33.5% | 0.3% | estimated ± 6.4 pp, low confidence |
| Sakana Fugu | 92.9% | 66.9% | estimated ± 6.4 pp, low confidence |
| Sakana Fugu-Ultra | 93.2% | 67.5% | estimated ± 6.4 pp, low confidence |
| ZAYA1-8B | 65.8% | 16.5% | estimated ± 6.4 pp, medium confidence |