Calibration
LiveCodeBench v6 → AA-SciCode
AA-SciCode is estimated from LiveCodeBench v6 with a Hill curve fitted on 10 models measured on both: y = 0.0000 + (0.5933 − 0.0000)·x^6.00 / (0.71797^6.00 + x^6.00), R² = 0.94, cross-validated error 3.7 pp. It is used for 12 estimates.
| Estimated model | LiveCodeBench v6 | AA-SciCode | Source |
|---|---|---|---|
| BTL-3 | 88.1% | 45.9% | estimated ± 3.7 pp, high confidence |
| BTL-4 | 66.1% | 22.5% | estimated ± 3.7 pp, high confidence |
| LLaDA2.2-mini | 28.1% | 0.2% | estimated ± 3.7 pp, medium confidence |
| MAI-Thinking-1 | 87.7% | 45.6% | estimated ± 3.7 pp, high confidence |
| Mellum2-12B-A2.5B-Instruct | 37.2% | 1.1% | estimated ± 3.7 pp, medium confidence |
| Mellum2-12B-A2.5B-Thinking | 69.9% | 27.3% | estimated ± 3.7 pp, high confidence |
| MiniCPM5-1B | 33.5% | 0.6% | estimated ± 3.7 pp, medium confidence |
| Qwen3.5 397B | 83.6% | 42.3% | estimated ± 3.7 pp, high confidence |
| Solar Open 2 | 92.4% | 48.6% | estimated ± 3.7 pp, medium confidence |
| Ternary Bonsai 2 27B | 90.1% | 47.2% | estimated ± 3.7 pp, high confidence |
| ZAYA1-74B-Preview | 65.7% | 21.9% | estimated ± 3.7 pp, high confidence |
| ZAYA1-8B | 65.8% | 22.1% | estimated ± 3.7 pp, high confidence |