Calibration
AA-SciCode → CursorBench 3.2
CursorBench 3.2 is estimated from AA-SciCode with a linear curve fitted on 17 models measured on both: y = 1.5964·x + -0.2613, R² = 0.56, cross-validated error 5.3 pp. It is used for 15 estimates.
| Estimated model | AA-SciCode | CursorBench 3.2 | Source |
|---|---|---|---|
| A.X K2 | 41.0% | 39.3% | estimated ± 5.3 pp, low confidence |
| DeepSeek V3 0324 | 39.0% | 36.1% | estimated ± 5.3 pp, low confidence |
| GPT-6.1 Sol | 54.2% | 60.4% | estimated ± 5.3 pp, medium confidence |
| GPT-6 Luna | 54.6% | 61.0% | estimated ± 5.3 pp, medium confidence |
| GPT-6 Sol | 57.6% | 65.8% | estimated ± 5.3 pp, medium confidence |
| Granite 4.2 3B | 25.3% | 14.3% | estimated ± 5.3 pp, low confidence |
| K-EXAONE 2.0 | 42.0% | 40.9% | estimated ± 5.3 pp, low confidence |
| Ling 3.0 Flash VL | 44.2% | 44.4% | estimated ± 5.3 pp, low confidence |
| Ling 3.0 Tiny | 24.2% | 12.5% | estimated ± 5.3 pp, low confidence |
| Ling 3.1 Flash | 54.1% | 60.2% | estimated ± 5.3 pp, medium confidence |
| Mercury 2.5 | 38.5% | 35.3% | estimated ± 5.3 pp, low confidence |
| Mistral Large 4 | 54.2% | 60.4% | estimated ± 5.3 pp, medium confidence |
| North Mini Code | 38.8% | 35.8% | estimated ± 5.3 pp, low confidence |
| Solar Pro 3 | 25.5% | 14.6% | estimated ± 5.3 pp, low confidence |
| Solar Pro 4 | 44.6% | 45.1% | estimated ± 5.3 pp, low confidence |