Calibration
AA-SciCode → CursorBench 4.0
CursorBench 4.0 is estimated from AA-SciCode with a Hill curve fitted on 12 models measured on both: y = 0.0000 + (0.7527 − 0.0000)·x^6.00 / (0.54360^6.00 + x^6.00), R² = 0.82, cross-validated error 3.5 pp. It is used for 22 estimates.
| Estimated model | AA-SciCode | CursorBench 4.0 | Source |
|---|---|---|---|
| A.X K2 | 41.0% | 11.7% | estimated ± 3.5 pp, medium confidence |
| Claude Haiku 5.5 | 55.0% | 39.0% | estimated ± 3.5 pp, high confidence |
| DeepSeek V3 0324 | 39.0% | 9.0% | estimated ± 3.5 pp, medium confidence |
| DeepSeek V4.1 Flash | 51.9% | 32.4% | estimated ± 3.5 pp, medium confidence |
| GPT-6.1 Sol | 54.2% | 37.3% | estimated ± 3.5 pp, high confidence |
| GPT-6 Luna | 54.6% | 38.1% | estimated ± 3.5 pp, high confidence |
| GPT-6 Sol | 57.6% | 44.1% | estimated ± 3.5 pp, high confidence |
| Granite 4.2 30B | 37.8% | 7.6% | estimated ± 3.5 pp, medium confidence |
| Granite 4.2 3B | 25.3% | 0.8% | estimated ± 3.5 pp, medium confidence |
| K-EXAONE 2.0 | 42.0% | 13.2% | estimated ± 3.5 pp, medium confidence |
| Ling 3.0 Flash VL | 44.2% | 16.9% | estimated ± 3.5 pp, medium confidence |
| Ling 3.0 Tiny | 24.2% | 0.6% | estimated ± 3.5 pp, medium confidence |
| Ling 3.1 Flash | 54.1% | 37.1% | estimated ± 3.5 pp, high confidence |
| Mercury 2.5 | 38.5% | 8.4% | estimated ± 3.5 pp, medium confidence |
| MiMo-V2.6-Flash | 51.3% | 31.2% | estimated ± 3.5 pp, medium confidence |
| MiMo-V2.6-Pro | 60.9% | 50.0% | estimated ± 3.5 pp, high confidence |
| MiniCPM5-2B | 26.3% | 1.0% | estimated ± 3.5 pp, medium confidence |
| Mistral Large 4 | 54.2% | 37.3% | estimated ± 3.5 pp, high confidence |
| North Mini Code | 38.8% | 8.8% | estimated ± 3.5 pp, medium confidence |
| Solar Pro 3 | 25.5% | 0.8% | estimated ± 3.5 pp, medium confidence |
| Solar Pro 4 | 44.6% | 17.6% | estimated ± 3.5 pp, medium confidence |
| Step 5 Preview | 58.9% | 46.5% | estimated ± 3.5 pp, high confidence |