Calibration
SWE Multilingual → NOVA-63
NOVA-63 is estimated from SWE Multilingual with a Michaelis–Menten curve fitted on 6 models measured on both: y = 1.3204·x / (0.98315 + x), R² = 0.39, cross-validated error 1.6 pp. It is used for 38 estimates.
| Estimated model | SWE Multilingual | NOVA-63 | Source |
|---|---|---|---|
| Claude Fable 5.1 | 89.1% | 62.8% | estimated ± 1.6 pp, low confidence |
| Claude Mythos 5 | 92.2% | 63.9% | estimated ± 1.6 pp, low confidence |
| Claude Opus 4.8 | 84.4% | 61.0% | estimated ± 1.6 pp, low confidence |
| Claude Opus 5 | 89.5% | 62.9% | estimated ± 1.6 pp, low confidence |
| Claude Opus 5.5 | 93.9% | 64.5% | estimated ± 1.6 pp, low confidence |
| Claude Sonnet 5 | 78.3% | 58.5% | estimated ± 1.6 pp, low confidence |
| Claude Sonnet 5.5 | 90.3% | 63.2% | estimated ± 1.6 pp, low confidence |
| Composer 2 | 73.7% | 56.6% | estimated ± 1.6 pp, low confidence |
| Composer 2.5 | 79.8% | 59.2% | estimated ± 1.6 pp, low confidence |
| DeepSeek V4 Flash 0731 | 73.3% | 56.4% | estimated ± 1.6 pp, low confidence |
| DeepSeek V4 Pro 0813 | 76.2% | 57.7% | estimated ± 1.6 pp, low confidence |
| dots3-note Preview | 75.7% | 57.4% | estimated ± 1.6 pp, low confidence |
| Granite 4.2 30B | 41.9% | 39.5% | estimated ± 1.6 pp, low confidence |
| Granite 4.2 8B | 30.8% | 31.5% | estimated ± 1.6 pp, low confidence |
| Grok 4.5 | 78.0% | 58.4% | estimated ± 1.6 pp, low confidence |
| Hy4 preview | 82.9% | 60.4% | estimated ± 1.6 pp, low confidence |
| Kimi K2.6 | 76.7% | 57.9% | estimated ± 1.6 pp, low confidence |
| Laguna M.1 | 63.1% | 51.6% | estimated ± 1.6 pp, low confidence |
| Laguna S 2.1 | 78.5% | 58.6% | estimated ± 1.6 pp, low confidence |
| Laguna XS.2 | 57.7% | 48.8% | estimated ± 1.6 pp, low confidence |
| Laguna XS 2.1 | 63.1% | 51.6% | estimated ± 1.6 pp, low confidence |
| Ling 3.0 Flash | 72.4% | 56.0% | estimated ± 1.6 pp, low confidence |
| LLaDA2.2-flash | 25.0% | 26.8% | estimated ± 1.6 pp, low confidence |
| LongCat-Flash-Lite-Sparse | 59.3% | 49.7% | estimated ± 1.6 pp, low confidence |
| MiniMax M2.7 | 76.5% | 57.8% | estimated ± 1.6 pp, low confidence |
| Nemotron 3.5 Lightning 30B A3B NVFP4 | 36.5% | 35.7% | estimated ± 1.6 pp, low confidence |
| Ornith-1.0-35B | 69.3% | 54.6% | estimated ± 1.6 pp, low confidence |
| Ornith-1.0-397B | 78.9% | 58.8% | estimated ± 1.6 pp, low confidence |
| Ornith-1.0-9B | 52.0% | 45.7% | estimated ± 1.6 pp, low confidence |
| Ornith-1.5-35B-A3B | 71.4% | 55.5% | estimated ± 1.6 pp, low confidence |
| Ornith-1.5-397B | 79.6% | 59.1% | estimated ± 1.6 pp, low confidence |
| Ornith-1.5-9B | 54.4% | 47.0% | estimated ± 1.6 pp, low confidence |
| Qwen3.6-27B | 71.3% | 55.5% | estimated ± 1.6 pp, low confidence |
| Qwen3.6-35B-A3B | 67.2% | 53.6% | estimated ± 1.6 pp, low confidence |
| Qwen3.8-Flash-Next | 81.0% | 59.6% | estimated ± 1.6 pp, low confidence |
| Qwen3.8-Omni-Flash | 80.5% | 59.4% | estimated ± 1.6 pp, low confidence |
| Beam | 78.0% | 58.4% | estimated ± 1.6 pp, low confidence |
| SWE-1.7 | 77.8% | 58.3% | estimated ± 1.6 pp, low confidence |