Calibration
VulcanBench Frontier v4 → DeepSWE
DeepSWE is estimated from VulcanBench Frontier v4 with a Hill curve fitted on 15 models measured on both: y = 0.0000 + (0.7887 − 0.0000)·x^6.00 / (0.59814^6.00 + x^6.00), R² = 0.76, cross-validated error 6.5 pp. It is used for 1 estimate.
| Estimated model | VulcanBench Frontier v4 | DeepSWE | Source |
|---|---|---|---|
| GPT-6.1 Terra | 90.0% | 72.6% | estimated ± 6.5 pp, medium confidence |
Fitted on
| Model | VulcanBench Frontier v4 | DeepSWE |
|---|---|---|
| Claude Fable 5.1 | 91.8% | 67.4% |
| Claude Fable 5.2 | 93.9% | 74.3% |
| Claude Opus 5.5 | 91.1% | 74.2% |
| Claude Sonnet 5.5 | 92.0% | 71.0% |
| DeepSeek V4.1 Flash 1017 | 88.6% | 77.0% |
| GPT-5.6 Luna | 84.2% | 67.2% |
| GPT-5.6 Sol | 87.2% | 72.7% |
| GPT-5.6 Terra | 89.1% | 69.6% |
| GPT-6.1 Sol | 88.8% | 71.9% |
| GPT-6 Astra | 89.3% | 74.1% |
| GPT-6 Luna | 71.9% | 66.6% |
| GPT-6 Sol | 86.8% | 68.8% |
| Grok 4.7 | 93.2% | 71.0% |
| Muse Wonder Full | 88.4% | 79.0% |
| Beam 1.0 501B-A23B | 64.7% | 43.3% |