Calibration
SWE-bench Multimodal → ParseBench
ParseBench is estimated from SWE-bench Multimodal with a Hill curve fitted on 5 models measured on both: y = 0.6030 + (0.9250 − 0.6030)·x^6.00 / (0.57927^6.00 + x^6.00), R² = 0.87, cross-validated error 4.8 pp. It is used for 1 estimate.
| Estimated model | SWE-bench Multimodal | ParseBench | Source |
|---|---|---|---|
| Claude Opus 5 | 59.4% | 77.6% | estimated ± 4.8 pp, medium confidence |
Fitted on
| Model | SWE-bench Multimodal | ParseBench |
|---|---|---|
| Claude Fable 5.1 | 54.7% | 78.1% |
| Claude Opus 4.8 | 38.4% | 62.3% |
| Claude Opus 5.5 | 61.4% | 78.9% |
| Claude Sonnet 5 | 28.1% | 61.2% |
| Claude Sonnet 5.5 | 54.3% | 69.3% |