Calibration
ITBench SRE → τ³-Bench Banking
τ³-Bench Banking is estimated from ITBench SRE with a linear curve fitted on 8 models measured on both: y = -0.2683·x + 0.5929, R² = 0.34, cross-validated error 2.6 pp. It is used for 10 estimates.
| Estimated model | ITBench SRE | τ³-Bench Banking | Source |
|---|---|---|---|
| Claude Opus 5.5 (max with fallback) | 38.2% | 49.0% | estimated ± 2.6 pp, medium confidence |
| GPT-6 Sol (max) | 49.4% | 46.0% | estimated ± 2.6 pp, medium confidence |
| Step 5 Preview | 55.6% | 44.4% | estimated ± 2.6 pp, low confidence |
| DeepSeek V4.1 Flash (max) | 46.9% | 46.7% | estimated ± 2.6 pp, medium confidence |
| Grok 4.7 (xhigh) | 42.1% | 48.0% | estimated ± 2.6 pp, medium confidence |
| GPT-5.6 Sol (max) | 56.2% | 44.2% | estimated ± 2.6 pp, low confidence |
| GPT-5.5 (xhigh) | 45.8% | 47.0% | estimated ± 2.6 pp, medium confidence |
| GPT-5.6 Terra (max) | 51.0% | 45.6% | estimated ± 2.6 pp, medium confidence |
| GLM-5.2 (max) | 42.7% | 47.8% | estimated ± 2.6 pp, medium confidence |
| Claude Opus 4.7 (max) | 46.7% | 46.8% | estimated ± 2.6 pp, medium confidence |