Calibration
BenchCAD Vision2Code (tools) → BenchCAD Vision2Code (no tools)
BenchCAD Vision2Code (no tools) is estimated from BenchCAD Vision2Code (tools) with a linear curve fitted on 5 models measured on both: y = 2.6989·x + -0.0184, R² = 0.96, cross-validated error 0.1 pp. It is used for 1 estimate.
| Estimated model | BenchCAD Vision2Code (tools) | BenchCAD Vision2Code (no tools) | Source |
|---|---|---|---|
| GPT-6 Astra | 1.0% | 0.8% | estimated ± 0.1 pp, medium confidence |
Fitted on
| Model | BenchCAD Vision2Code (tools) | BenchCAD Vision2Code (no tools) |
|---|---|---|
| Claude Fable 5.1 | 0.8% | 0.4% |
| Claude Opus 5 | 0.8% | 0.4% |
| Claude Opus 5.5 | 1.0% | 0.7% |
| Claude Sonnet 5.5 | 1.0% | 0.7% |
| Kimi K3.1 Pro | 0.9% | 0.7% |