benchgap
Calibration

BenchCAD Vision2Code (tools) → BenchCAD Vision2Code (no tools)

BenchCAD Vision2Code (no tools) is estimated from BenchCAD Vision2Code (tools) with a linear curve fitted on 5 models measured on both: y = 2.6989·x + -0.0184, R² = 0.96, cross-validated error 0.1 pp. It is used for 1 estimate.

Estimated modelBenchCAD Vision2Code (tools)BenchCAD Vision2Code (no tools)Source
GPT-6 Astra1.0%0.8%estimated ± 0.1 pp, medium confidence

Fitted on

ModelBenchCAD Vision2Code (tools)BenchCAD Vision2Code (no tools)
Claude Fable 5.10.8%0.4%
Claude Opus 50.8%0.4%
Claude Opus 5.51.0%0.7%
Claude Sonnet 5.51.0%0.7%
Kimi K3.1 Pro0.9%0.7%