benchgap
Calibration

Chartography (no tools) → Chartography (tools)

Chartography (tools) is estimated from Chartography (no tools) with a linear curve fitted on 6 models measured on both: y = 0.2301·x + 0.7491, R² = 0.85, cross-validated error 2.3 pp. It is used for 5 estimates.

Estimated modelChartography (no tools)Chartography (tools)Source
Gemini 4 Carbon72.5%91.6%estimated ± 2.3 pp, medium confidence
Gemini 4 Argon71.6%91.4%estimated ± 2.3 pp, medium confidence
Gemini 4 Hydrogen66.6%90.2%estimated ± 2.3 pp, medium confidence
Qwen4 Max54.2%87.4%estimated ± 2.3 pp, medium confidence
Claude Haiku 5.546.4%85.6%estimated ± 2.3 pp, medium confidence

Fitted on

ModelChartography (no tools)Chartography (tools)
Claude Fable 5.142.6%86.2%
Claude Fable 5.273.4%91.6%
Claude Opus 529.6%83.0%
Claude Opus 5.564.4%89.0%
Claude Sonnet 5.561.6%90.2%
Inkling 2 Small36.7%80.4%