benchgap
Calibration

AA AutomationBench → CWE-Bench-AA

CWE-Bench-AA is estimated from AA AutomationBench with a Hill curve fitted on 11 models measured on both: y = 0.0000 + (0.6745 − 0.0000)·x^6.00 / (0.44399^6.00 + x^6.00), R² = 0.63, cross-validated error 9.0 pp. It is used for 3 estimates.

Estimated modelAA AutomationBenchCWE-Bench-AASource
Qwen4 Max72.6%64.1%estimated ± 9.0 pp, medium confidence
Muse Ember68.1%62.6%estimated ± 9.0 pp, medium confidence
Muse Wonder Full45.4%36.0%estimated ± 9.0 pp, low confidence

Fitted on

ModelAA AutomationBenchCWE-Bench-AA
Claude Opus 5.569.5%58.3%
GLM-5.362.2%55.8%
GPT-6.2 Astra77.2%65.0%
GPT-6.2 Sol71.3%61.7%
GPT-6 Luna53.2%56.7%
Grok 4.765.6%68.3%
Kimi K3.1 Pro73.3%60.8%
MiMo-V2.6-Pro58.6%63.3%
Mistral Large 457.8%58.3%
Qwen3.8-27B48.2%26.7%
Beam 1.0 501B-A23B48.5%49.2%