Calibration
GDP.pdf → AutomationBench
AutomationBench is estimated from GDP.pdf with a linear curve fitted on 13 models measured on both: y = -0.7118·x + 0.5879, R² = 0.32, cross-validated error 7.6 pp. It is used for 6 estimates.
| Estimated model | GDP.pdf | AutomationBench | Source |
|---|---|---|---|
| Claude Haiku 5.5 | 20.8% | 44.0% | estimated ± 7.6 pp, low confidence |
| Claude Sonnet 5.5 | 25.8% | 40.4% | estimated ± 7.6 pp, low confidence |
| Gemini 3.8 Flash | 21.0% | 43.8% | estimated ± 7.6 pp, low confidence |
| GPT-6 Luna | 22.8% | 42.6% | estimated ± 7.6 pp, low confidence |
| Grok 4.7 | 20.0% | 44.6% | estimated ± 7.6 pp, low confidence |
| Mistral Large 4 | 18.6% | 45.5% | estimated ± 7.6 pp, low confidence |