Calibration

Agents' Last Exam → AutomationBench

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

AutomationBench is estimated from Agents' Last Exam with a offset logistic curve fitted on 14 models measured on both: y = 0.2789 + (0.4516 − 0.2789) / (1 + exp(−200.00·(x − 0.2609))), R² = 0.34, cross-validated error 10.8 pp. It is used for 0 estimates.