benchgap
Calibration

APEX-Agents → AutomationBench

AutomationBench is estimated from APEX-Agents with a Michaelis–Menten curve fitted on 6 models measured on both: y = 1.7629·x / (1.44205 + x), R² = 0.78, cross-validated error 7.4 pp. It is used for 5 estimates.

Estimated modelAPEX-AgentsAutomationBenchSource
Apodex 1.138.5%37.1%estimated ± 7.4 pp, low confidence
dots3-note Preview30.8%31.0%estimated ± 7.4 pp, low confidence
Apodex 1.1 Mini27.7%28.4%estimated ± 7.4 pp, low confidence
Solar Pro 418.7%20.2%estimated ± 7.4 pp, low confidence
Solar Open 216.6%18.2%estimated ± 7.4 pp, low confidence

Fitted on

ModelAPEX-AgentsAutomationBench
ERNIE 5.5 Flash Preview64.2%61.9%
Hy475.2%56.2%
Hy4 flash63.1%52.2%
Hy4 preview37.1%32.1%
Kimi K337.6%30.8%
Step 5 Preview37.8%44.0%