benchgap
Calibration

ProgramBench → CursorBench 3.2

CursorBench 3.2 is estimated from ProgramBench with a Michaelis–Menten curve fitted on 5 models measured on both: y = 2.0000·x / (1.66105 + x), R² = 0.92, cross-validated error 3.6 pp. It is used for 8 estimates.

Estimated modelProgramBenchCursorBench 3.2Source
Step 5 Preview80.5%65.3%estimated ± 3.6 pp, medium confidence
Qwen4 Max31.5%31.9%estimated ± 3.6 pp, low confidence
Hy430.0%30.6%estimated ± 3.6 pp, low confidence
ERNIE 5.5 Flash Preview29.5%30.2%estimated ± 3.6 pp, low confidence
MiMo-V2.6-Pro26.5%27.5%estimated ± 3.6 pp, low confidence
MiMo-V2.6-Flash26.0%27.1%estimated ± 3.6 pp, low confidence
LongCat-2.525.5%26.6%estimated ± 3.6 pp, low confidence
Hy4 flash16.0%17.6%estimated ± 3.6 pp, low confidence

Fitted on

ModelProgramBenchCursorBench 3.2
Claude Fable 5.187.6%73.4%
Claude Opus 593.0%70.0%
GLM-5.263.7%55.0%
Kimi K2.7 Code53.6%49.7%
Kimi K377.8%60.8%