benchgap
Calibration

AA-SciCode → cursorBench41

cursorBench41 is estimated from AA-SciCode with a offset logistic curve fitted on 9 models measured on both: y = 0.4480 + (0.5937 − 0.4480) / (1 + exp(−200.00·(x − 0.6078))), R² = 0.86, cross-validated error 3.7 pp. It is used for 61 estimates.

Estimated modelAA-SciCodecursorBench41Source
Muse Wonder Full62.6%59.0%estimated ± 3.7 pp, high confidence
Claude Fable 561.0%53.7%estimated ± 3.7 pp, high confidence
MiMo-V2.6-Pro60.9%52.9%estimated ± 3.7 pp, high confidence
Kimi K3.1 Pro60.8%52.2%estimated ± 3.7 pp, high confidence
Kimi K359.5%45.8%estimated ± 3.7 pp, high confidence
Step 5 Preview58.9%45.1%estimated ± 3.7 pp, high confidence
Muse Spark 1.158.8%45.1%estimated ± 3.7 pp, high confidence
Gemini 3.1 Pro58.7%45.0%estimated ± 3.7 pp, high confidence
Muse Spark Open58.4%44.9%estimated ± 3.7 pp, high confidence
GPT-6 Sol57.6%44.8%estimated ± 3.7 pp, high confidence
Muse Spark 1.257.4%44.8%estimated ± 3.7 pp, high confidence
Gemini 3.7 Flash57.2%44.8%estimated ± 3.7 pp, high confidence
Gemini 3.8 Flash56.6%44.8%estimated ± 3.7 pp, high confidence
GPT-6 Astra56.5%44.8%estimated ± 3.7 pp, high confidence
GPT-5.555.8%44.8%estimated ± 3.7 pp, high confidence
GPT-6.1 Terra55.1%44.8%estimated ± 3.7 pp, high confidence
Grok 4.555.0%44.8%estimated ± 3.7 pp, high confidence
GPT-6 Luna54.6%44.8%estimated ± 3.7 pp, medium confidence
Claude Opus 4.854.4%44.8%estimated ± 3.7 pp, medium confidence
GPT-6.1 Sol54.2%44.8%estimated ± 3.7 pp, medium confidence
Ling 3.1 Flash54.1%44.8%estimated ± 3.7 pp, medium confidence
Gemini 3.5 Flash53.9%44.8%estimated ± 3.7 pp, medium confidence
Gemini 3.6 Flash53.4%44.8%estimated ± 3.7 pp, medium confidence
MiniMax M3.1 Flash53.4%44.8%estimated ± 3.7 pp, medium confidence
Inkling 2 Small52.3%44.8%estimated ± 3.7 pp, medium confidence
GPT-5.4 mini52.1%44.8%estimated ± 3.7 pp, medium confidence
Qwen3.8 Max Preview52.1%44.8%estimated ± 3.7 pp, medium confidence
Kimi K2.651.5%44.8%estimated ± 3.7 pp, medium confidence
MiMo-V2.6-Flash51.3%44.8%estimated ± 3.7 pp, medium confidence
GLM-5.251.2%44.8%estimated ± 3.7 pp, medium confidence
DeepSeek V4 Pro51.0%44.8%estimated ± 3.7 pp, medium confidence
MiMo-V2.5-Pro50.6%44.8%estimated ± 3.7 pp, medium confidence
Qwen3.8-Flash-Next50.6%44.8%estimated ± 3.7 pp, medium confidence
DeepSeek V4 Flash 073150.3%44.8%estimated ± 3.7 pp, medium confidence
MiniMax M2.750.1%44.8%estimated ± 3.7 pp, medium confidence
Inkling-Small49.7%44.8%estimated ± 3.7 pp, medium confidence
Qwen3.7 Max49.5%44.8%estimated ± 3.7 pp, medium confidence
Hy348.6%44.8%estimated ± 3.7 pp, medium confidence
Hy3 Preview48.6%44.8%estimated ± 3.7 pp, medium confidence
Beam 1.0 501B-A23B48.5%44.8%estimated ± 3.7 pp, medium confidence
Grok 4.348.3%44.8%estimated ± 3.7 pp, medium confidence
Quasar 438B48.1%44.8%estimated ± 3.7 pp, medium confidence
Solar Pro 448.0%44.8%estimated ± 3.7 pp, medium confidence
Kimi K2.7 Code47.8%44.8%estimated ± 3.7 pp, medium confidence
Nemotron 3.5 Super VL 120B A12B47.8%44.8%estimated ± 3.7 pp, medium confidence
Nemotron 3.5 Super VL 120B A12B NVFP447.8%44.8%estimated ± 3.7 pp, medium confidence
GPT-5.4 nano47.2%44.8%estimated ± 3.7 pp, medium confidence
MiniMax M347.1%44.8%estimated ± 3.7 pp, medium confidence
Inkling47.0%44.8%estimated ± 3.7 pp, medium confidence
Qwen3.8-27B46.6%44.8%estimated ± 3.7 pp, medium confidence
Gemini 2.5 Pro46.3%44.8%estimated ± 3.7 pp, medium confidence
Qwen3.7 Plus46.1%44.8%estimated ± 3.7 pp, medium confidence
Apodex 1.145.5%44.8%estimated ± 3.7 pp, medium confidence
Apodex 1.1 Mini45.5%44.8%estimated ± 3.7 pp, medium confidence
Gemma 4 31B45.5%44.8%estimated ± 3.7 pp, medium confidence
Muse Glimmer 30B44.9%44.8%estimated ± 3.7 pp, medium confidence
GLM-5.144.8%44.8%estimated ± 3.7 pp, medium confidence
Qwen3.5 397B44.8%44.8%estimated ± 3.7 pp, medium confidence
Ling 3.0 Flash VL44.2%44.8%estimated ± 3.7 pp, medium confidence
Step 3.7 Flash43.9%44.8%estimated ± 3.7 pp, medium confidence
Qwen3.6-27B42.8%44.8%estimated ± 3.7 pp, medium confidence

Fitted on

ModelAA-SciCodecursorBench41
Claude Fable 5.267.7%61.3%
Claude Haiku 5.555.0%48.4%
Claude Opus 5.566.9%58.1%
Claude Sonnet 5.561.0%54.4%
Gemini 4 Argon61.8%56.5%
GPT-5.6 Sol57.1%40.9%
Grok 4.656.5%44.3%
Grok 4.757.4%48.6%
Muse Spark 1.358.8%41.9%