Calibration
BrowseComp → CyberGym
CyberGym is estimated from BrowseComp with a Michaelis–Menten curve fitted on 13 models measured on both: y = 2.0000·x / (1.27417 + x), R² = 0.52, cross-validated error 4.6 pp. It is used for 21 estimates.
| Estimated model | BrowseComp | CyberGym | Source |
|---|---|---|---|
| Agents-A1-4B | 66.8% | 68.8% | estimated ± 4.6 pp, medium confidence |
| Claude Opus 4.8 | 84.3% | 79.6% | estimated ± 4.6 pp, high confidence |
| GLM-4.7 | 52.0% | 58.0% | estimated ± 4.6 pp, medium confidence |
| GPT-5.2 | 65.8% | 68.1% | estimated ± 4.6 pp, medium confidence |
| GPT-5.4 Pro | 89.3% | 82.4% | estimated ± 4.6 pp, high confidence |
| GPT-5.5 Pro | 90.1% | 82.8% | estimated ± 4.6 pp, high confidence |
| Inkling | 77.1% | 75.4% | estimated ± 4.6 pp, high confidence |
| Inkling-Small | 77.4% | 75.6% | estimated ± 4.6 pp, high confidence |
| Kimi K2.6 | 83.2% | 79.0% | estimated ± 4.6 pp, high confidence |
| Kimi K2.5 | 60.6% | 64.5% | estimated ± 4.6 pp, medium confidence |
| Kimi K2.5 (Reasoning) | 60.6% | 64.5% | estimated ± 4.6 pp, medium confidence |
| Ling 3.0 Flash | 72.2% | 72.3% | estimated ± 4.6 pp, high confidence |
| LongCat-Flash-Lite-Sparse | 48.6% | 55.2% | estimated ± 4.6 pp, medium confidence |
| MiniMax M3 | 83.5% | 79.2% | estimated ± 4.6 pp, high confidence |
| Nemotron 3.5 Lightning 30B A3B NVFP4 | 36.8% | 44.8% | estimated ± 4.6 pp, medium confidence |
| Qwen3.5-122B-A10B | 63.8% | 66.7% | estimated ± 4.6 pp, medium confidence |
| Qwen3.5-27B | 61.0% | 64.8% | estimated ± 4.6 pp, medium confidence |
| Qwen3.5-35B-A3B | 61.0% | 64.8% | estimated ± 4.6 pp, medium confidence |
| Qwen3.5 397B | 62.0% | 65.5% | estimated ± 4.6 pp, medium confidence |
| Beam | 77.4% | 75.6% | estimated ± 4.6 pp, high confidence |
| Solar Pro 4 | 49.2% | 55.7% | estimated ± 4.6 pp, medium confidence |