Calibration
BrowseComp → CyberGym
CyberGym is estimated from BrowseComp with a inverse Michaelis–Menten curve fitted on 15 models measured on both: y = 1.07522·(x − 0.0000) / (0.0000 + 2.0000 − x), R² = 0.60, cross-validated error 7.3 pp. It is used for 19 estimates.
| Estimated model | BrowseComp | CyberGym | Source |
|---|---|---|---|
| GPT-5.5 Pro | 90.1% | 88.2% | estimated ± 7.3 pp, medium confidence |
| GPT-5.4 Pro | 89.3% | 86.7% | estimated ± 7.3 pp, medium confidence |
| dots3-note Preview | 83.3% | 76.7% | estimated ± 7.3 pp, medium confidence |
| Sage-A1.5 | 81.7% | 74.3% | estimated ± 7.3 pp, medium confidence |
| Ling 3.0 Flash Sante | 80.2% | 72.0% | estimated ± 7.3 pp, medium confidence |
| Beam 1.0 501B-A23B FP8 | 76.7% | 66.9% | estimated ± 7.3 pp, medium confidence |
| Beam 1.0 501B-A23B NVFP4 | 75.6% | 65.3% | estimated ± 7.3 pp, medium confidence |
| Agents-A1 | 75.5% | 65.2% | estimated ± 7.3 pp, medium confidence |
| Nemotron 3.5 Bolt 74B-A9B BF16 | 70.0% | 57.9% | estimated ± 7.3 pp, medium confidence |
| Qwen3.5 397B | 69.0% | 56.6% | estimated ± 7.3 pp, medium confidence |
| Ornith-1.5-35B-A3B | 67.6% | 54.9% | estimated ± 7.3 pp, medium confidence |
| Agents-A1-4B | 66.8% | 53.9% | estimated ± 7.3 pp, medium confidence |
| GPT-5.2 | 65.8% | 52.7% | estimated ± 7.3 pp, medium confidence |
| Qwen3.5-35B-A3B | 61.0% | 47.2% | estimated ± 7.3 pp, low confidence |
| Ornith-1.5-9B | 56.4% | 42.2% | estimated ± 7.3 pp, low confidence |
| Spark-X2.6-4B | 53.1% | 38.9% | estimated ± 7.3 pp, low confidence |
| Sarvam 105B | 49.5% | 35.4% | estimated ± 7.3 pp, low confidence |
| LongCat-Flash-Lite-Sparse | 48.6% | 34.5% | estimated ± 7.3 pp, low confidence |
| Sarvam 30B | 35.5% | 23.2% | estimated ± 7.3 pp, low confidence |
Fitted on
| Model | BrowseComp | CyberGym |
|---|---|---|
| Atria Dawn Preview | 92.5% | 86.5% |
| Claude Mythos 5 | 88.0% | 83.8% |
| Claude Opus 4.6 | 83.7% | 66.6% |
| Claude Opus 4.7 | 79.3% | 73.1% |
| DeepSeek V4 Flash 0731 | 73.2% | 76.7% |
| DeepSeek V4 Pro | 83.4% | 83.3% |
| GLM-5 | 62.0% | 43.2% |
| GLM-5.1 | 68.0% | 68.7% |
| GPT-5.4 | 82.7% | 79.0% |
| GPT-5.5 | 84.4% | 81.8% |
| GPT-5.6 Luna | 83.3% | 77.9% |
| GPT-5.6 Sol | 92.2% | 84.5% |
| GPT-5.6 Terra | 87.5% | 81.8% |
| Ling 3.1 Flash | 91.7% | 87.9% |
| Step 5 Preview | 88.7% | 84.7% |