Calibration
BrowseComp → Agents' Last Exam
Agents' Last Exam is estimated from BrowseComp with a Hill curve fitted on 6 models measured on both: y = 0.1026 + (1.2000 − 0.1026)·x^6.00 / (1.05694^6.00 + x^6.00), R² = 0.50, cross-validated error 11.6 pp. It is used for 30 estimates.
| Estimated model | BrowseComp | Agents' Last Exam | Source |
|---|---|---|---|
| Inkling 2 Small | 90.2% | 40.8% | estimated ± 11.6 pp, low confidence |
| GPT-5.5 Pro | 90.1% | 40.7% | estimated ± 11.6 pp, low confidence |
| GPT-5.4 Pro | 89.3% | 39.5% | estimated ± 11.6 pp, low confidence |
| ERNIE 5.5 Flash Preview | 87.4% | 36.8% | estimated ± 11.6 pp, low confidence |
| Ornith-1.5-397B | 86.6% | 35.7% | estimated ± 11.6 pp, low confidence |
| dots3-note Preview | 83.3% | 31.5% | estimated ± 11.6 pp, low confidence |
| Kimi K2.6 | 83.2% | 31.3% | estimated ± 11.6 pp, low confidence |
| Sage-A1.5 | 81.7% | 29.6% | estimated ± 11.6 pp, low confidence |
| Ling 3.0 Flash Sante | 80.2% | 27.8% | estimated ± 11.6 pp, low confidence |
| Nemotron 3.5 Super VL 120B A12B | 78.0% | 25.5% | estimated ± 11.6 pp, low confidence |
| Nemotron 3.5 Super VL 120B A12B NVFP4 | 77.5% | 25.0% | estimated ± 11.6 pp, low confidence |
| Inkling-Small | 77.4% | 24.9% | estimated ± 11.6 pp, low confidence |
| Inkling | 77.1% | 24.6% | estimated ± 11.6 pp, low confidence |
| Beam 1.0 501B-A23B FP8 | 76.7% | 24.2% | estimated ± 11.6 pp, low confidence |
| Step 3.7 Flash | 75.8% | 23.4% | estimated ± 11.6 pp, low confidence |
| Beam 1.0 501B-A23B NVFP4 | 75.6% | 23.2% | estimated ± 11.6 pp, low confidence |
| Agents-A1 | 75.5% | 23.1% | estimated ± 11.6 pp, low confidence |
| Ling 3.0 Flash | 72.2% | 20.4% | estimated ± 11.6 pp, low confidence |
| Nemotron 3.5 Bolt 74B-A9B BF16 | 70.0% | 18.8% | estimated ± 11.6 pp, low confidence |
| Qwen3.5 397B | 69.0% | 18.1% | estimated ± 11.6 pp, low confidence |
| Ornith-1.5-35B-A3B | 67.6% | 17.3% | estimated ± 11.6 pp, low confidence |
| Agents-A1-4B | 66.8% | 16.8% | estimated ± 11.6 pp, low confidence |
| Nemotron 3.5 Bolt 74B-A9B | 66.8% | 16.8% | estimated ± 11.6 pp, low confidence |
| GPT-5.2 | 65.8% | 16.3% | estimated ± 11.6 pp, low confidence |
| Qwen3.5-122B-A10B | 63.8% | 15.3% | estimated ± 11.6 pp, low confidence |
| GLM-5 | 62.0% | 14.6% | estimated ± 11.6 pp, low confidence |
| Qwen3.5-27B | 61.0% | 14.2% | estimated ± 11.6 pp, low confidence |
| Qwen3.5-35B-A3B | 61.0% | 14.2% | estimated ± 11.6 pp, low confidence |
| Kimi K2.5 | 60.6% | 14.0% | estimated ± 11.6 pp, low confidence |
| Ornith-1.5-9B | 56.4% | 12.7% | estimated ± 11.6 pp, low confidence |
Fitted on
| Model | BrowseComp | Agents' Last Exam |
|---|---|---|
| DeepSeek V4 Flash 0731 | 73.2% | 25.2% |
| DeepSeek V4 Pro | 83.4% | 25.7% |
| GPT-6 Astra | 91.5% | 59.3% |
| Kimi K3.1 Pro | 92.1% | 43.2% |
| LongCat-2.5 | 91.9% | 38.5% |
| Step 5 Preview | 88.7% | 29.5% |