Calibration
CyberGym → DeepSearchQA
DeepSearchQA is estimated from CyberGym with a offset logistic curve fitted on 5 models measured on both: y = 0.7675 + (1.2000 − 0.7675) / (1 + exp(−199.64·(x − 0.8661))), R² = 0.77, cross-validated error 11.5 pp. It is used for 5 estimates.
| Estimated model | CyberGym | DeepSearchQA | Source |
|---|---|---|---|
| Claude Mythos Preview | 83.1% | 76.8% | estimated ± 11.5 pp, low confidence |
| Gemini 3.5 Flash Cyber | 83.2% | 76.8% | estimated ± 11.5 pp, low confidence |
| Gemini 3.8 Flash Cyber | 86.2% | 90.0% | estimated ± 11.5 pp, low confidence |
| GLM-5 | 43.2% | 76.8% | estimated ± 11.5 pp, low confidence |
| Fugu Cyber | 86.9% | 100.0% | estimated ± 11.5 pp, low confidence |