benchgap
Calibration

JobBench → DeepSearchQA

DeepSearchQA is estimated from JobBench with a linear curve fitted on 8 models measured on both: y = 0.3148·x + 0.7021, R² = 0.44, cross-validated error 8.8 pp. It is used for 15 estimates.

Estimated modelJobBenchDeepSearchQASource
Muse Wonder Full75.3%93.9%estimated ± 8.8 pp, low confidence
Muse Wonder70.5%92.4%estimated ± 8.8 pp, low confidence
Ultramarine 1 Flash62.9%90.0%estimated ± 8.8 pp, low confidence
Muse Spark Open61.8%89.7%estimated ± 8.8 pp, low confidence
Hy4 preview61.7%89.6%estimated ± 8.8 pp, low confidence
Qwen3.8-Flash-Next55.7%87.8%estimated ± 8.8 pp, low confidence
GPT-5.1-Codex26.2%78.5%estimated ± 8.8 pp, low confidence
GPT-5.2-Codex26.0%78.4%estimated ± 8.8 pp, low confidence
Claude 4.1 Opus21.9%77.1%estimated ± 8.8 pp, low confidence
Qwen3.5 Plus18.5%76.0%estimated ± 8.8 pp, low confidence
Claude 4 Sonnet18.4%76.0%estimated ± 8.8 pp, low confidence
Claude Haiku 4.516.0%75.3%estimated ± 8.8 pp, low confidence
Gemini 3 Flash11.4%73.8%estimated ± 8.8 pp, low confidence
Gemini 3 Pro11.4%73.8%estimated ± 8.8 pp, low confidence
GPT-58.5%72.9%estimated ± 8.8 pp, low confidence

Fitted on

ModelJobBenchDeepSearchQA
Atria Dawn Preview50.3%96.0%
Claude Opus 4.636.7%73.7%
GPT-5.438.9%73.6%
Inkling 2 Small70.5%90.9%
Kimi K2.58.7%77.1%
Kimi K352.9%95.0%
Muse Spark 1.154.7%84.9%
Muse Spark 1.364.9%89.4%