benchgap
Calibration

CharXiv w/o tools → ParseBench-1

ParseBench-1 is estimated from CharXiv w/o tools with a offset logistic curve fitted on 6 models measured on both: y = 0.6157 + (0.7123 − 0.6157) / (1 + exp(−160.18·(x − 0.8298))), R² = 1.00, cross-validated error 1.0 pp. It is used for 11 estimates.

Estimated modelCharXiv w/o toolsParseBench-1Source
Kimi K3.1 Pro89.0%71.2%estimated ± 1.0 pp, low confidence
Claude Mythos 588.9%71.2%estimated ± 1.0 pp, low confidence
Qwen3.8 Max88.4%71.2%estimated ± 1.0 pp, low confidence
Inkling 2 Small85.6%71.1%estimated ± 1.0 pp, medium confidence
Kimi K384.8%70.7%estimated ± 1.0 pp, medium confidence
Qwen3.8-Flash-Next84.6%70.6%estimated ± 1.0 pp, medium confidence
Qwen3.8-Omni-Flash83.5%68.3%estimated ± 1.0 pp, medium confidence
dots3-note Preview83.1%66.9%estimated ± 1.0 pp, medium confidence
Inkling78.1%61.6%estimated ± 1.0 pp, medium confidence
Inkling-Small77.4%61.6%estimated ± 1.0 pp, medium confidence
LFM3-VL-2B70.2%61.6%estimated ± 1.0 pp, low confidence

Fitted on

ModelCharXiv w/o toolsParseBench-1
Claude Opus 4.782.1%63.3%
Claude Opus 4.880.5%62.3%
Claude Sonnet 577.0%61.2%
Gemini 3.7 Flash84.5%70.4%
Gemini 3.8 Flash86.2%71.2%
Qwen3.8-27B83.7%69.0%