Calibration
CharXiv w/o tools → ParseBench-1
ParseBench-1 is estimated from CharXiv w/o tools with a offset logistic curve fitted on 6 models measured on both: y = 0.6157 + (0.7123 − 0.6157) / (1 + exp(−160.18·(x − 0.8298))), R² = 1.00, cross-validated error 1.0 pp. It is used for 11 estimates.
| Estimated model | CharXiv w/o tools | ParseBench-1 | Source |
|---|---|---|---|
| Kimi K3.1 Pro | 89.0% | 71.2% | estimated ± 1.0 pp, low confidence |
| Claude Mythos 5 | 88.9% | 71.2% | estimated ± 1.0 pp, low confidence |
| Qwen3.8 Max | 88.4% | 71.2% | estimated ± 1.0 pp, low confidence |
| Inkling 2 Small | 85.6% | 71.1% | estimated ± 1.0 pp, medium confidence |
| Kimi K3 | 84.8% | 70.7% | estimated ± 1.0 pp, medium confidence |
| Qwen3.8-Flash-Next | 84.6% | 70.6% | estimated ± 1.0 pp, medium confidence |
| Qwen3.8-Omni-Flash | 83.5% | 68.3% | estimated ± 1.0 pp, medium confidence |
| dots3-note Preview | 83.1% | 66.9% | estimated ± 1.0 pp, medium confidence |
| Inkling | 78.1% | 61.6% | estimated ± 1.0 pp, medium confidence |
| Inkling-Small | 77.4% | 61.6% | estimated ± 1.0 pp, medium confidence |
| LFM3-VL-2B | 70.2% | 61.6% | estimated ± 1.0 pp, low confidence |
Fitted on
| Model | CharXiv w/o tools | ParseBench-1 |
|---|---|---|
| Claude Opus 4.7 | 82.1% | 63.3% |
| Claude Opus 4.8 | 80.5% | 62.3% |
| Claude Sonnet 5 | 77.0% | 61.2% |
| Gemini 3.7 Flash | 84.5% | 70.4% |
| Gemini 3.8 Flash | 86.2% | 71.2% |
| Qwen3.8-27B | 83.7% | 69.0% |