Calibration
Video-MME (with subtitle) → CountBench
CountBench is estimated from Video-MME (with subtitle) with a offset logistic curve fitted on 7 models measured on both: y = 0.9164 + (0.9839 − 0.9164) / (1 + exp(−200.00·(x − 0.8717))), R² = 0.83, cross-validated error 2.0 pp. It is used for 2 estimates.
| Estimated model | Video-MME (with subtitle) | CountBench | Source |
|---|---|---|---|
| MiMo-V2.5 | 87.7% | 96.7% | estimated ± 2.0 pp, medium confidence |
| MiniMax M3 | 85.4% | 91.8% | estimated ± 2.0 pp, medium confidence |
Fitted on
| Model | Video-MME (with subtitle) | CountBench |
|---|---|---|
| Claude Opus 4.5 | 77.6% | 90.6% |
| Gemini 3 Pro | 88.4% | 97.3% |
| GPT-5.2 | 86.0% | 91.9% |
| Kimi K2.5 | 87.4% | 94.1% |
| LFM3-VL-2B | 69.0% | 93.0% |
| Qwen3.5 397B | 87.5% | 97.2% |
| Qwen3.6-27B | 87.7% | 97.8% |