Calibration
VideoMMMU → CharXiv
CharXiv is estimated from VideoMMMU with a Hill curve fitted on 10 models measured on both: y = 0.0000 + (1.2000 − 0.0000)·x^6.00 / (0.75823^6.00 + x^6.00), R² = 0.35, cross-validated error 5.9 pp. It is used for 0 estimates.
Fitted on
| Model | VideoMMMU | CharXiv |
|---|---|---|
| Claude Opus 4.5 | 84.4% | 68.5% |
| Gemini 3 Pro | 87.6% | 81.4% |
| GPT-5.2 | 85.9% | 82.1% |
| Kimi K2.5 | 86.6% | 77.5% |
| Qwen3.5 397B | 84.7% | 80.8% |
| Qwen3.6-27B | 84.4% | 78.4% |
| Qwen3.6-35B-A3B | 83.7% | 78.0% |
| Qwen3.6 Plus | 84.0% | 81.5% |
| Qwen3.7 Plus | 85.4% | 85.9% |
| Qwen3.8 Max | 88.7% | 93.5% |