Calibration
MMLU-ProX → SWE Multilingual
SWE Multilingual is estimated from MMLU-ProX with a inverse Michaelis–Menten curve fitted on 8 models measured on both: y = 0.84681·(x − 0.0000) / (0.0000 + 1.8176 − x), R² = 0.48, cross-validated error 3.2 pp. It is used for 4 estimates.
| Estimated model | MMLU-ProX | SWE Multilingual | Source |
|---|---|---|---|
| Qwen3.5-122B-A10B | 82.2% | 69.9% | estimated ± 3.2 pp, low confidence |
| Qwen3.5-27B | 82.2% | 69.9% | estimated ± 3.2 pp, low confidence |
| Qwen3.5-35B-A3B | 81.0% | 68.1% | estimated ± 3.2 pp, low confidence |
| Qwen3 235B 2507 | 79.4% | 65.7% | estimated ± 3.2 pp, low confidence |
Fitted on
| Model | MMLU-ProX | SWE Multilingual |
|---|---|---|
| Claude Opus 4.5 | 85.7% | 77.5% |
| GLM-5 | 83.1% | 73.3% |
| Kimi K2.5 | 82.3% | 73.0% |
| Nemotron 3 Ultra | 83.0% | 67.7% |
| Qwen3.5 397B | 84.7% | 69.3% |
| Qwen3.6 Plus | 84.7% | 73.8% |
| Qwen3.7 Max | 87.0% | 78.3% |
| Qwen3.7 Plus | 85.4% | 75.8% |