Calibration
MMLU → Artificial Analysis Intelligence Index
Artificial Analysis Intelligence Index is estimated from MMLU with a linear curve fitted on 14 models measured on both: y = 0.4470·x + -0.2834, R² = 0.30, cross-validated error 3.2 pp. It is used for 16 estimates.
| Estimated model | MMLU | Artificial Analysis Intelligence Index | Source |
|---|---|---|---|
| Sarvam 105B | 90.6% | 12.2% | estimated ± 3.2 pp, medium confidence |
| Ripple-65B Preview | 89.6% | 11.7% | estimated ± 3.2 pp, medium confidence |
| Grok 2 | 87.5% | 10.8% | estimated ± 3.2 pp, medium confidence |
| Trinity-Large-Preview | 87.2% | 10.6% | estimated ± 3.2 pp, medium confidence |
| Grok 2 mini | 86.2% | 10.2% | estimated ± 3.2 pp, medium confidence |
| Particle-1 | 86.1% | 10.1% | estimated ± 3.2 pp, medium confidence |
| Llama 3.1 70B | 86.0% | 10.1% | estimated ± 3.2 pp, medium confidence |
| Llama 3.3 70B | 86.0% | 10.1% | estimated ± 3.2 pp, medium confidence |
| LongCat-Flash-Lite-Sparse | 85.3% | 9.8% | estimated ± 3.2 pp, medium confidence |
| GPT-OSS 20B | 85.3% | 9.8% | estimated ± 3.2 pp, medium confidence |
| Sarvam 30B | 85.1% | 9.7% | estimated ± 3.2 pp, medium confidence |
| Phi-4 | 84.8% | 9.6% | estimated ± 3.2 pp, medium confidence |
| Mixtral 8x22B Instruct v0.1 | 77.8% | 6.4% | estimated ± 3.2 pp, low confidence |
| Gemma 2 27B | 76.2% | 5.7% | estimated ± 3.2 pp, low confidence |
| Llama 3.1 8B | 73.0% | 4.3% | estimated ± 3.2 pp, low confidence |
| Phi-3.5-mini | 69.0% | 2.5% | estimated ± 3.2 pp, low confidence |
Fitted on
| Model | MMLU | Artificial Analysis Intelligence Index |
|---|---|---|
| Claude 3.5 Haiku | 80.3% | 8.9% |
| DeepSeek-R1 | 90.8% | 13.1% |
| GPT-4.1 | 90.2% | 12.7% |
| GPT-4.1 mini | 87.5% | 10.2% |
| GPT-4.1 nano | 80.1% | 7.8% |
| GPT-4.5 | 90.8% | 9.6% |
| GPT-4o mini | 82.0% | 6.7% |
| GPT-OSS 120B | 90.0% | 3.8% |
| Kimi K2 | 89.5% | 12.7% |
| Mistral Small 3.1 | 80.6% | 7.1% |
| o1 | 91.8% | 15.2% |
| o1-mini | 85.2% | 9.8% |
| o3-mini | 86.9% | 12.5% |
| o4-mini | 90.0% | 16.7% |