benchgap
Calibration

MMLU → Artificial Analysis Intelligence Index

Artificial Analysis Intelligence Index is estimated from MMLU with a linear curve fitted on 14 models measured on both: y = 0.4470·x + -0.2834, R² = 0.30, cross-validated error 3.2 pp. It is used for 16 estimates.

Estimated modelMMLUArtificial Analysis Intelligence IndexSource
Sarvam 105B90.6%12.2%estimated ± 3.2 pp, medium confidence
Ripple-65B Preview89.6%11.7%estimated ± 3.2 pp, medium confidence
Grok 287.5%10.8%estimated ± 3.2 pp, medium confidence
Trinity-Large-Preview87.2%10.6%estimated ± 3.2 pp, medium confidence
Grok 2 mini86.2%10.2%estimated ± 3.2 pp, medium confidence
Particle-186.1%10.1%estimated ± 3.2 pp, medium confidence
Llama 3.1 70B86.0%10.1%estimated ± 3.2 pp, medium confidence
Llama 3.3 70B86.0%10.1%estimated ± 3.2 pp, medium confidence
LongCat-Flash-Lite-Sparse85.3%9.8%estimated ± 3.2 pp, medium confidence
GPT-OSS 20B85.3%9.8%estimated ± 3.2 pp, medium confidence
Sarvam 30B85.1%9.7%estimated ± 3.2 pp, medium confidence
Phi-484.8%9.6%estimated ± 3.2 pp, medium confidence
Mixtral 8x22B Instruct v0.177.8%6.4%estimated ± 3.2 pp, low confidence
Gemma 2 27B76.2%5.7%estimated ± 3.2 pp, low confidence
Llama 3.1 8B73.0%4.3%estimated ± 3.2 pp, low confidence
Phi-3.5-mini69.0%2.5%estimated ± 3.2 pp, low confidence

Fitted on

ModelMMLUArtificial Analysis Intelligence Index
Claude 3.5 Haiku80.3%8.9%
DeepSeek-R190.8%13.1%
GPT-4.190.2%12.7%
GPT-4.1 mini87.5%10.2%
GPT-4.1 nano80.1%7.8%
GPT-4.590.8%9.6%
GPT-4o mini82.0%6.7%
GPT-OSS 120B90.0%3.8%
Kimi K289.5%12.7%
Mistral Small 3.180.6%7.1%
o191.8%15.2%
o1-mini85.2%9.8%
o3-mini86.9%12.5%
o4-mini90.0%16.7%