Calibration
Artificial Analysis Intelligence Index → SuperGPQA
SuperGPQA is estimated from Artificial Analysis Intelligence Index with a Michaelis–Menten curve fitted on 16 models measured on both: y = 1.3448·x / (0.20445 + x), R² = 0.44, cross-validated error 10.4 pp. It is used for 14 estimates.
| Estimated model | Artificial Analysis Intelligence Index | SuperGPQA | Source |
|---|---|---|---|
| Claude 3 Opus | 8.7% | 40.2% | estimated ± 10.4 pp, low confidence |
| Claude 4.1 Opus | 18.6% | 64.0% | estimated ± 10.4 pp, low confidence |
| Claude 4.1 Opus Thinking | 22.9% | 71.0% | estimated ± 10.4 pp, low confidence |
| Claude Haiku 5.5 | 43.4% | 91.4% | estimated ± 10.4 pp, low confidence |
| DeepSeek R1 Distill Qwen 32B | 8.4% | 39.1% | estimated ± 10.4 pp, low confidence |
| Gemini 1.0 Pro | 5.3% | 27.9% | estimated ± 10.4 pp, low confidence |
| Gemini 1.5 Pro | 7.9% | 37.5% | estimated ± 10.4 pp, low confidence |
| GLM-5.3-Flash | 41.8% | 90.3% | estimated ± 10.4 pp, low confidence |
| GPT-4 Turbo | 7.0% | 34.4% | estimated ± 10.4 pp, low confidence |
| GPT-4o mini | 6.7% | 33.0% | estimated ± 10.4 pp, low confidence |
| o1-preview | 11.4% | 48.1% | estimated ± 10.4 pp, low confidence |
| o3-pro | 21.9% | 69.5% | estimated ± 10.4 pp, low confidence |
| Phi-4 Multimodal Instruct | 5.8% | 29.8% | estimated ± 10.4 pp, low confidence |
| Qwen2.5 Coder 32B Instruct | 6.7% | 33.3% | estimated ± 10.4 pp, low confidence |