Calibration
Artificial Analysis Intelligence Index → BioMysteryBench (human-difficult)
BioMysteryBench (human-difficult) is estimated from Artificial Analysis Intelligence Index with a offset logistic curve fitted on 6 models measured on both: y = 0.3523 + (0.5012 − 0.3523) / (1 + exp(−200.00·(x − 0.3889))), R² = 0.71, cross-validated error 8.1 pp. It is used for 13 estimates.
| Estimated model | Artificial Analysis Intelligence Index | BioMysteryBench (human-difficult) | Source |
|---|---|---|---|
| Claude 3 Opus | 8.7% | 35.2% | estimated ± 8.1 pp, low confidence |
| Claude 4.1 Opus | 18.6% | 35.2% | estimated ± 8.1 pp, low confidence |
| DeepSeek R1 Distill Qwen 32B | 8.4% | 35.2% | estimated ± 8.1 pp, low confidence |
| Gemini 1.0 Pro | 5.3% | 35.2% | estimated ± 8.1 pp, low confidence |
| Gemini 1.5 Pro | 7.9% | 35.2% | estimated ± 8.1 pp, low confidence |
| GPT-4 Turbo | 7.0% | 35.2% | estimated ± 8.1 pp, low confidence |
| GPT-4o mini | 6.7% | 35.2% | estimated ± 8.1 pp, low confidence |
| o1-preview | 11.4% | 35.2% | estimated ± 8.1 pp, low confidence |
| o1-pro | 12.4% | 35.2% | estimated ± 8.1 pp, low confidence |
| o3-mini | 12.5% | 35.2% | estimated ± 8.1 pp, low confidence |
| o3-pro | 21.9% | 35.2% | estimated ± 8.1 pp, low confidence |
| Phi-4 Multimodal Instruct | 5.8% | 35.2% | estimated ± 8.1 pp, low confidence |
| Qwen2.5 Coder 32B Instruct | 6.7% | 35.2% | estimated ± 8.1 pp, low confidence |