Calibration
AA-HLE → AA-Omniscience Accuracy
AA-Omniscience Accuracy is estimated from AA-HLE with a Hill curve fitted on 183 models measured on both: y = 0.1722 + (1.2000 − 0.1722)·x^2.33 / (0.66413^2.33 + x^2.33), R² = 0.69, cross-validated error 6.9 pp. It is used for 11 estimates.
| Estimated model | AA-HLE | AA-Omniscience Accuracy | Source |
|---|---|---|---|
| Claude 3 Opus | 2.8% | 17.3% | estimated ± 6.9 pp, medium confidence |
| Claude 4.1 Opus Thinking | 12.5% | 19.3% | estimated ± 6.9 pp, medium confidence |
| Claude Haiku 5.5 | 44.4% | 46.1% | estimated ± 6.9 pp, medium confidence |
| DeepSeek R1 Distill Qwen 32B | 4.6% | 17.4% | estimated ± 6.9 pp, medium confidence |
| Gemini 1.0 Pro | 4.2% | 17.4% | estimated ± 6.9 pp, medium confidence |
| Gemini 1.5 Pro | 4.6% | 17.4% | estimated ± 6.9 pp, medium confidence |
| GLM-5.3-Flash | 39.9% | 41.3% | estimated ± 6.9 pp, medium confidence |
| GPT-4 Turbo | 3.1% | 17.3% | estimated ± 6.9 pp, medium confidence |
| GPT-4o mini | 4.2% | 17.4% | estimated ± 6.9 pp, medium confidence |
| Phi-4 Multimodal Instruct | 5.0% | 17.5% | estimated ± 6.9 pp, medium confidence |
| Qwen2.5 Coder 32B Instruct | 3.5% | 17.3% | estimated ± 6.9 pp, medium confidence |