Calibration
HLE w/o tools → AA-GPQA Diamond
AA-GPQA Diamond is estimated from HLE w/o tools with a Michaelis–Menten + offset curve fitted on 25 models measured on both: y = 0.6876 + 0.3709·x / (0.29205 + x), R² = 0.80, cross-validated error 2.7 pp. It is used for 12 estimates.
| Estimated model | HLE w/o tools | AA-GPQA Diamond | Source |
|---|---|---|---|
| Claude Haiku 5.5 | 45.9% | 91.4% | estimated ± 2.7 pp, high confidence |
| Claude Mythos 5 | 59.0% | 93.6% | estimated ± 2.7 pp, high confidence |
| Hy4 preview | 43.4% | 90.9% | estimated ± 2.7 pp, high confidence |
| Ornith-1.5-35B-A3B | 25.6% | 86.1% | estimated ± 2.7 pp, high confidence |
| Ornith-1.5-397B | 44.6% | 91.2% | estimated ± 2.7 pp, high confidence |
| Ornith-1.5-9B | 20.2% | 83.9% | estimated ± 2.7 pp, high confidence |
| Pareto 26.9 | 49.0% | 92.0% | estimated ± 2.7 pp, high confidence |
| Qwen3.8 Max | 43.6% | 91.0% | estimated ± 2.7 pp, high confidence |
| Beam | 36.2% | 89.3% | estimated ± 2.7 pp, high confidence |
| Sakana Fugu | 47.2% | 91.7% | estimated ± 2.7 pp, high confidence |
| Sakana Fugu-Ultra | 50.0% | 92.2% | estimated ± 2.7 pp, high confidence |
| Solar Open 2 | 28.8% | 87.2% | estimated ± 2.7 pp, high confidence |