Calibration
GPQA Diamond → AA-HLE
AA-HLE is estimated from GPQA Diamond with a inverse Michaelis–Menten curve fitted on 43 models measured on both: y = 0.19196·(x − 0.4818) / (0.4818 + 0.6451 − x), R² = 0.90, cross-validated error 4.0 pp. It is used for 23 estimates.
| Estimated model | GPQA Diamond | AA-HLE | Source |
|---|---|---|---|
| Hy4 preview | 92.3% | 41.5% | estimated ± 4.0 pp, high confidence |
| Interfaze Beta | 89.9% | 35.1% | estimated ± 4.0 pp, high confidence |
| LFM2.5-230M | 25.4% | 0.0% | estimated ± 4.0 pp, medium confidence |
| LLaDA2.2-mini | 44.4% | 0.0% | estimated ± 4.0 pp, medium confidence |
| LongCat-Flash-Lite-Sparse | 69.5% | 9.5% | estimated ± 4.0 pp, high confidence |
| MAI-Thinking-1 | 84.2% | 24.3% | estimated ± 4.0 pp, high confidence |
| Mellum2-12B-A2.5B-Instruct | 40.9% | 0.0% | estimated ± 4.0 pp, medium confidence |
| Mellum2-12B-A2.5B-Thinking | 57.6% | 3.3% | estimated ± 4.0 pp, medium confidence |
| MiniCPM5-1B | 26.3% | 0.0% | estimated ± 4.0 pp, medium confidence |
| Ornith-1.5-35B-A3B | 89.2% | 33.5% | estimated ± 4.0 pp, high confidence |
| Ornith-1.5-397B | 92.8% | 43.1% | estimated ± 4.0 pp, high confidence |
| Ornith-1.5-9B | 86.4% | 27.9% | estimated ± 4.0 pp, high confidence |
| Pareto 26.10 Preview | 92.4% | 41.8% | estimated ± 4.0 pp, high confidence |
| Qwen3.8 Max | 92.6% | 42.5% | estimated ± 4.0 pp, high confidence |
| Qwen3.8-Omni-Flash | 91.0% | 37.9% | estimated ± 4.0 pp, high confidence |
| Beam | 90.5% | 36.6% | estimated ± 4.0 pp, high confidence |
| Sakana Fugu | 95.5% | 52.9% | estimated ± 4.0 pp, high confidence |
| Sakana Fugu-Ultra | 95.5% | 52.9% | estimated ± 4.0 pp, high confidence |
| Solar Open 2 | 86.3% | 27.7% | estimated ± 4.0 pp, high confidence |
| Soofi S 30B-A3B | 43.4% | 0.0% | estimated ± 4.0 pp, medium confidence |
| Ternary Bonsai 2 27B | 85.8% | 26.8% | estimated ± 4.0 pp, high confidence |
| ZAYA1-74B-Preview | 57.3% | 3.2% | estimated ± 4.0 pp, medium confidence |
| ZAYA1-8B | 71.0% | 10.5% | estimated ± 4.0 pp, high confidence |