Calibration
GPQA Diamond → AA-GPQA Diamond
AA-GPQA Diamond is estimated from GPQA Diamond with a linear curve fitted on 40 models measured on both: y = 1.0466·x + -0.0472, R² = 0.72, cross-validated error 5.6 pp. It is used for 3 estimates.
| Estimated model | GPQA Diamond | AA-GPQA Diamond | Source |
|---|---|---|---|
| LLaDA2.2-mini | 44.4% | 41.8% | estimated ± 5.6 pp, low confidence |
| Mercury 2.5 | 79.0% | 78.0% | estimated ± 5.6 pp, medium confidence |
| Pareto 26.10 Preview | 92.4% | 92.0% | estimated ± 5.6 pp, medium confidence |