Calibration
AA-GPQA Diamond → CritPt
CritPt is estimated from AA-GPQA Diamond with a offset logistic curve fitted on 171 models measured on both: y = 0.0056 + (0.2707 − 0.0056) / (1 + exp(−49.21·(x − 0.9095))), R² = 0.80, cross-validated error 2.9 pp. It is used for 2 estimates.
| Estimated model | AA-GPQA Diamond | CritPt | Source |
|---|---|---|---|
| o1-preview | 76.5% | 0.6% | estimated ± 2.9 pp, high confidence |
| o3-pro | 84.5% | 1.6% | estimated ± 2.9 pp, high confidence |