Calibration
GPQA Diamond → CritPt
CritPt is estimated from GPQA Diamond with a inverse Michaelis–Menten curve fitted on 52 models measured on both: y = 0.46832·(x − 0.8394) / (0.8394 + 0.3034 − x), R² = 0.80, cross-validated error 4.6 pp. It is used for 8 estimates.
| Estimated model | GPQA Diamond | CritPt | Source |
|---|---|---|---|
| Pareto 26.10 Preview | 92.4% | 18.1% | estimated ± 4.6 pp, high confidence |
| Particle-1 | 90.4% | 12.7% | estimated ± 4.6 pp, high confidence |
| Mercury 3 | 89.2% | 9.9% | estimated ± 4.6 pp, high confidence |
| Nemotron 3.5 Bolt 74B-A9B BF16 | 88.8% | 9.0% | estimated ± 4.6 pp, high confidence |
| Grok 4.20 | 88.5% | 8.3% | estimated ± 4.6 pp, high confidence |
| OrcaCyber-Nano-1.0 | 88.3% | 7.8% | estimated ± 4.6 pp, high confidence |
| LLaDA2.2-mini | 44.4% | 0.0% | estimated ± 4.6 pp, medium confidence |
| Mellum2.1-12B-A2.5B-Thinking | 64.6% | 0.0% | estimated ± 4.6 pp, high confidence |