Calibration
HLE w/ tools → AA Harvey LAB
AA Harvey LAB is estimated from HLE w/ tools with a Michaelis–Menten curve fitted on 5 models measured on both: y = 1.1186·x / (0.13760 + x), R² = 0.94, cross-validated error 1.7 pp. It is used for 9 estimates.
| Estimated model | HLE w/ tools | AA Harvey LAB | Source |
|---|---|---|---|
| Agents-A1 | 47.6% | 86.8% | estimated ± 1.7 pp, medium confidence |
| Apodex 1.1 | 56.1% | 89.8% | estimated ± 1.7 pp, medium confidence |
| DeepSeek V4.1 Flash | 63.9% | 92.0% | estimated ± 1.7 pp, medium confidence |
| dots3-note Preview | 52.6% | 88.7% | estimated ± 1.7 pp, medium confidence |
| Hy4 preview | 55.4% | 89.6% | estimated ± 1.7 pp, medium confidence |
| Ornith-1.5-35B-A3B | 33.4% | 79.2% | estimated ± 1.7 pp, low confidence |
| Ornith-1.5-397B | 56.1% | 89.8% | estimated ± 1.7 pp, medium confidence |
| Ornith-1.5-9B | 30.5% | 77.1% | estimated ± 1.7 pp, low confidence |
| Step 3.7 Flash | 47.2% | 86.6% | estimated ± 1.7 pp, medium confidence |