Calibration
τ³-bench results → DeepPlanning
DeepPlanning is estimated from τ³-bench results with a offset logistic curve fitted on 6 models measured on both: y = 0.1297 + (0.3539 − 0.1297) / (1 + exp(−200.00·(x − 0.6698))), R² = 0.80, cross-validated error 8.3 pp. It is used for 7 estimates.
| Estimated model | τ³-bench results | DeepPlanning | Source |
|---|---|---|---|
| MiMo-V2.5-Pro | 72.9% | 35.4% | estimated ± 8.3 pp, low confidence |
| Nemotron 3 Ultra | 70.9% | 35.4% | estimated ± 8.3 pp, low confidence |
| GLM-5.1 | 70.6% | 35.4% | estimated ± 8.3 pp, low confidence |
| LFM3-1.7B | 68.0% | 32.9% | estimated ± 8.3 pp, low confidence |
| MiniMax M2.7 | 67.6% | 30.4% | estimated ± 8.3 pp, low confidence |
| Pokee-Isaac 28B | 66.2% | 16.9% | estimated ± 8.3 pp, low confidence |
| Granite 4.2 30B | 62.0% | 13.0% | estimated ± 8.3 pp, low confidence |
Fitted on
| Model | τ³-bench results | DeepPlanning |
|---|---|---|
| Claude Opus 4.5 | 70.2% | 26.4% |
| GLM-5 | 65.6% | 14.6% |
| Kimi K2.5 | 65.7% | 14.4% |
| Qwen3.5 397B | 68.4% | 37.6% |
| Qwen3.6-35B-A3B | 67.2% | 25.9% |
| Qwen3.6 Plus | 70.7% | 41.5% |