Calibration
IFBench → IFEval
IFEval is estimated from IFBench with a offset logistic curve fitted on 16 models measured on both: y = 0.7828 + (0.9282 − 0.7828) / (1 + exp(−40.88·(x − 0.5221))), R² = 0.84, cross-validated error 3.5 pp. It is used for 21 estimates.
| Estimated model | IFBench | IFEval | Source |
|---|---|---|---|
| A.X K2 | 75.9% | 92.8% | estimated ± 3.5 pp, high confidence |
| Granite 4.2 30B | 77.2% | 92.8% | estimated ± 3.5 pp, high confidence |
| Granite 4.2 3B | 74.3% | 92.8% | estimated ± 3.5 pp, high confidence |
| Granite 4.2 8B | 79.3% | 92.8% | estimated ± 3.5 pp, high confidence |
| Hy3 Preview | 63.1% | 92.7% | estimated ± 3.5 pp, high confidence |
| Inkling | 79.8% | 92.8% | estimated ± 3.5 pp, high confidence |
| Inkling-Small | 82.2% | 92.8% | estimated ± 3.5 pp, medium confidence |
| LFM2.5-2.6B | 59.2% | 92.0% | estimated ± 3.5 pp, high confidence |
| Ling 3.0 Flash | 74.5% | 92.8% | estimated ± 3.5 pp, high confidence |
| Ling 3.0 Flash FP8 | 73.4% | 92.8% | estimated ± 3.5 pp, high confidence |
| LLaDA2.2-mini | 24.9% | 78.3% | estimated ± 3.5 pp, medium confidence |
| MAI-Thinking-1 | 85.0% | 92.8% | estimated ± 3.5 pp, medium confidence |
| Mercury 2.5 | 77.0% | 92.8% | estimated ± 3.5 pp, high confidence |
| Muse Glimmer 30B | 77.0% | 92.8% | estimated ± 3.5 pp, high confidence |
| Nemotron 3.5 Lightning 30B A3B NVFP4 | 72.9% | 92.8% | estimated ± 3.5 pp, high confidence |
| Qwen3.8-27B | 79.5% | 92.8% | estimated ± 3.5 pp, high confidence |
| Qwen3.8-Flash-Next | 81.3% | 92.8% | estimated ± 3.5 pp, medium confidence |
| Qwen3.8 Max | 82.8% | 92.8% | estimated ± 3.5 pp, medium confidence |
| Qwen3.8-Omni-Flash | 81.5% | 92.8% | estimated ± 3.5 pp, medium confidence |
| Beam | 79.7% | 92.8% | estimated ± 3.5 pp, high confidence |
| Solar Open 2 | 80.0% | 92.8% | estimated ± 3.5 pp, high confidence |