Calibration
NL2Repo → OpenHarmony Bench
OpenHarmony Bench is estimated from NL2Repo with a Michaelis–Menten + offset curve fitted on 10 models measured on both: y = 0.0000 + 0.9654·x / (0.37363 + x), R² = 0.70, cross-validated error 2.7 pp. It is used for 13 estimates.
| Estimated model | NL2Repo | OpenHarmony Bench | Source |
|---|---|---|---|
| Claude Opus 4.5 | 43.2% | 51.8% | estimated ± 2.7 pp, high confidence |
| dots3-note Preview | 49.8% | 55.2% | estimated ± 2.7 pp, high confidence |
| Hy4 preview | 58.9% | 59.1% | estimated ± 2.7 pp, high confidence |
| Ornith-1.0-35B | 34.6% | 46.4% | estimated ± 2.7 pp, medium confidence |
| Ornith-1.0-397B | 48.2% | 54.4% | estimated ± 2.7 pp, high confidence |
| Ornith-1.0-9B | 27.2% | 40.7% | estimated ± 2.7 pp, medium confidence |
| Ornith-1.5-35B-A3B | 46.2% | 53.4% | estimated ± 2.7 pp, high confidence |
| Ornith-1.5-397B | 59.5% | 59.3% | estimated ± 2.7 pp, high confidence |
| Ornith-1.5-9B | 32.4% | 44.8% | estimated ± 2.7 pp, medium confidence |
| Qwen 3.6 Max (preview) | 42.9% | 51.6% | estimated ± 2.7 pp, high confidence |
| Qwen3.8-Omni-Flash | 48.9% | 54.7% | estimated ± 2.7 pp, high confidence |
| Seed 2.1 Pro | 47.0% | 53.8% | estimated ± 2.7 pp, high confidence |
| Seed 2.1 Turbo | 43.7% | 52.0% | estimated ± 2.7 pp, high confidence |