Calibration
OSWorld 2.1 Offline → MKA
MKA is estimated from OSWorld 2.1 Offline with a offset logistic curve fitted on 9 models measured on both: y = 0.3729 + (0.5400 − 0.3729) / (1 + exp(−107.06·(x − 0.7398))), R² = 0.93, cross-validated error 3.4 pp. It is used for 2 estimates.
| Estimated model | OSWorld 2.1 Offline | MKA | Source |
|---|---|---|---|
| Claude Fable 5.1 | 83.5% | 54.0% | estimated ± 3.4 pp, high confidence |
| Claude Haiku 4.5 | 15.7% | 37.3% | estimated ± 3.4 pp, medium confidence |
Fitted on
| Model | OSWorld 2.1 Offline | MKA |
|---|---|---|
| Claude Fable 5.2 | 89.3% | 56.6% |
| Claude Haiku 5.5 | 72.4% | 41.0% |
| Claude Opus 5.5 | 87.2% | 53.0% |
| Claude Sonnet 5.5 | 83.9% | 52.0% |
| Gemini 4 Argon | 74.7% | 51.3% |
| Gemini 4 Carbon | 81.5% | 54.3% |
| Gemini 4 Hydrogen | 77.6% | 53.4% |
| GPT-6.1 Terra | 74.2% | 43.8% |
| GPT-6 Luna | 48.9% | 36.8% |