benchgap
Calibration

OSWorld 2.1 Offline → MKA

MKA is estimated from OSWorld 2.1 Offline with a offset logistic curve fitted on 9 models measured on both: y = 0.3729 + (0.5400 − 0.3729) / (1 + exp(−107.06·(x − 0.7398))), R² = 0.93, cross-validated error 3.4 pp. It is used for 2 estimates.

Estimated modelOSWorld 2.1 OfflineMKASource
Claude Fable 5.183.5%54.0%estimated ± 3.4 pp, high confidence
Claude Haiku 4.515.7%37.3%estimated ± 3.4 pp, medium confidence

Fitted on

ModelOSWorld 2.1 OfflineMKA
Claude Fable 5.289.3%56.6%
Claude Haiku 5.572.4%41.0%
Claude Opus 5.587.2%53.0%
Claude Sonnet 5.583.9%52.0%
Gemini 4 Argon74.7%51.3%
Gemini 4 Carbon81.5%54.3%
Gemini 4 Hydrogen77.6%53.4%
GPT-6.1 Terra74.2%43.8%
GPT-6 Luna48.9%36.8%