Calibration

HLE → MMLU-Pro

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

MMLU-Pro is estimated from HLE with a offset logistic curve fitted on 19 models measured on both: y = 0.6846 + (0.8634 − 0.6846) / (1 + exp(−38.20·(x − 0.1385))), R² = 0.68, cross-validated error 3.6 pp. It is used for 2 estimates.

Estimated model	HLE	MMLU-Pro	Source
Agents-A1	47.6%	86.3%	estimated ± 3.6 pp, high confidence
dots3-note Preview	52.6%	86.3%	estimated ± 3.6 pp, high confidence