# How the gaps are filled, and when not to trust it

Method · https://benchgap.net/method · data retrieved 2026-11-21

benchgap is an LLM benchmark leaderboard that fills in the missing scores. Most models are only ever run on a handful of benchmarks, so benchgap calibrates benchmarks against each other on the models measured on both, then estimates each missing score with its cross-validated error and a confidence level. Measured and estimated scores are always marked apart.

- Measured scores come from public evaluation leaderboards and are never altered; each benchmark is calibrated only against benchmarks of the same capability.
- For every ordered pair of benchmarks with at least 5 models measured on both, 6 monotone curves are fitted and the rising one with the lowest leave-one-out cross-validated error is kept (a falling fit is an overfit to too few models, never a calibration). That error, in percentage points, is the ± shown with every estimate.
- A pair keeps no calibration unless its best curve reaches R² ≥ 0.3 and an error of at most 15 pp; such gaps stay empty.
- A missing score is estimated from the best calibration out of a benchmark the model was measured on, if its score there lies within one training-range width of the scores the calibration was fitted on; further out the gap stays empty. Estimates are never used to make further estimates.
- Several same-capability benchmarks may be combined into a multivariate mapping: an elastic net with non-negative coefficients whose lasso zeroes the useless sources or a greedy forward search finds the features, and then a linear fit and a multivariate Michaelis–Menten curve compete. It is stored only if it passes the gate and beats the target's best single calibration, and preferred for a model measured on all of its sources - a model missing one source falls back to the univariate path.
- Cross-domain predictability and the multivariate view are analyses only: no estimate crosses a capability.
- Confidence: **high** for an error up to 5 pp, **medium** up to 10 pp, **low** above; extrapolation, a fit on fewer than 8 models or R² below 0.5 each lower it by one level.
- The harness tax (see [/harness-tax](https://benchgap.net/harness-tax.md)) measures how much harnesses disagree about the same models on the same benchmark, from measured scores only; it never feeds the estimates, and it is why each of them holds for its source harness's evaluation setup only.
- Estimates are predictions, not measurements, and hold for the source leaderboard's evaluation setup only.
- The whole pipeline is validated end to end: masking measured scores and refitting everything, estimates fill about 70% of masked cells at about 4.7 pp mean absolute error (23% lower than a matrix-completion baseline on the same cells); the confidence levels are correctly ordered (3.1 / 5.1 / 7.5 pp realized MAE for high/medium/low), though the ± labels understate the realized RMSE by roughly 40–70%; reweighted to the published mix, a published estimate should be expected to carry about 5.9 pp MAE (9.1 pp RMSE).
