Calibration

MMLU-Redux → SuperGPQA

Name: benchgap: LLM benchmark scores, measured and estimated
Creator: benchgap

SuperGPQA is estimated from MMLU-Redux with a Hill curve fitted on 8 models measured on both: y = 0.0000 + (1.2000 − 0.0000)·x^6.00 / (0.89988^6.00 + x^6.00), R² = 0.96, cross-validated error 4.5 pp. It is used for 3 estimates.

Estimated model	MMLU-Redux	SuperGPQA	Source
Mellum2-12B-A2.5B-Instruct	78.1%	35.9%	estimated ± 4.5 pp, high confidence
Mellum2-12B-A2.5B-Thinking	86.2%	52.3%	estimated ± 4.5 pp, high confidence
Ternary Bonsai 2 27B	89.1%	58.2%	estimated ± 4.5 pp, high confidence