benchgap
Calibration

Mystery Game Puzzles → FrontierPhysics

FrontierPhysics is estimated from Mystery Game Puzzles with a Hill curve fitted on 20 models measured on both: y = 0.0261 + (0.2851 − 0.0261)·x^6.00 / (0.67987^6.00 + x^6.00), R² = 0.78, cross-validated error 4.2 pp. It is used for 36 estimates.

Estimated modelMystery Game PuzzlesFrontierPhysicsSource
Qwen3.8 Max38.0%3.4%estimated ± 4.2 pp, high confidence
Claude Sonnet 535.0%3.1%estimated ± 4.2 pp, high confidence
DeepSeek V4 Flash 073134.0%3.0%estimated ± 4.2 pp, high confidence
Gemini 3.5 Flash32.0%2.9%estimated ± 4.2 pp, high confidence
Qwen3.7 Max32.0%2.9%estimated ± 4.2 pp, high confidence
o329.0%2.8%estimated ± 4.2 pp, high confidence
Muse Spark 1.325.0%2.7%estimated ± 4.2 pp, high confidence
GPT-523.0%2.6%estimated ± 4.2 pp, high confidence
Qwen3.6-35B-A3B22.0%2.6%estimated ± 4.2 pp, high confidence
Gemini 3 Flash20.0%2.6%estimated ± 4.2 pp, high confidence
Nemotron 3 Ultra20.0%2.6%estimated ± 4.2 pp, high confidence
Qwen3.5 Flash20.0%2.6%estimated ± 4.2 pp, high confidence
Gemini 3.5 Flash-Lite19.0%2.6%estimated ± 4.2 pp, high confidence
Qwen 3.6 Max (preview)19.0%2.6%estimated ± 4.2 pp, high confidence
Qwen3.5 397B18.0%2.6%estimated ± 4.2 pp, high confidence
Claude Sonnet 4.517.0%2.6%estimated ± 4.2 pp, high confidence
Qwen3.5-122B-A10B17.0%2.6%estimated ± 4.2 pp, high confidence
Qwen3.5 Plus17.0%2.6%estimated ± 4.2 pp, high confidence
Qwen3.7 Plus17.0%2.6%estimated ± 4.2 pp, high confidence
Claude Sonnet 4.616.0%2.6%estimated ± 4.2 pp, medium confidence
GPT-5.116.0%2.6%estimated ± 4.2 pp, medium confidence
GLM-5.215.0%2.6%estimated ± 4.2 pp, medium confidence
Qwen3.7 Flash15.0%2.6%estimated ± 4.2 pp, medium confidence
GPT-4o mini12.0%2.6%estimated ± 4.2 pp, medium confidence
Qwen3.6 Plus12.0%2.6%estimated ± 4.2 pp, medium confidence
GLM-5.3-Flash8.0%2.6%estimated ± 4.2 pp, medium confidence
MiniMax M38.0%2.6%estimated ± 4.2 pp, medium confidence
GPT-4.1 mini7.0%2.6%estimated ± 4.2 pp, medium confidence
GPT-5.4 mini7.0%2.6%estimated ± 4.2 pp, medium confidence
o3-mini7.0%2.6%estimated ± 4.2 pp, medium confidence
Qwen3.6-27B7.0%2.6%estimated ± 4.2 pp, medium confidence
Inkling-Small6.0%2.6%estimated ± 4.2 pp, medium confidence
GPT-5.4 nano5.0%2.6%estimated ± 4.2 pp, medium confidence
o4-mini5.0%2.6%estimated ± 4.2 pp, medium confidence
Qwen3 Max5.0%2.6%estimated ± 4.2 pp, medium confidence
GPT-OSS 120B0.0%2.6%estimated ± 4.2 pp, medium confidence

Fitted on

ModelMystery Game PuzzlesFrontierPhysics
Claude Fable 5.158.0%11.3%
Claude Haiku 5.530.0%4.2%
Claude Opus 559.0%9.5%
Claude Opus 5.571.0%27.4%
DeepSeek V4.1 Flash 101749.0%3.6%
Gemini 3.8 Flash47.0%0.0%
Gemini 4 Argon64.0%8.3%
GLM-5.333.0%4.2%
GPT-5.6 Luna21.0%3.6%
GPT-5.6 Sol58.0%7.1%
GPT-5.6 Terra35.0%3.6%
GPT-6.1 Sol80.0%19.6%
GPT-6.1 Terra74.0%14.3%
GPT-6 Astra84.0%23.2%
GPT-6 Sol56.0%14.3%
Grok 4.634.0%2.4%
Grok 4.729.0%4.2%
Kimi K3.1 Pro42.0%6.5%
Kimi K326.0%2.4%
Mistral Large 417.0%0.0%