benchgap
Model compare

Kimi K3.1 Pro vs GPT-6.1 Sol

Kimi K3.1 Pro stands at 91.5 on 190 benchmarks, 60 measured (rank #20 of 394), GPT-6.1 Sol at 92.5 on 168 benchmarks, 55 measured (rank #16 of 394) - a frontier model. Of the 161 benchmarks both have a score on, Kimi K3.1 Pro leads on 4 and GPT-6.1 Sol on 24 where both are measured.

BenchmarkKimi K3.1 ProGPT-6.1 SolΔ (pp)
AA-GPQA Diamond92.4%96.0%-3.6
AA-HLE52.5%52.9%-0.4
AA-SciCode60.8%54.2%+6.6
Artificial Analysis Intelligence Index46.5%54.6%-8.2
CritPt24.6%31.7%-7.1
GDPval-AA67.3%54.8%+12.5
GPQA Diamond94.2%96.2%-2.0
HLE70.0%69.4%+0.6
AA-LCR86.3%83.0%+3.3
LiveCodeBench v6100.0%94.7%+5.3
AA-Omniscience Accuracy48.9%61.8%-12.9
SciCode60.3%53.9%+6.4
Terminal-Bench 2.189.1%88.9%+0.2
BrowseComp92.1%91.4%+0.7
HLE w/ tools70.0%62.4%+7.6
LongBench v271.5%68.4%+3.2
AA Coding Index76.0%78.7%-2.8
AA-MMMU-Pro83.9%86.0%-2.1
APEX-Agents69.4%37.5%+31.9
APEX-Agents-AA39.0%40.0%-1.0
BioMysteryBench (human-difficult)49.0%49.9%-0.9
DeepSearchQA93.7%89.6%+4.1
SWE-bench Verified90.8%89.0%+1.7
AutomationBench47.4%36.1%+11.3
CyberGym96.3%83.0%+13.3
JobBench66.5%59.3%+7.2
SWE-bench Pro77.0%81.4%-4.4
WideResearch83.1%84.3%-1.1
AA Agentic Index68.0%50.6%+17.4
MMLU-Pro85.9%85.5%+0.3
FailureSensorIQ68.3%68.6%-0.3
GPQA94.2%95.8%-1.6
Vals GPQA Diamond93.1%98.8%-5.7
Vals LiveCodeBench89.0%85.8%+3.2
Vals MMLU-Pro89.9%89.8%+0.1
MMMLU91.2%92.0%-0.8
MMMU88.5%89.9%-1.4
Chess Puzzles45.0%68.0%-23.0
AA MMLU-Pro88.5%90.5%-2.0
Gert Labs80.3%73.2%+7.1
AA-AnalystAgent38.8%50.0%-11.2
AA Tau3 Banking100.0%45.7%+54.3
ApprenticeBench17.6%51.7%-34.1
ARC-AGI-195.1%96.5%-1.4
ARC-AGI-279.9%94.2%-14.2
BabaIsBench98.6%100.0%-1.4
CursorBench 3.170.6%54.4%+16.3
CursorBench 3.261.1%68.2%-7.2
EBR-bench37.1%54.3%-17.2
Terminal-Bench 3.046.4%47.1%-0.8
FrontierCode 1.1 Main50.9%52.4%-1.4
FrontierSWE v232.7%56.0%-23.3
Furniture Assembly55.8%80.0%-24.2
MaxxBench62.6%69.4%-6.8
MazeBench2.0%17.0%-15.0
MirrorCode43.3%61.1%-17.8
Mystery Game Puzzles42.0%80.0%-38.0
NYT Connections93.4%97.8%-4.4
OfficeQA Pro67.4%67.6%-0.2
OSWorld-Verified87.1%84.4%+2.7
ParseBench-171.2%69.6%+1.6
Terminal-Bench 4.159.7%60.9%-1.2
Terminal-Bench Hard69.0%69.4%-0.3
Vals SWE-bench95.3%97.4%-2.1
Terminal-Bench 2.1 (Vals AI)85.3%79.9%+5.4
VulcanBench v387.6%87.6%+0.0
CWE-Bench-AA60.8%63.3%-2.5
AA Cyber Index64.1%30.4%+33.7
DeepsecBench-AA36.7%30.8%+5.9
Harvey LAB-AA v1.112.5%6.9%+5.6
HealthBench Professional (length-adjusted, Artificial Analysis)60.0%62.5%-2.5
HealthBench Professional (raw, Artificial Analysis)67.9%70.4%-2.5
AA ITBench50.9%47.6%+3.3
Bug Hunt Bench36.0%44.3%-8.3
Chartography (tools)90.2%86.9%+3.3
CursorBench 4.046.7%54.7%-8.0
CWE-bench v156.4%63.9%-7.4
DeepSWE75.5%71.9%+3.6
DRACO86.9%87.9%-1.0
FrontierCode 2.019.9%39.6%-19.7
FrontierPhysics6.5%19.6%-13.1
HLE w/o tools51.1%59.6%-8.5
OSWorld 2.179.6%78.8%+0.8
OSWorld 2.1 Offline86.2%70.6%+15.6
PostTrainBench v1.139.5%41.9%-2.4
SpireClimb0.0%23.8%-23.8
StealthMark56.7%67.8%-11.1
SWE-bench Multimodal55.4%56.7%-1.3
Terminal-Bench 4.065.7%62.0%+3.7
Terminal-Bench-Science 0.138.1%57.0%-18.9
Toolathlon-Verified72.8%77.9%-5.2
VulcanBench Frontier v488.0%88.8%-0.8
AA AutomationBench73.3%74.0%-0.7
GDP.pdf31.4%27.0%+4.4
AA Terminal-Bench 4.067.2%60.5%+6.7
ARC-AGI-312.8%52.7%-39.9
ARC-AGI-3 Provider Adapter94.5%96.2%-1.6
cursorBench4152.2%44.8%+7.4
CWE-bench45.3%63.2%-17.9
MKA43.9%47.7%-3.8
SkillsBench official leaderboard (BenchFlow)56.9%59.5%-2.5
BinaryAudit35.8%35.8%+0.0
OTelBench22.5%22.5%+0.0
FACTS Multimodal46.4%52.6%-6.2
Hermes Bench73.2%73.9%-0.7
Hermes Index92.5%57.1%+35.4
ParseBench-278.4%76.2%+2.3
CharXiv93.4%87.3%+6.1
CharXiv w/o tools89.0%84.6%+4.4
ExploitBench74.7%99.7%-25.0
ExploitGym20.7%35.1%-14.4
AI2D_TEST93.5%94.2%-0.7
C-Eval93.5%93.4%+0.1
CountBench100.0%100.0%+0.0
DynaMath88.5%85.5%+3.0
ERQA79.1%68.8%+10.3
MathVision95.3%95.0%+0.3
MCP Atlas84.7%82.1%+2.6
MMLU-Redux97.1%98.7%-1.6
MMMU-Pro84.1%85.3%-1.2
NL2Repo63.2%61.6%+1.5
PostTrainBench v140.1%42.5%-2.4
RealWorldQA88.5%89.7%-1.2
ScreenSpot Pro85.0%89.1%-4.1
SuperGPQA100.0%100.0%+0.0
Toolathlon63.4%55.9%+7.5
V*100.0%100.0%+0.0
Video-MME (w/o subtitle)87.5%91.3%-3.7
Video-MME (with subtitle)92.0%91.8%+0.2
HealthBench Hard36.5%36.2%+0.3
MedXpertQA (MM)82.7%77.8%+4.9
React Native Evals81.3%78.5%+2.8
ResearchClawBench24.0%21.6%+2.4
PostTrainBench36.5%38.8%-2.3
DAYJOB4.7%15.5%-10.8
DAYJOB Healthcare1.5%10.8%-9.3
Finance Agent v257.6%62.3%-4.7
BioMysteryBench (human-solvable)89.4%89.5%-0.1
HealthBench (raw)59.4%56.7%+2.7
HealthBench (length-adjusted)58.3%58.5%-0.2
HealthBench Professional58.8%64.2%-5.4
HealthBench Professional (raw)69.5%67.2%+2.3
HLE-Verified54.4%55.4%-1.1
LABBench285.1%83.1%+2.0
ScarfBench69.6%68.4%+1.2
BlenderWorld (text-only)75.0%85.3%-10.3
Cookie-Bench85.2%89.7%-4.5
STEMBench2.6%2.8%-0.2
Agents' Last Exam43.2%51.7%-8.5
BabyVision w/ Python95.4%94.4%+1.0
OpenHarmony Bench v1.058.9%58.9%+0.0
OpenHarmony Bench v1.252.4%63.8%-11.4
SEC-Bench Pro77.3%78.8%-1.5
MRCRv286.0%85.9%+0.1
MMLongBench-Doc61.7%61.5%+0.1
SWE-Marathon51.7%59.1%-7.4
BioBasicsBench41.9%62.6%-20.7
LifeVal80.7%80.1%+0.6
MMMU-Pro w/ Python90.0%88.1%+1.9
ExploitBench Internal Port0.9%21.5%-20.6
SWE-Atlas Codebase QnA61.7%61.0%+0.7

General performance is the share of benchmark duels a model is expected to win against every other ranked model: a Bradley-Terry fit over every head-to-head result on the listed benchmarks, measured and estimated, each pair held against each other on the benchmarks both have a score on. A model measured only on benchmarks apart from the main field has no standing. The frontier is the strongest tenth. Every estimate carries its error and confidence on the model pages.