The Vibe Father
Explainer
Every benchmark has a shelf life problem: the moment it becomes famous, its problems and answers start leaking into the training data of the models it is supposed to measure. A model can "ace" a benchmark by having memorized it. LiveCodeBench is the suite built specifically to dodge that trap — and that contamination resistance is why it earns 30% of our Vibe Coding Index despite measuring something narrower than its siblings.
The core idea: stay ahead of the training data
LiveCodeBench continuously collects new programming problems from competitive-programming platforms — the LeetCode/Codeforces/AtCoder style of contest problem — and, crucially, tags each problem with the date it was published. When evaluating a model, you score it only on problems that appeared after that model's training cutoff. By construction, the model cannot have seen the answers, because the problems did not exist when it was trained.
That is the whole trick, and it is a good one. A static benchmark measures a blend of two things you cannot separate: ability and exposure. A rolling, dated benchmark isolates ability. When Gemini 3.1 Pro posts an 88.5 on LiveCodeBench, that is 88.5 on problems that postdate its training — genuinely fresh reasoning, not recall wearing a costume.
What it actually measures
Contest problems are a specific genre: self-contained algorithmic puzzles with precise specifications and automatically checkable answers. Parse this input, find the optimal arrangement, count the paths, do it within time limits. Solving them well requires real skills — algorithmic reasoning, edge-case discipline, translating a tricky spec into correct code — and the grading is merciless in the best way: your program either produces the right outputs or it does not.
What contest problems are not is software engineering. There is no codebase to navigate, no legacy conventions to respect, no test suite to avoid breaking, no ambiguity about what "done" means. LiveCodeBench measures algorithmic chops, not repo surgery — and the scoreboard shows those are different muscles.
The canonical example: Gemini 3.1 Pro
Look at one model and the whole distinction snaps into focus. Gemini 3.1 Pro scores 88.5 on LiveCodeBench — second on our entire board, ahead of Opus 4.8's 87.8 and behind only Fable 5's 89.8. Its SWE-bench Verified score is 75.6 — a thirteen-point gap, and well behind models it beats on LiveCodeBench. Same model, two benchmarks, two very different verdicts.
Neither number is lying. Gemini 3.1 Pro is genuinely excellent at fresh, self-contained algorithmic problems, and genuinely mid-pack at navigating a real repository. If your work is data-structure-heavy leaf functions, the 88.5 is the number that describes your experience; if your work is threading changes through an existing system, the 75.6 is. We unpack this profile in our Gemini 3.1 Pro review. The general lesson: a high LiveCodeBench score with a lower SWE score is a specific, recognizable model personality — brilliant puzzle-solver, unproven surgeon.
The current LiveCodeBench top five on our board: Fable 5 at 89.8, Gemini 3.1 Pro at 88.5, Gemini 3.5 Flash at 87.6, DeepSeek V4 Pro at 87.5, Grok 4.5 at 87.4. Notice how compressed that is — barely two points across five models from four labs — and note that for brand-new Grok 4.5 this is the only benchmark score published at all so far, making LiveCodeBench the earliest honest signal we get on new releases.
The caveats
Three, in the interest of honesty. First, the narrowness cuts both ways: contamination-free measurement of a skill that is not the whole job. A model could in principle grind contest-style training and inflate here without getting better at your codebase. Second, the rolling window means scores are computed over different problem sets for different models — that is the design working as intended, but it adds comparison noise a static benchmark would not have. Third, difficulty of the incoming problem stream varies over time, which the maintainers manage but cannot fully eliminate.
How we use it
LiveCodeBench carries 30% of our Vibe Coding Index — equal to Terminal-Bench, behind SWE-bench Verified's 40%. Its job in the blend is to be the contamination-resistant fresh signal: SWE-bench tells us about real repository work, Terminal-Bench about agentic shell competence, and LiveCodeBench keeps everyone honest with problems nobody could have memorized. Every score on our board is cited to its public source, and missing scores renormalize rather than count as zero. The full weighting rationale is in how we rank coding models; the live numbers are at /benchmarks.