The Vibe Father
Explainer
Every model leaderboard asks for your trust; very few explain why they deserve it. This post is our explanation. The Vibe Coding Index — the composite score behind the rankings at /benchmarks — is a weighted blend of three public benchmark suites, and every design decision in it optimizes for one thing: predicting how a model will actually perform when you hand it real coding work. Here is exactly how it is built, and why.
The formula
The index is a weighted average of three suites:
| Suite | Weight | What it measures |
|---|---|---|
| SWE-bench Verified | 40% | Real GitHub issues in real repos, graded by the repos' own tests |
| Terminal-Bench | 30% | Driving a real shell: multi-step ops, installs, builds, debugging |
| LiveCodeBench | 30% | Fresh contest problems collected after training cutoffs |
Why these three, at these weights
SWE-bench Verified gets the largest share, 40%, because it is the closest public proxy for the job itself: 500 human-verified-solvable issues from real repositories, where the model must produce a patch that passes the project's own test suite. Nothing else in public benchmarking gets as near to "can this model fix an actual bug in an actual codebase without breaking things." Its limits — Python-heavy, bounded tasks — are real, which is why it is 40% and not 80%. Full breakdown in our SWE-bench Verified explainer.
Terminal-Bench gets 30% because modern coding models are agents, and agents live in shells. The benchmark measures a model actually operating a terminal through multi-step tasks, and it captures a skill that pure coding scores demonstrably miss — GPT-5.5 tops this suite while trailing badly on SWE-bench, and Haiku 4.5's 35.5 exposes an agentic cliff its coding scores hide. Details in our Terminal-Bench explainer.
LiveCodeBench gets the other 30% for a different reason: contamination resistance. It continuously collects contest problems published after models' training cutoffs, so a high score cannot be memorization. It measures algorithmic ability rather than repo skill, and it is our earliest honest signal on brand-new models. See our LiveCodeBench explainer.
Together the three triangulate: real repository work, agentic shell operation, and un-memorizable fresh reasoning. A model has to be genuinely good to score well on all three, because they cheat-proof each other.
Every score is cited — none are invented
Two rules we treat as non-negotiable. First: every score on the leaderboard is cited to its public source, with a link, right on the board. We do not launder numbers into an unsourced table; you can click through and verify whose measurement you are looking at.
Second: missing scores are never invented, estimated, or — worse — counted as zero. Labs publish benchmarks on their own schedules, and a model that shipped last week (Grok 4.5, right now, has only a LiveCodeBench score) should not be punished for a number that does not exist yet. When a suite is missing, we renormalize: the model's index is computed from the suites it has, with their weights rescaled proportionally. A model with only SWE-bench and LiveCodeBench published is scored on those two at a 40:30 ratio, and the leaderboard plainly marks which scores are absent. A missing benchmark never counts as zero — it counts as unknown, because that is what it is.
Kept fresh, and pruned honestly
The data refreshes nightly. When a lab publishes a new score or a suite posts new results, the board moves the next day — no quarterly editorial cycle, no stale screenshots.
Freshness also means retiring suites that stop earning their place. As of July 2026 we retired two: Aider Polyglot, whose public board froze in late 2025 and stopped covering new releases, and HumanEval+, which has effectively no 2026 model coverage. Both were fine benchmarks in their day, but a suite that cannot score this year's models can only distort a 2026 ranking — old models keep their scores, new models show gaps, and the composite quietly rewards being old. Cutting them was the honest move, and it is why the index is three suites now instead of five.
What the index is not
It is not a verdict on your use case. The index deliberately excludes price, speed, and context window — those are on the board as separate columns, because "best" and "best for you" are different questions. It cannot capture code taste, instruction-following on your stack, or how a model behaves in hour three of a session. And it inherits every caveat of its inputs, including the fact that agent scaffolds influence published scores. Treat it as the best available shortlist generator, then test the shortlist on your own work.
The live board is at /benchmarks, and the standing methodology page — kept current as weights and suites evolve — is at /benchmarks/methodology. When our methodology changes, we will say so there, in writing, before the rankings move.