The Vibe Father
Autonomy check
Few products carried a bigger promise into the AI-coding era than the fully autonomous software engineer. Give it a ticket, walk away, come back to a merged pull request. Devin was the flag-bearer for that pitch, and the pitch itself — an agent that plans, writes, tests, and ships without a human in the loop — set the terms for how everyone talked about autonomous coding. Two years on, it's worth asking plainly: how much of the promise is real, where does it genuinely deliver, and where does it still hand you back a problem instead of a solution? We build a harness, not an autonomous agent, so we have no product in this particular race — which lets us be honest about it.
What "autonomous" actually buys you
Start with the good news, because it is real. On a well-scoped, self-contained task — the kind you could write a crisp one-paragraph ticket for — a capable autonomous agent genuinely earns its keep. "Add pagination to this endpoint and cover it with tests." "Migrate these three files off the deprecated API." "Reproduce this bug from the stack trace and propose a fix." Tasks with a clear definition of done, a small blast radius, and an obvious verification path are exactly where autonomy shines. The agent can iterate against the test suite, run the code, read the errors, and converge without you babysitting each step. That loop — write, run, observe, fix — is the thing autonomous agents do well, and when the task fits the loop, they save you real time.
This is not a small category. A meaningful slice of professional software work is well-scoped maintenance: dependency bumps, mechanical refactors, filling test coverage, wiring up boilerplate that follows an existing pattern. Point an autonomous agent at that pile and it will clear it faster than you would, because it never gets bored and never context-switches. The promise, narrowed to its honest core, holds: for tasks the agent can verify itself, autonomy works.
Where the promise thins out
The trouble is that the marketing rarely narrows the claim, and the gap between "autonomous on well-scoped tasks" and "autonomous, period" is where teams get burned. Three failure modes recur, and none of them is a matter of the model just needing to get smarter.
Ambiguity. The moment a ticket contains an unstated assumption — which of two plausible behaviors is correct, whose edge case matters, what the product actually wants — an autonomous agent has to guess. It will guess confidently, produce working code for the wrong interpretation, and hand you a clean diff that solves a problem you didn't have. A human engineer feels the ambiguity and asks. An agent optimized to finish the task plows through it. The result compiles, passes its own tests, and is wrong.
Architecture. Autonomy is strongest inside an existing shape and weakest when the task is to choose the shape. Deciding how a system should be structured, what the right abstraction is, which trade-off to make between two defensible designs — that's judgment that depends on context the agent doesn't have: the team's roadmap, the tech debt it's carrying, the constraint nobody wrote down. Let an agent make an architectural decision unsupervised and you inherit a decision no human signed off on, embedded in code that now works.
Verification. This is the deep one. An autonomous agent's confidence in "done" is only as good as the checks it can run — and a model grading its own work is the weakest evidence in the building. Passing the tests it wrote proves the tests it wrote pass. Whether the change is correct, safe, and actually what you meant is a separate question that the agent, by construction, can't answer about itself. The more autonomous the loop, the more this matters, because there's no human checkpoint between "the agent believes it's done" and "the change is in your codebase."
The oversight the honest version requires
None of this means autonomous agents are a dead end — it means the useful mental model isn't "hire a robot engineer," it's "delegate a well-defined task and review the result like you'd review a junior's PR." The teams getting real value from autonomy aren't the ones who walk away; they're the ones who scope tightly, let the agent run, and then verify externally before anything merges. The autonomy saves the typing and the iteration. The human keeps the judgment and the sign-off. That division of labor is where the productivity is actually banked.
We're wary of any number attached to autonomous performance, and we won't invent one for Devin or anyone else. Benchmark scores on well-scoped issue-fixing tell you something about the agent's ceiling on clean tasks, but they're measured on curated, verifiable problems — precisely the category where autonomy already works. They don't measure the ambiguous, architectural, hard-to-verify work where the promise breaks down, because that work is hard to benchmark for the same reason it's hard to automate. Read a leaderboard as "how good is this on the easy-to-verify slice," not as "how autonomous is this in your job." We keep our own scores, cited to source, at /benchmarks, and we're clear-eyed about what they don't cover.
How to actually use an autonomous agent
The practical playbook is unglamorous and it works. Feed the agent tasks with a crisp definition of done and a verification path it can execute — a failing test to make pass, a lint rule to satisfy, a reproduction to fix. Keep the blast radius small: one concern per run, reviewable in a sitting. Treat ambiguity as a stop sign — if you can't write the ticket without hand-waving, the task isn't ready for autonomy, it's ready for a conversation. And never let the agent's self-assessment be the last checkpoint; run your own tests, read the diff, and gate "merged" on your judgment rather than the model's confidence.
Do that, and autonomous agents become a genuine force multiplier on the large, boring, verifiable middle of your backlog — which is worth a lot. Skip it, and you get a very fast way to merge confident, wrong code. The promise was real; it was just narrower than the pitch. For the bigger version of this question — whether any of this replaces the engineer rather than the typing — we wrote will AI replace programmers, and the honest answer there is the same as the honest answer here: it replaces the parts you can verify, and needs you for the parts you can't.
