How to Document Code With AI (Accurately)

Playbook

AI writes beautiful documentation for functions that don't exist. It will confidently describe a retryWithBackoff option your library never had, document a parameter that was renamed two versions ago, and give you a "usage example" that imports from the wrong path. The prose is fluent, the formatting is clean, and it's wrong in ways that are expensive to discover — because bad docs don't crash, they mislead. Someone reads the confident sentence, builds on it, and finds out the hard way.

Documentation is the one place where an agent's fluency is a liability. It's very good at sounding like accurate docs. Your entire job is to force every sentence to be grounded in the code that actually exists. Here's how we do it.

The core rule: ground everything in real code

The failure mode has a name — the agent generates plausible API surface from patterns it's seen in similar libraries, not from your source. So the fix is a hard constraint: the agent may only document what it has read in the actual code, and every non-trivial claim must cite where it came from. No memory, no "this is probably how it works," no filling gaps with convention.

Step 1 — Point the agent at the source, not the vibe

Don't ask "document the auth module." Give the agent the actual files and tell it to read them first. Its first output should be a list of the real public surface it found — the exported functions, their real signatures, the real parameter names and types — each with a file:line citation. If it can't cite where a function is defined, it doesn't get to document it. This single step kills most invented APIs before they reach the page.

Step 2 — Document signatures from the code, verbatim

Parameter names, types, defaults, and return shapes must be copied from the source, not paraphrased from what the agent thinks the function probably takes. This is where subtle rot creeps in: a param renamed from timeout to timeoutMs, a default that changed from true to false, an optional argument that became required. The agent's training data remembers the old shape; your code has the new one. Force it to match the code.

Step 3 — Every example must actually run

A usage example is a claim that this code works. So verify it like a claim: the agent runs the example, or writes it as a test, and pastes the real output. An example that imports from a path that doesn't exist, or calls the function with the wrong argument order, is the most common and most embarrassing doc bug. If it can't be executed, it can't be published.

Document <module>. Rules:

1. READ the actual source files first. List the public API you
   found - functions, signatures, param names/types/defaults,
   return shapes - each with a file:line citation. Do not include
   anything you cannot cite.

2. Copy signatures VERBATIM from the code. Do not paraphrase
   parameter names, types, or defaults from memory.

3. Every usage example must actually run. Execute it (or write it
   as a test) and paste the real output. Remove any example you
   cannot run.

4. Flag anything ambiguous instead of guessing. If behavior isn't
   clear from the code, write "UNVERIFIED" and ask, rather than
   inventing an explanation.

Do not describe features, options, or parameters that are not
present in the source you read.

Step 4 — Verify every claim, then trust none of them

After the draft, do a fact-checking pass — ideally with a fresh agent that didn't write the docs, so it has no loyalty to its own sentences. Its job is adversarial: for each claim, find the code that supports it or flag it. This is the same cross-model discipline that powers good review; the reviewer's skepticism catches what the author's fluency papered over.

Where AI docs pay off — and where they don't

The pattern: AI documents specific code it has read well, and hallucinates the moment it's asked to describe something from a distance. Keep it close to the source.

A documentation checklist

1. Did the agent read the actual source, and cite file:line for the public surface?
2. Are signatures, param names, and defaults copied from the code, not paraphrased?
3. Did every example run, with real output pasted?
4. Is anything ambiguous marked UNVERIFIED rather than guessed?
5. Did a second pass fact-check each claim against the code?

Keep docs from rotting

Documentation lies most when the code moves and the docs don't. The durable fix is to generate docs from the code close to when the code changes — regenerate the affected docstrings in the same session you change a signature, so the two never drift apart. When you review a change, treat stale docs in the diff as a finding, exactly as you would in the AI code review workflow.

Documentation is cheap to generate and expensive to trust, which is why grounding matters more here than anywhere. A value model like Sonnet 5 handles docstrings and READMEs cleanly; save a stronger model for prose that explains genuinely subtle behavior. If you're documenting an API you just built, keep the docs honest by verifying them the same way you verify the code — see writing tests with AI that actually catch bugs, and for the security angle on what you expose, the AI coding security checklist. The Vibe Father's memory keeps a grounded picture of your codebase so docs draw from what's really there, but the rule above — cite it or cut it — needs nothing but discipline.