My laptop talked me out of RAG for coding agents

For a while my MacBook Pro (M1, 16 GB) could not keep three things alive at once: the IDE, a dynamic code indexer, and a Claude Code agent. One of them had to go, and it was the indexer. The agent’s code search got better after that, not worse. I didn’t start out anti-indexer. When coding agents became a daily part of my work, the standard advice was to give them semantic search. Embed the repo into a vector index, let the agent query it, get relevant chunks back. On paper that sounds right. Code search looks like a retrieval problem, retrieval is what RAG is for. So I set it up and moved on. It felt like a fair trade on the start - some background compute in exchange for a smarter agent.

The first cracks

The first thing I noticed was staleness. An index is current at the moment it’s built and starts drifting the moment one file changes. So I was resyncing it constantly. Half of the time an answer still came from how the code looked yesterday. For an agent that edits code all day, yesterday is a long time ago. It would grep-by-embedding its way to a function that had already been renamed, then confidently patch a call site that no longer existed. I resynced it, it drifted again. The second thing was precision. When I asked for an exact symbol, vector search kept handing me conceptually adjacent code instead - things that mean roughly the same, written with different names. That’s the whole design: embeddings retrieve by similarity of meaning. But most questions a coding agent asks are not similarity questions. They are exact questions. Where is this identifier defined, who calls it, when did this line appear. Answering an exact question with a fuzzy guess is a category error. And I was paying RAM and CPU for the privilege.

Which brings me to the third thing. Dynamic indexing (the kind that re-embeds as you edit) ate memory and CPU the entire session. My machine has 16 GB. With the IDE up and an agent running, the indexer was the one always sitting near the top of Activity Monitor, holding memory for answers I had already stopped trusting. Eventually the machine just could not hold all three. Something had to go. The indexer was the only one not doing exact work.

What replaced it

Nothing exotic replaced it. I wired plain CLI tools into the agent, the kind that read the filesystem as it is right now, so the agent can call them the same way I would from a terminal:

rg 'handleRefund' -t ts                 # exact text, current state, no index
ast-grep -p 'catch ($ERR) { $$$ }'      # a shape of code, not a string
git log -S handleRefund --oneline       # when this name appeared or left

Speed was my first worry, turned out to be a non-issue - ripgrep scans the Linux kernel tree (75,000 files) in about 0.08 seconds where GNU grep needs about 0.67, and it does that with no daemon and no sync job, it reads the actual files on every call so there is no staleness window and nothing to babysit. The kit goes deeper than grep too. ast-grep matches code structurally (it parses with tree-sitter, and patterns are ordinary code with $VAR wildcards). A formatting change can’t hide a match from it the way it can from a regex. git log -S is a search axis an embedding index doesn’t even have: it finds the commits where the count of a string changed. That answers “when did this appear and who removed it”. Its sibling -G catches lines that merely moved, which -S misses because the count stayed the same. The language server answers the questions text search can’t: find references through re-exports and barrel files, resolve the overload, rename safely across aliased imports. So that’s four kinds of tool for four kinds of question, and every one of them answers exactly.

The part I underestimated

What I underestimated is that code questions are iterative, and that matters more than any single tool does. A typical chain: grep a function name, get zero hits because someone renamed it, run git log -S on the old name to find the rename commit. Then grep the new name. Ask the language server for references to catch a call site hidden behind a re-export. Four steps, all of them cheap and exact. No single retrieval pass could return that. The answer didn’t exist as a chunk anywhere in the repo, it only exists at the end of that walk. One-shot retrieval bets everything on the first query being right. An agent with tools gets to be wrong three times on the way to being right. That is how people search too.

I’m not the only one who landed here. In a Pragmatic Engineer interview, Boris Cherny described how the Claude Code team tried vector-store indexing and model-driven indexing. Plain glob plus grep with iterative refinement worked better for their use case. And the direction has peer-reviewed support. The SWE-agent paper (NeurIPS 2024) reports an agent with tool access resolving 12.47% of SWE-bench issues where a non-interactive retrieval-augmented baseline managed 1.96%. That is one team’s account and one benchmark, not an industry verdict, I want to be careful there. But it matched what my own machine had already told me.

Where this doesn’t hold

No head-to-head benchmark exists that isolates CLI-tool search against embedding search on identical coding tasks. I looked. My claim rests on architecture and on my own experience, not on a controlled study. Weight it accordingly. There are also real limits. Every ripgrep call is a linear scan. The cost scales with repo size on every query, that is the price of zero staleness. On a truly huge monorepo that math can shift. Language servers need warmup on big repos, sometimes 30 seconds, sometimes minutes, before they answer anything. And to be fair, a language server is an index too, it holds the project graph in RAM - I keep it because it answers exactly, which is more than the embedding index ever did for me. And there is one question shape where semantic search has a genuine story: “find the code that deals with retries, whatever it’s called here.” That is a similarity question, the kind embeddings were built for. My answer is usually ast-grep plus reading, and it works. But I’ll admit the vector index would have had a shot at answering it in one step.

If your machine is comfortable and your index somehow stays fresh, I’m not telling you to rip anything out. Mine wasn’t, so I did, and I haven’t put it back. What replaced the indexer is a handful of small tools that read the code as it is right now, plus an agent that’s allowed to search the way people search - ask, read, adjust, ask again. For the questions a coding agent actually asks, that turned out to be enough. And my laptop runs everything at once again.