←新闻详情

Modem:Blog更新于 10月11日 07:29

AI 编程智能体如何阅读你的代码:Modem 谈用 grep 提升代码可发现性

How AI coding agents read your code | Modem Blog

AI 导读

Modem 在博客中指出,Claude Code、Codex、OpenCode 等主流编程智能体底层主要靠 grep/ripgrep 做文本搜索来导航代码库,而非依赖编译器依赖图或语言服务器。在其约 2900 个 TypeScript 文件的项目中,搜索 create 会返回 1585 个无关命中,而 createStripeClient 仅 43 个且全部相关,命名越具体越能减少 token 消耗与智能体轮次、提升 bug 检出率。

正文

当前显示原文,中文全文译文尚未就绪。

Part 1 of Writing Code for Agents: how agents navigate your repo by string search, and how to write code they can actually find, parse, and trust.

When we founded Modem in early 2025, we believed that AI coding was real, that it would rapidly accelerate software development, and that legacy product processes would become the new bottleneck for execution.

To commit to that future, we built Modem with AI codegen tools from day one - back when Sonnet 3.7 was the leading coding model. A year later, Modem is a real product with paying customers: 360,000 lines of TypeScript application code and another 320,000 lines of test code, 99.9% of it generated by LLMs.

Getting Modem to a functional codebase wasn't easy. It required experimentation, trial and error, and continuous investment to work effectively with agentic codegen. In this blog post series, Writing Code for Agents, we're going to share some of the tips we've learned along the way.

Part 1 - this post - covers how to write your code so that agents navigate, parse, and read it more effectively: the names you choose, the types you define, and where you put your explanations. The results aren't hypothetical; in our tests, following the tips in this guide led to fewer tokens and fewer agent turns spent performing code retrieval, a higher rate of bug detection, and fewer confidently wrong answers.

But to understand how that's possible, we need to first understand: how do agents even search code?

How agents search code (spoiler: it's text search)

Your coding agent frequently searches your repository for code. For example, want to rename a function? Claude Code, Codex, OpenCode, etc. will all search your code to find all the references to that function that might need updating.

Under the hood, most coding agents are just running grep – or rather, its faster cousin ripgrep, aka rg – to rip through your code and find matching text references. The agent reads what comes back, and if it isn't enough context, repeats scanning your code until it has what it needs.

Looking up text strings in files isn't the only way agents search: they also search by filename too. For example, if you ask a coding agent, "how does the session broker work?", agents are likely to scan your repository for lookalikes:

So paths are search terms too. A directory named session-broker/ is a hit before the agent reads a single line of code.

Notice it's all just strings. There's no compiler feeding the agent a dependency graph, and no language server resolving symbols (usually, more on this below). The Claude Code team tried embeddings and a vector database early on and threw them out because plain text search worked better. Academic agents landed in the same place; SWE-agent got its results by giving the model purpose-built keyword search tools and nothing fancier.

Even frontier models, given capable tools, typically navigate your code with what is essentially grep. And notice that the loop has one of the most important inputs the code's author controls: whether the words in your code and the names on your files make good search terms.

What about language servers (LSPs)?

Editors solved code navigation with the Language Server Protocol years ago, and coding agents can use LSP servers to resolve code symbols (some natively and some via MCP). But that requires extra config everywhere you run your agents: an LSP server per language, on every machine and for every checkout. And even if you have that, it's unclear if the results are any better.

grep / rg is what most coding agents use out of the box, and the focus of this blog post.

Not all code is equally discoverable

Let's consider 3 hypothetical functions. They all do the same thing: create an API client object that makes HTTP requests to an external service (e.g. Stripe).

The only difference is how they're named:

And here's how those functions might get invoked elsewhere in the codebase:

Let's say you're working on this code with an agent. You want to change the API client. Say, by adding a new method that calls a new HTTP API endpoint you've introduced. The agent will naturally try to discover all of the locations where this object is used. As we learned earlier, it will use grep/ripgrep.

Here's how many matches grep will return for these strings in our monorepo — about 2,900 TypeScript files — and how long each search takes using ripgrep (rg) on an M5-class MacBook.

The 1,585 hits for create span hundreds of unrelated things: test fixtures, database inserts, workflow registrations. The 43 hits for createStripeClient are the definition, the call sites, and the tests — all about the same client.

Try it on your repo

Run ripgrep to search for symbols in your codebase:

You probably also noticed that ripgrep is fast. There's no meaningful time difference searching for create (50ms) or createStripeClient (47ms). But that's not the issue. The issue is how those hits affect your agent's context.

Poor code discoverability eats context

Grep hands back matching lines, not answers. A hit like const client = create(config) doesn't tell the agent whether this is the client you care about or one of the hundreds of other things named create.

To find out, the agent reads a window around the hit, and often expands or opens additional sections. That can add up to anywhere from a few dozen lines to hundreds or thousands of lines of code.

At roughly ten tokens a line, each file read is hundreds to thousands of tokens spent to rule out one wrong candidate. Work through a dozen and you've burned tens of thousands of tokens separating signal from noise, before a line of the actual task is written.

Whereas searching for the specific name skips the sifting:

Three files, all about the same client. And the saving compounds. Context is finite, so every token spent ruling out create fixtures is one the agent no longer has for your change. Worse, as its context fills with near-misses, models get measurably less reliable at using what's actually in there. A name that grep resolves in one hop keeps both the agent's attention and its budget on the task.

names affect more than retrieval

Researchers have shown you can alter a code model's predictions just by renaming identifiers without touching the logic. Another benchmark found that misleading names drop model accuracy on code reasoning tasks by around 23 percent, and that chain-of-thought reasoning only recovers part of the loss.

The models trust what the code says it does. Humans do this as well (an old study found developers locate defects 19% faster when identifiers are full words), but a person eventually builds up immunity to your codebase's lies. The agent shows up fresh every session and believes them all over again.

Modules won't save a generic name

So far we've been talking about pure symbol lookup. But what about imports and modules?

Let's return to our earlier client TypeScript example:

At the top of this file is a pretty clear clue about the location of the client function definition. Why grep through the codebase when we can just look up the file here?

If the goal was strictly to find the definition, you'd be right. An import is a forward pointer: given a call site, it tells you where the definition lives.

But the task that opened this post was a rename. That's the reverse direction — given a definition, find every caller — and plain-text search provides no reverse index. Nothing in api.ts records who imports it. (Some languages are better at this; more on this below.)

Reverse symbol lookups

Say create has three call sites, each using an ordinary import style:

Now try to find them all by text:

Neither query above finds all three cleanly — except a name distinctive enough that grepping it returns only real usages. The import graph exists in your code, but it isn't greppable in reverse. For a grep-first agent, a distinctive symbol name is the most portable reverse lookup.

Barrels and package specifiers

Even the forward direction — call site to definition — degrades in a monorepo. The import at the top of our file says create comes from @modem/common. But @modem/common isn't a path; it's a package specifier. To follow it, the agent first has to find out where that package lives, which means opening package.json or grepping for the package name.

That lands it on the barrel file:

Which re-export has create? The barrel doesn't say. The export * erases the names it forwards, so the agent is back to grepping inside the package to find the actual definition. The same search it could have started with, now 3 hops later.

Methods on objects

Methods never had imports at all. Look at the call we've been working with:

No import statement anywhere mentions getUsers; imports traffic in modules and their top-level exports, and a method is neither. The only way a grep-first agent finds where getUsers is defined, or who calls it, is searching the name. And unlike the two failures above, this isn't TypeScript's fault: client.getUsers() leaves the same nonexistent trail in Python, Go, Ruby, and Java. Without semantic tooling, method names are usually the best textual handle an agent has.

The research on grep vs dependency graphs

You'd expect a parsed dependency graph to beat dumb text search. But one recent study found otherwise. GrepRAG (Jan 2026) compared LLM-driven ripgrep against retrieval systems built on real import graphs, like GraphCoder, on repository-level code completion benchmarks. Grep matched or beat them. Where grep did fail, the paper's failure analysis points at generic names: search for a term like init and you get back hundreds of matches that contain the keyword but not the answer. Sound familiar?

Modules point one direction, blur behind barrels and package specifiers, and say nothing about the methods on your objects. The name still does the lifting.

Types force better discoverability

Names help the agent find your code; types tell it how to use that code. And unlike a comment or a README, a type is enforced.

Put another way: the agent may skip your docs. But when it runs the compiler, type errors provide fast, concrete feedback. An invalid call produces an error the agent can use to correct its approach - often within one turn.

Before the compiler catches anything, though, types are already paying for themselves, because a precise signature can often answer the agent’s first questions without requiring it to read the implementation. Consider this one:

The agent sees this in a grep result and keeps moving. What goes in (User), what comes out (EnrichedUser). Depending on the question being asked, there may be no reason to open the function body. Now the untyped version:

To learn what data is, the agent likely has to read the implementation. If the implementation passes data along to something else, it reads that too, and probably a caller or two. One missing annotation can turn a one-line read into several file reads.

Note that TypeScript's any does the same damage while looking like a type; every any increases the chance that the agent will need to read the implementation.

Type quality matters

So just the presence of types can reduce agent reads. But the descriptiveness of those types also matters.

Consider this function:

Since every argument is typed string, the compiler doesn't catch an accidental argument swap. But what if instead the IDs have distinct types (branded types in TypeScript, newtypes in Rust)?

Now the swap becomes a build error:

The agent hits that error, fixes it, and moves on, usually in one turn. Researchers have watched this loop work: one study of compiler feedback saw models roughly triple their success rate over a handful of error-and-retry rounds.

Here's where it ties back to everything above. That error names a type — OrgId — and OrgId is almost certainly defined in another file. To act on the error, the agent has to go read that definition, and it gets there the only way it knows how: it searches the name.

A type name is a search term exactly like a function name, and it obeys the same rule. OrgId or UserEnrichmentResult greps to one definition; a Result, a Data, a Config drops the agent back into the multi-hit haystack from earlier. The compiler tells it where to look; the name decides how long the looking takes.

Which is the full case against any: nothing for the compiler to say, nothing for the agent to search. A bad name at least leaves a trail.

What's the sweet spot for uniqueness?

We measured this in Modem's TypeScript repo — 7,922 exported names, checked whether grepping each one lands on a single definition:

Three words is roughly where a name stops being ambiguous and starts working as an address. Give exported things two to three words where one is a domain word — e.g. diffUserObjects, not diff.

Note these thresholds come from a flat TypeScript export namespace. In qualified-call languages like Go, the package name counts as one of the words — e.g. stripe.NewClient is already a three-word token.

Smaller applications of the same idea

By now the pattern is clear: make the words in your code good search terms. Most of what's left is the same idea applied in smaller places.

Put the comment on the definition. An agent reaches your code by searching a name, and the search resolves to where that name is defined — so the definition is the one spot you can count on it reading. That makes the one-line comment directly above a definition the most cost-effective documentation you can write: it sits exactly where searches land. Write a sentence about the thing the code can't say itself.

Pick one spelling per concept. Try to name distinct concepts uniquely. For example, don't call it organizations in one place and customers somewhere else - even using a local import alias. Even synonyms in your codebase can lead to confusion (e.g. orgId vs organizationId), but agents seem to navigate those more successfully.

Name tests after the source they cover. We intentionally name test files after the code they cover (e.g. stripe.test.ts to test stripe.ts). We've observed that without obvious names, agents take extra turns to find a match.

Mark legacy paths @deprecated. If you're keeping legacy code around for one reason or another, be sure to mark it as deprecated (e.g. using TypeScript's @deprecated tag). Otherwise agents will discover and use code you don't want them to. Better yet, strive to remove the code entirely.

Write the conventions file. You probably already have an AGENTS.md or CLAUDE.md. Make sure to record anything non-obvious, like naming conventions, source code locations, etc. It's the highest-leverage habit for ensuring agents navigate your code effectively.

Is this all vibes? Let's go to the tapes

So far, we’ve made the case in theory. But does it make a difference in practice?

To try and answer that, we had Fable 5 distill this blog post into a write-discoverable-code skill file - a one-page set of rules loaded into the agent's context when it writes code. We then compared how other agents retrieved and reasoned about code produced with and without those rules, in a set of isolated experiments. You can install that skill from agent skills for your codebase.

One caveat before we get into the results: this is surprisingly hard to test cleanly. How easily an agent navigates a codebase depends on the code itself, the task, the model and harness, and plenty else. Plus the advice in this post (captured in the skill file) changes several things at once: names, files, types, structure, etc. It's a lot of variables.

These aren’t meant to be definitive benchmarks. They’re smaller, scoped experiments asking a simpler question: when the task and surrounding repo stay roughly the same, is code written for discoverability easier for search-driven agents to navigate?

Experiment 1: Generated library (TypeScript)

The goal of this experiment was to see if brand-new agent-generated code using our skill file led to better agent retrieval results vs without it.

Two author agents each write the library twice—unassisted and with a skill file—producing four code variants. Each variant is buried in the same 500-file repository. Fourteen separate reader configs then search and read those repositories; the readers never receive the skill file.

To do that, we started with a neutral spec for a small notification pipeline library, featuring retries, rate limiting, payload signing, etc. Then we gave it two different author agents: Haiku 4.5 in Claude Code and GPT-5.6 Sol in Codex CLI. Each author generated the library twice: once unassisted and once with the write-discoverable-code.md skill loaded. That gave us four versions of the same library:

Haiku 4.5 (Claude Code) - unassisted; Haiku 4.5 (Claude Code) - with skill file loaded; GPT-5.6 Sol (Codex CLI) - unassisted; GPT-5.6 Sol (Codex CLI) - with skill file loaded

We chose Haiku 4.5 as an intentionally small author model, and its unassisted version produced exactly the kind of code this post warns about. With the skill loaded, the difference showed up immediately in the names:

On the other hand, GPT-5.6 Sol’s unassisted code was already shaped for discoverability: calculateBackoffDelay in backoff.ts, TokenBucket in token-bucket.ts. In this run, Sol produced specific, discoverable names without any additional guidance.

With the skill loaded, Sol went further – e.g. backoff.ts became notification-retry-backoff.ts – but the gap between its two versions was visibly narrower than Haiku’s. (As a matter of personal taste, I felt this code was arguably over-specified.)

Next, we tested how other agents read the four library versions. We buried each one among the same 500 files of real code from our monorepo (to see the hypothetical impact in our existing code). We then started fresh agent CLI sessions restricted to search and read tools, and asked questions like:

Fourteen model/harness combinations answered 10 questions three times against each version: 1,680 runs in total. For each run, we recorded token spend and whether the answer was correct.

Author 1: Haiku 4.5 (Claude Code)

Author 2: GPT-5.6 Sol (Codex CLI)

Read together, the tables show a larger effect where the unassisted author produced more generic code. In these runs, the median reduction was 32% for Haiku-authored code and 12% for Sol-authored code. Twelve of the fourteen Sol reader configurations improved; two did not. The skill acted like a floor in these runs when the author left more room for improvement, though the experiment doesn’t isolate naming from the other changes introduced by the skill.

Reader behavior also mattered. One row deserves its own sentence: Sol reading its own unassisted code still got 13 to 38 percent cheaper (depending on the harness driving it) when that code was written under the skill. The model that wrote the code has no privileged memory of where things live. It runs the same searches everyone else does.

The wrong answers, meanwhile, turned out to belong to one table only. On Haiku's generic code, unassisted sessions returned confidently wrong answers - asked about the notification library's delivery history, an agent pointed at a Slack delivery ledger elsewhere in the repo; asked where the HMAC signature gets attached, two pointed at our media-proxy's URL signer. The skill-named code produced zero, because near-miss code can't wear a name like hmac-payload-signer.ts. On Sol's code this failure mode essentially vanished in both versions (834 of 840 runs correct, the errors one-off and split across both arms). In these runs, the confidently wrong answers clustered in the generic-named Haiku version.

Experiment 2: PewDiePie's Odysseus repo (Python)

Synthetic codebases only show so much, so we also tested the idea against a difficult real-world case: Odysseus’s unusually large email subsystem. Odysseus is the self-hosted AI workspace that picked up tens of thousands of GitHub stars in its first week — an impressive thing for a self-taught coder to ship — and routes/email_routes.py (see on GitHub) is 4,943 lines, about 3,900 of them a single function holding all 81 route handlers as nested closures. One function, the entire email client.

We refactored just that one file (email_routes.py) using Claude Code + Sonnet 5 under the skill - same behavior, same routes, verified mechanically - reorganized into concept-named modules:

Before

After

Then asked a mix of agents / harnesses the same six questions about both versions:

Here's how each model / agent harness performed. (Fewer model/harness combinations this time because we're not made of infinite credits.)

The pattern from the generated-library tables shows up again: the models that were lost in the monolith saved the most. Haiku had been burning 400,000-token sessions paging through one wall of code; after the refactor, it stopped. Grok saved a third. Sol and Sonnet were never really lost, and broke roughly even.

The traces suggest filenames were a major contributor, although the refactor also changed file sizes and module boundaries. When the agent is looking for email folder logic, and it lives in a reasonably-named file like email_folders.py, the agent finds it without opening anything else.

But the refactor's bigger effect is on the worst case. Nearly every catastrophe run - e.g. a 400k-token search that ends with no answer - happened in the original monolith. Nearly, because Haiku produced one in the refactored version too, and the location is telling. Our experiment only refactored a single file, and Haiku got lost in a related-but-unaltered 1,800-line email_helpers.py grab-bag. The refactor raised the floor, and the new floor became the worst file left standing.

Grep can't find code that doesn't exist

Question six in this experiment intentionally had no answer. It asked where received email HTML gets sanitized, and that's nowhere, it never is. That question got more expensive after the refactor - a well-organized codebase keeps offering plausible places to look next, so agents spend their whole budget before concluding "this doesn't exist." Grep can find code; it can't find the absence of code. If your codebase deliberately doesn't do something a reader would expect, write that down where they'll go looking for it.

Experiment 2.5: Odysseus bug hunt

One last test: does legibility change what agents get right, not just what they spend? We planted four subtle bugs in two versions of that email code - byte-identical edits, so each version contains exactly the same defects - and asked four models to verify the relevant requirements under a fixed eight-turn review budget. Each model reviewed each bug twice.

The newest and most advanced frontier models we tested, Grok 4.5 and GPT-5.6 Sol, caught every bug in both versions. Eight turns is enough for a disciplined searcher to grep its way to any of these defects, even inside a 4,943-line file.

But Haiku and Sonnet couldn't pull off the same feat: they caught 4 of 8 and 3 of 8, and the failures were rarely wrong answers - they were reviews that burned the entire budget paging through the file and never reached a verdict at all. In the reorganized code, both went a perfect 8/8.

The Sonnet row is also worth noticing: in the earlier Odysseus retrieval test it barely benefited from a refactor at all. And under a review budget its accuracy still went from 3 of 8 to 8 of 8. Being able to afford lookups in a monolith is not the same as being able to reach conclusions in one.

Across these experiments, easier retrieval left agents more budget to do the actual work. That showed up as cheaper lookups, fewer wrong answers, and more bugs caught before the turn limit.

So, should I just use this skill file?

These experiments are illustrative, not definitive: one generated library, one real-world refactor, and a limited set of models and harnesses.

What we do think it shows, is that there is real merit to the concepts in this blog post. But your mileage may vary, and we don't believe the answer is to just blindly toss a code-rewriting skill file to your agent and hope for the best.

We believe code needs to work for humans and agents, and that you need to make the right tradeoffs for you, your team, and your code. We encourage you to experiment with some of the ideas in this post and see if they work for you. Note many of our tests involved our Modem code repository - we wanted to know if this stuff works for us, not just in a lab.

Wrapping up: yes, the code matters

To recap: agents navigate your codebase by string search, they trust what your names tell them, the compiler is the one reviewer they can't ignore, and what they read first is largely determined by where search lands. And when we put that to the test in a series of experiments, it held up.

In our synthetic generation test using a weaker code author (Haiku 4.5), all fourteen model/harness pairs read the skill-written code for fewer tokens - between 6% and 66% fewer - and confidently wrong answers disappeared. On the stronger author's code (GPT-5.6 Sol), 12 of 14 model/harness pairs performed better, and the remaining two did not improve.

In our experiment involving an existing vibe-generated repo (Odysseus), the same refactor cut token spend by about a third for the models that were getting lost — and broke even for the ones that weren't. In the bug hunt, reviews went from 23 of 32 correct in the monolith to 32 of 32 in the refactor, with the two weaker models going from 7/16 to a perfect 16/16, because they spent their budget judging instead of finding.

We don't want to oversell the data. Our repo measurements are observational, and the controlled experiment is one spec and a couple thousand runs. Harnesses will keep evolving too - Cursor has published respectable numbers showing embeddings help on top of grep, so plain text search may not be the endgame. But grep, repo maps, and embeddings all feed on the same input: the words in your source. Better names improve all of them. That's about as safe as bets get in this business.

Prebuilt indexes bring a failure mode of their own. Why Devin struggles once a codebase gets past 15-20k lines looks at what happens when the repo map stops covering the repo.

The funny thing is that almost none of this is new advice. Descriptive names, precise types, comments that explain the why - we've been telling each other to do these things for decades, and mostly forgiving ourselves when we didn't, because a person can always ask a teammate or dig through git blame. The agent can't. It has your text, your types, and a search box. Write for that reader, and the code gets better for everyone.

Stay tuned for Part 2 of Writing Code for Agents: The Power of Determinism.

来源

Modem:Blog