AI News

Anthropic's Lydia Hallie on Reassessing Claude Code Best Practices

Shao MengUpdated Nov 13 11:50

AI summary

Lydia Hallie, who leads developer relations for Claude Code at Anthropic, argued in a talk at the AI Engineer Code Summit that, for the Claude 5.5 series models, many Claude Code usage patterns accumulated over the past year no longer apply. She recommended switching the model to Opus 5.5, changing permissions to auto mode, and trimming the emphasis instructions written for older models in CLAUDE.md. She also offered safety and quality fallback measures: deny rules and PreToolUse hooks take effect before the classifier, sandboxes and cloud sessions limit side effects, hooks use exit code 2 to feed failures back to the model, and CI uniformly covers the pull requests opened by each session.

Why read this

AI score 66

The talk systematically reviewed which old practices should be abandoned now that models have become more capable, and which safety and quality checks should be added, all of which can be transferred to one's own session workflows.

Topics

Tags

Body · AI translation

Lydia Hallie, a member of Anthropic's technical team, handles developer relations for Claude Code. In her talk at the AI Engineer Code Summit, "Rethinking Claude Code Best Practices," her theme was: the models have gotten better, and many of the practices piled onto Claude Code over the past year are now actually holding you back; she wanted to discuss best practices for the Claude 5.5 series models—which are still worth keeping and which are no longer worth the effort.

Her own daily driver is Claude Opus 5.5. Earlier in the week of the talk, Anthropic released Claude Fable 5.2. In her view, these newer models are better at long tasks and at understanding codebases, and Fable is also better at verifying its own output. If your way of working is still stuck in earlier this year, you'll fall behind.

The shape of sessions has changed, and so has people's role

Hallie recalled that earlier this year, typical sessions were still short. The models commonly used then were in the Opus 4.5 tier, which was already quite usable by the standards of the time, but they would stop earlier, and people had to keep correcting and reminding them, and had to add emphasis like "important" and exclamation marks to CLAUDE.md. Auto mode didn't exist yet, so you either approved every command by hand or went with the riskier "skip permission checks." People were still in almost every step.

Now, inside Anthropic, it's no longer "one or two sessions at a time." She described dozens of sessions running simultaneously, for hours on end, verifying their own results along the way, and some even orchestrating other sessions. People no longer watch almost every step; they're more like managing a large fleet of sessions.

She mentioned the company's October 13 engineering blog post, "Redesigning source control at Anthropic": the number of code changes Anthropic merges per day went from about 2,500 in May to more than 30,000 in September. Hallie cited it to make the point that developers must rethink where their attention should go.

Her judgment is this: the best practices from earlier this year centered on guiding sessions, approving or rejecting commands, limiting output, and starting sessions by hand. Once people step back from these steps, the model, the runtime, and the environment it runs in have to take over. The two most important questions next are: when so many sessions run at once and you're no longer approving each one, what keeps code quality? And when sessions run for hours with no one nearby, what keeps things safe?

First, change the defaults: write fewer instructions, hand approvals to the classifier

She suggested first switching sessions to today's default configuration: change the model to Opus 5.5 and the permission setting to auto mode. Anthropic made auto mode the default permission mode in August. The official product post "Auto mode is now the default in Claude Code" says: starting August 14, new sessions on the Pro, Max, and Team plans enter auto mode by default. The new models are better at understanding intent and codebases, so you no longer need to correct them as often mid-task, and you no longer need to approve every command.

CLAUDE.md should change too. For today's models, "important," "never," and strings of exclamation marks are often unnecessary and can even mislead the model, leading to more tool calls and worse results. This file should now contain only information the model can't see from the code: background like a service still using a legacy source that isn't directly referenced in the code is worth keeping; what the tech stack is and where to find files usually doesn't need to be written anymore. Hallie recommends using /doctor prompt-audit to check CLAUDE.md and skills against a specific model and flag instructions written for older models or no longer needed, which is also consistent with the official best practices documentation.

auto mode itself is a separate classifier layer that reviews almost everything other than "reading or modifying files in the project": it lets through commands it considers safe, such as running tests or installing dependencies, and blocks commands it isn't sure about. By default it trusts only the current folder and current repository, so if you want to push to another deployment repository, the classifier may still block it, and you're back in the loop. She said that in most cases the defaults are enough; if you still have to click approve frequently even with auto mode on, first check your settings and use natural language to specify which repositories, buckets, and domains are trusted, and which operations should always be allowed and which must never be done. The official documentation puts this configuration under autoMode.environment and also offers /auto-mode-setup to draft it. Once the defaults are switched, guidance and most approvals can be handed off; the harder part is how to still hold up when no one is present.

The classifier is not a guarantee: rules come first, the environment is the backstop

Hallie reminds us that upgrading the model and turning on auto mode still only amounts to "working with the model." Most of the time it will do what you say, but that is not a guarantee. She cites a study published in August: when faced with an obviously dangerous command, only about 14% of people stopped it, while auto mode stopped 89%, so roughly 1 in 10 still slips through.

These figures can be matched in the official product document mentioned above. The company hired 1,053 paid professional testers through a research vendor, who worked in a purpose-built test environment, not in their own codebases; midway through each session, one of the permission prompts was replaced with an obviously dangerous command, but it was not actually executed—only the text the testers saw was changed. The result: people stopped 13.6% (143/1053), and auto mode stopped 89% (937/1053). The longer the session, the worse people performed: in the early part they stopped about 17%, and after they had already handled more than 50 prompts this dropped to about 5%, whereas auto mode did not decline as sessions grew longer. Hallie summarizes this as "about 14%" and "about one in ten still gets through," consistent with the original text.

If what you want is a "guarantee it will not happen," the only way is to write it into the runtime and enforce it there. This is exactly why permission rules still matter. If a project has a deny rule targeting a certain type of tool call, Claude Code checks it before the auto mode classifier ever sees it; once it matches, the classifier does not look at it at all. The official permissions documentation also states: deny rules and PreToolUse hooks take effect before the classifier, and are valid in all permission modes.

So what should be written as rules, and what can be left to the now-stronger model? The criterion is very concrete: if Claude gets it wrong this time, can you still take it back? If it writes the wrong code, you can still roll back—this kind of thing can be left to the model; if it discards uncommitted changes, or reads out secrets and the secrets have already entered the session transcript, it should be written as a rule. For example, by default auto mode can approve reading .env; the official documentation says this as well. If you want it to never be able to read this file, you should add a deny rule, so that even cat cannot open it.

Permission rules can only decide whether a tool call starts; they cannot restrict what it can touch once it has started. She gives the example of npm test: auto mode will let this common test command through, but when the script runs it executes code you did not write, such as dependencies, or scripts Claude wrote on the fly to do checks. Once any of that has been tampered with, it may still open .env or SSH keys and send them to a server you have never seen. Even if a person is still approving each item one by one, when faced with "just running tests" they will often simply click approve. So you cannot simply ban this command, and you must restrict its side effects—that is, restrict the runtime environment.

On a local machine, you can put this kind of command into a sandbox, where the operating system controls which files it can read, which directories it can write, and which hosts it can access, and this applies to child processes as well. Once the sandbox is on, file and network restrictions are handed to the operating system, and if a dependency then tries to hit .env it will be blocked. She also reminds us: by default the sandbox may still be able to read quite a lot on the machine, so if you care about SSH keys you need to add separate rules; unapproved hosts are also unreachable. More importantly, the sandbox only governs shell commands and their child processes; file tools, MCP servers, and hooks are still outside the sandbox. The official sandbox environment documentation also states: the built-in Bash sandbox only constrains the shell; to shut these categories inside the boundary as well, you need a sandbox runtime, a container, a virtual machine, or a cloud session.

Therefore, she more strongly recommends putting the entire session somewhere "where those local secrets simply do not exist"—that is, Claude's cloud sessions: an independent machine, a fresh copy of the repository, and it keeps running even when the laptop is closed, which is exactly suited to tasks that run for hours. If an API key is still needed midway, it can be made into a network secret: the session can prompt that the key is missing and use it, but the key value is not sent to the model. By default, network access is also only open to addresses such as package repositories and GitHub; other domains must be explicitly added to the environment. The official cloud session documentation also states that such credentials are attached by a proxy after the request leaves the session.

She also adds a very important point: this machine is disposable, but what leaves it is not—for example, commits that have already been pushed, or messages sent through connectors. So even in the cloud, you should still keep auto mode on and commit deny rules into the repository, so that the rules travel with the code, in order to block things like force pushes or secrets smuggled into commits.

When no one is present, rely on checkpoints that can judge right from wrong

Beyond safety, you also need to confirm that Claude is doing what you actually want. When a human is in the loop, the sense of quality comes from having seen every change and approved every command; now it runs in the background for hours, and people will no longer read every file. She gives the example of migrating authentication to a new API: the model understands the goal well, but there are still many small decisions along the way. With the same prompt, one run might migrate it over but leave the old API behind, one might migrate only the new login, and another might log everyone out. These all seem plausible, so you need to set up verification checkpoints along the way, letting Claude continuously verify its own results. A checkpoint can be anything that gives a pass or fail: tests, builds, screenshots, all work. If a given run would log everyone out, it fails the first check and has to find another way.

Setting them up also comes in layers. The simplest is to write it in the prompt: explain what counts as done, and have it break the work into tasks with tests; she also likes to require it to show actual output, because you are not there at the time. But prompts are likewise not a guarantee. For checks that must be enforced, she uses hooks, which are scripts that Claude Code runs at specific moments: for example, run the linter after every file change, run the relevant tests when marking a task as complete, and run the full test suite when the whole session is about to end. If it fails, it has to go back and fix it until it passes.

There is an easily overlooked detail here: most test tools exit with code 1 on failure, and the task may still be marked as complete. To send the failure back to Claude to fix, the script should use exit code 2. The official best practices and the hooks documentation both state: the Stop hook runs a script when Claude tries to end the turn, and if it exits with code 2, it blocks the end and feeds the reason back to the model, keeping it working.

Only after you have a session you are comfortable letting run on its own is it appropriate to open more at the same time. Each session has its own machine, so you no longer need to worry as much about local worktrees; but if you still start each session and still review each one to the end, the human is in fact still in the "opening" and "closing." She believes what should be filled in first at this point is CI: these sessions will all eventually open pull requests, and CI covers each one with the same set of checks, regardless of how the session was originally configured. With auto-fix turned on, Claude can also go back and fix things when checks fail, avoiding spending review time on problems the checks could already have caught. For some changes, you can also auto-merge after all checks pass, but she stresses this is not the default and depends on risk.

She also described her own team's practice: every pull request is first reviewed by Claude, but not the session that wrote the code—rather, one with fresh context, plus another set of agents that re-check every defect it finds. How much humans need to review depends on risk: for smaller changes like docs or copy, after automated checks pass, no human hand may be needed; ordinary product changes may need only one reviewer; higher-risk ones may need two. This is her description of her team's process, not the product's default policy.

From opening sessions yourself to having an orchestration session open them

One step further, you no longer have to start every session by hand. She admits she is not good at switching context frequently, and managing too many sessions at once is not always smooth. The alternative is to keep just one main session for orchestration: tell it what needs to be done, and it splits the tasks, starts other sessions for each task, and tracks the results coming back. These new sessions are called threads in Claude Code. The human only needs to talk to this one main session.

The main session itself does not write code, so it is better suited to a model that is good at deciding the next step and judging whether returned results are sufficient—she names Fable; the threads that actually do the work are better suited to Opus 5.5. If a returned result is in fact not done yet, the orchestration session sends it back until it is truly complete and passes the various checks. This structure also means you no longer need to manage context yourself, or think about when to compact or clear it. She maps this approach to Claude Code Projects, which is still in testing. The official documentation confirms it is currently in public beta, gradually rolling out to Pro, Max, and Ultra plans: a project has a long-running coordinating session and multiple parallel threads it starts, usually cloud sessions, and the models on each side can be specified separately. The documentation defaults both sides to Opus; Hallie offers a different division of labor: Fable for judgment, Opus 5.5 for execution.

Looking back at those old best practices, they have not disappeared, they have just moved. The model guides, auto mode approves, deny rules and environment limits constrain what can happen, tests, hooks, and CI check, and in the end Claude can also spin up sessions itself. What truly remains for the human is starting the first session and reviewing the final result.

To try it in your own environment, her advice is very specific: first hand Claude a task much larger than usual, confirm that CI really catches the problems you care about, then configure the environment, including network access, secrets, and startup scripts, then start it in the cloud and observe where the session gets stuck and where you are still in the loop. Where it gets stuck is where you should hand off next.

Source

Shao Meng