npm founder Laurie Voss on how to evaluate whether an AI agent is up to standard

In a Tesla research report written by an AI, the analyst consensus looked like this: 50% recommend buy, 52.6% recommend hold, 13.1% recommend sell. The three numbers add up to 116%.

The agent that wrote this report saw nothing wrong at all—the formatting was beautiful, the tone was professional, and every section that should be there was there. It had stitched together and summed the results of two different web searches.

This session is from the AI Engineer workshop in New York, and the speaker is Laurie Voss, co-founder of npm, who now runs developer relations at Arize AI. For an hour and a half he talks about only one thing: how do you know whether an agent is any good, and once you've changed it, how do you prove it got better.

First read a dozen or so records, then write the first eval

He draws a very clear distinction between two words. A trace is a log, an eval is a test, and an eval is running tests against traces. The difference lies in how detailed the trace is: every round of model calls, every tool call, inputs and outputs, latency, tokens.

Most AI features ship in nothing like this way. You build it, try it yourself three times, it looks good, you ship it. He says trying it three times doesn't count as a test suite. What about unit tests? No—the same prompt produces different text every time it runs, two completely different answers can both be right, a thousand different answers can all be right, and there's no expected string you can assert against. So everyone falls back on human eyeballing, and human eyeballing can't go into CI, where nobody is watching.

So he always starts from the same place: read the records first, read a dozen or so from beginning to end, look at the inputs and outputs, and see which step went wrong. If you build automation before you understand how the agent fails, you'll only end up measuring the things that are easy to measure.

There's a method to reading. First open coding: write down in plain language what you see, and when something is good, say why, and when something is bad, say why; then axial coding: group the notes into piles. When he couldn't get through it, he handed the job to Claude Code, and the agent read all the records in nine minutes and even invented a category of its own called harness leakage, referring to the agent constantly asking to do things it didn't have permission to do.

Confidently wrong, or smarter than your rules

The 116% mistake is worth taking apart. The agent did two searches, one finding that 50% of people held one view, the other finding that 13% of people held another view, and it directly summed numbers from two different datasets and then wrote that into the conclusion. Each search was correct on its own; the error was in the reasoning. Only by looking at the records can you tell which step went wrong.

The opposite direction happens too. Anthropic internally ran a benchmark simulating a customer service conversation, where the task was to help a user change a flight. By design this task should fail, because the user is flying economy, and economy doesn't allow seat changes. It turned out the model figured out it could first upgrade to first class, where seat changes are allowed, make the change, and then downgrade back to economy—and just like that, the user's request was fulfilled. The test marked it as failed, because it wasn't supposed to be able to do that in the first place.

So he has a hard requirement for writing evals: evaluate the result, not the steps. Don't check whether the agent called search first and then another tool—that's too rigid, and you'll end up holding back a system that's better at finding solutions than you are.

Free code evals, and a correctness judge that gets everything wrong from the start

Code evals are the least trouble: no model, no API calls, a function that asserts against a string, runs in a few milliseconds, costs nothing. The first check was laughably crude: does the report actually mention the stock I asked about. Two failed—a Tesla report didn't mention Tesla, and an Amazon report didn't mention Amazon. The question was about AWS profitability, so the agent wrote only about AWS and dropped Amazon the e-commerce company.

Going a level up, you have to bring in an LLM as judge. He started with Phoenix's built-in correctness eval, and all 16 reports were judged incorrect, every single one. The judge used its own knowledge to decide what was right, and its knowledge cutoff was around January, while the agent was writing about October stock prices—to the judge, this was a future event, so it must be a hallucination. Seven of the 16 explanations complained that it couldn't verify the latest data.

He changed the question to ask about faithfulness: based on the research material you have in hand, does the report make sense, are the conclusions supported? The judge could answer this question—the original search results and research notes were all in the records, and fed in, it could check them. 14 of the 16 were judged unfaithful, but the explanations were far more useful: the claim that Azure is the second-largest cloud provider isn't supported by context, and the 2027 target can't be found in any of the search results.

The same batch of outputs, a different question, and one side is all zeros while the other is usable. His conclusion is that choosing the right eval matters far more than tuning the eval.

Judges have to be tested too

A judge is essentially a classifier in machine learning, and it needs human-labeled answers to check against. He labeled all 16 reports one by one using the same criteria he gave the judge, and this is called a golden dataset. Partway through labeling he caught himself cheating: the rubric clearly said the report must list specific risks, and a Microsoft report mentioned not a single risk, yet he still labeled it as actionable. The judge was stricter than he was.

He also tested the judge with examples. He ran three rounds with examples, and not one round was fully correct; the best was 15 of 16 matching him. Without examples, two rounds had all 16 matching, and the remaining round was off by one. Removing the examples made it more accurate because the examples said risks must be listed, so the judge followed the examples while the human was slacking off.

Don't expect the judge to agree with you 100% of the time either. When two human experts score the same outputs against the same criteria, Cohen's kappa is often only 0.2 to 0.3, barely better than random. The judge he wants doesn't have to be consistent every time, just more reliable than another human judge.

There's also a self-check: for results judged as failing, you should be able to see at a glance where the agent went wrong. If you look and say there's no problem here, then there's a good chance the eval is the problem. In a benchmark testing whether an agent can reproduce papers, Opus 4.5 initially scored only 42%; later someone went through the actual interactions and found the model had answered correctly, but the test required six digits of precision and it gave four. After fixing the test, the score became 95%.

One line of prompt turned 5/16 into 48/48, without touching the fabricated data at all

After reading the logs before changing the prompt, it's no longer guessing. Each change corresponds to a problem that was found: the judge said no recommendation was given, so require a clear conclusion of buy, hold, or sell; the judge said risks were mentioned without evidence, so require data support when mentioning risks; the logs showed the agent repeatedly trying to save the report as a file, so tell it to just reply with the content.

As a result, actionability went from 5 of 16 passing to all 48 passing, 16 in each of three runs. Sounds great, but in the same batch of experiments faithfulness went from 2/16 to 3/48. The prompt clearly added a rule that only numbers appearing in the research could be used, yet it still made things up. His exact words were: only one line was changed, fixing the problem we were measuring, but not solving the agent's making-things-up problem.

He was blunt about the trade-off: prioritize recall. The cost of a false positive is you spending a minute or two reading a report that's actually fine; the cost of a miss is a wrong report reaching the client.

He also laid out the order of improvement: data quality first, because if the data is bad no amount of tuning later will help; then change the prompt; then switch models, usually bigger and more expensive; hyperparameters like temperature come last, and current Claude doesn't even let you pass them anymore.

After switching models, it learned to always say hold

After upgrading to Haiku 5.5, faithfulness more than tripled, but it was still bad, and the agent still made things up. But recommendation consistency clearly improved: the conclusion was the same every time. He glanced at the numbers: 41 of 45 calls recommended hold. The way to improve consistency is to pick the same answer every time. Cost quadrupled; whether it's worth it, he said, is for you to judge.

Only when this stuff ends up in CI will it not be overturned by the next change. Their npm test has two gates: getting the stock ticker right is the floor, and if it's wrong it must not ship; actionability is the goal, with the three-run average needing to reach 0.8. One build failed because the Rivian report was not actionable, averaging 0.75.

He also distinguished two kinds of evals. Capability evals test what the agent can't do yet, a mountain to climb; once something passes reliably every time, don't delete the test, turn it into a regression eval and pin it in CI. To see stability you look at pass^k, not pass@k. A coding assistant getting it right on the second try is acceptable; a financial analyst getting it wrong once every five times is a big problem.

He kept coming back to one plain statement: if you haven't written clearly what counts as usable, you can't say your agent is unusable. The value is in that loop, not in any single score.

More hardcore podcast content: Podwise