Search “Search”

Matched fieldsSummary

In part 10 of the "How to build great evals" series, Madhu Guru argues that evaluating AI agents cannot rely only on the final answer; the steps taken to reach that answer matter too: both may arrive at 42, but one trajectory is 4 clean tool calls, while the other is 17 calls, 3 repeated searches, and recovery from 2 errors. He recommends first defining the complete workflow, breaking down each step's task, designing an evaluation method for each step, and distinguishing medium-difficulty from h

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary

Matched fieldsSummary