Dimka

Scalp Misalignment

Why is evaluating agents so difficult relative to evaluating a standard LLM? An LLM generates a single response to a prompt. An agent instead interacts with an environment by reasoning, calling tools, observing the results, and repeating. Rather than evaluating a single output, we are evaluating a full trajectory for a long horizon problem. Outcome & environment. Success is usually determined by the final outcome. Instead of checking the agent’s output, however, we usually check the actual state of the environment after a trajectory completes. For example: - A coding agent can say it fixed a bug but still fail test cases. - A personal assistant could say “your flight has been booked!”, but we don’t know if this is true until we check the booking database. The environment plays a huge role in the evaluation process. For example, we need a stable environment that does not change over time in order to run reliable evals, and we need this environment to be efficient so that evals can be run efficiently. Additionally, the environment must be realistic and expose the proper harness / tools for the agent to be effective. Environment state. The final answer is only one part of the evaluation. We also need to understand how the agent reached that answer. Did the agent use the correct tools? Did it interact efficiently with the environment? We can validate certain tool calls, check for intermediate solutions in the trajectory, and look at how many turns / tokens were used. Verifying intermediate trajectory states is common in agentic evaluation. If the agent solves a task, then this provides a valuable signal. But, what do we do if the agent solves a small ratio of tasks in a benchmark? In this case, we need to have a more granular measure of performance that allows us to understand how close the agent came to solving certain problems. Agent evals are expensive. Unlike traditional LLM benchmarks, agent evaluations can span long time horizons with many reasoning steps, tool calls, and environment interactions. A single evaluation may require vast amounts of generated tokens and dozens or hundreds of interaction turns with the environment. For this reason, agent evaluations are much more difficult to both curate and run! This data is super complex and even solving a single task can be expensive. Less data, more noise. Because each evaluation is costly, we typically run far fewer examples than in traditional LLM benchmarks. This is both because each task can take a long time to run and because curating the data / setup for each task is expensive. Rather than evaluating single dataset examples, we are asking the agent to solve a real problem. As a result, we usually have smaller evaluation dataset with higher variance, making reliable measurement difficult.