Part three of the Agent Eval Practical Guide: a systematic overview of when to use code-based evaluation, LLM judging, and human evaluation, how to translate business goals into quantifiable metrics, and how evaluators themselves should be evaluated and calibrated.
Part two of the Agent Eval Practical Guide: from query sources, reverse synthesis, and judgment design, to the necessity of world_state and trajectory_anchors — a complete breakdown of what makes an eval case production-grade.
The opening piece of the Agent Eval Practical Guide series: unpacking the non-determinism and complexity accumulation of Agent systems, arguing why evaluation is not a report card but an objective function, and how to compress feedback from days to minutes with a minimal evaluation loop.