Part three of the Agent Eval Practical Guide: a systematic overview of when to use code-based evaluation, LLM judging, and human evaluation, how to translate business goals into quantifiable metrics, and how evaluators themselves should be evaluated and calibrated.
Part two of the Agent Eval Practical Guide: from query sources, reverse synthesis, and judgment design, to the necessity of world_state and trajectory_anchors — a complete breakdown of what makes an eval case production-grade.
The opening piece of the Agent Eval Practical Guide series: unpacking the non-determinism and complexity accumulation of Agent systems, arguing why evaluation is not a report card but an objective function, and how to compress feedback from days to minutes with a minimal evaluation loop.
A comprehensive survey of agent-memory benchmarks: from LoCoMo and LongMemEval to PersonaMem, covering evaluation methodology, metrics, and data synthesis pipelines across conversational memory, user profiling, and multimodal memory.
How to convert agent trajectories into reusable skills through extraction, validation, deduplication, evaluation, and continuous optimization.
A practical 2026 guide to the minimum architecture of an AI agent harness: loops, tools, filesystems, context management, memory, subagents, and observability.