LZY’s Blog
Technical notes and reflections on AI Agents, RAG, and LLM engineering. Code and words.
AI Decouples Production from Cultivation
AI is decoupling output from cultivation: as entry-level tasks are absorbed, how talent debt forms, where judgment comes from, and why growth no longer just happens on its own.
Agent Eval Practical Guide 03: How to Choose Evaluation Methods
Part three of the Agent Eval Practical Guide: a systematic overview of when to use code-based evaluation, LLM judging, and human evaluation, how to translate business goals into quantifiable metrics, and how evaluators themselves should be evaluated and calibrated.
Agent Eval Practical Guide 02: How to Build an Eval Set
Part two of the Agent Eval Practical Guide: from query sources, reverse synthesis, and judgment design, to the necessity of world_state and trajectory_anchors — a complete breakdown of what makes an eval case production-grade.
Agent Eval Practical Guide 01: The More Complex the System, the More Evaluation Must Come First
The opening piece of the Agent Eval Practical Guide series: unpacking the non-determinism and complexity accumulation of Agent systems, arguing why evaluation is not a report card but an objective function, and how to compress feedback from days to minutes with a minimal evaluation loop.
Beyond SOTA: What Agent-Memory Benchmarks Actually Measure — and What They Miss
A comprehensive survey of agent-memory benchmarks: from LoCoMo and LongMemEval to PersonaMem, covering evaluation methodology, metrics, and data synthesis pipelines across conversational memory, user profiling, and multimodal memory.
The Hardest Part of AI Transformation Isn't AI — It's the Dirty Work Nobody Wants to Do
Why AI transformation stalls after model selection: dirty data, undocumented workflows, integration debt, evaluation gaps, and organizational resistance are the real bottlenecks.
From Agent Trajectory to Agent Skill: How to Turn Experience into Compounding Returns
How to convert agent trajectories into reusable skills through extraction, validation, deduplication, evaluation, and continuous optimization.
On Agent Harness: A Minimalist Design Guide for AI Agent Harnesses in 2026
A practical 2026 guide to the minimum architecture of an AI agent harness: loops, tools, filesystems, context management, memory, subagents, and observability.
Stop Overthinking: A Practical Guide to LLM Thinking Budget Control
A practical analysis of LLM overthinking and thinking budget control solutions, comparing reasoning length and compute budget configurations across mainstream model APIs.