If you pull apart agent memory, there are really two distinct threads running through it.

The first is personal memory: how an agent comes to understand you — your preferences, long-term context, implicit constraints, and past decisions. This kind of problem lives closer to long-context QA, and the evaluation tends to focus on “does it remember, does it understand, and can it retrieve the right user information at the right time.”

The second is procedural memory: how an agent avoids repeating mistakes, distills reusable lessons from both successes and failures, and gets better the more it runs — compounding improvement over time. This thread is concerned with things like task success rates on agentic benchmarks, tool-calling efficiency, error-rate reduction, token-cost reduction, and whether real-world usage builds genuine engineering compound interest.

This article follows the second thread.

These days, agent harnesses like Claude Code, Hermes-agent, OpenClaw, and others accumulate a wealth of trajectories over extended use. Those trajectories capture some of the most valuable signals from an agent’s interaction with its environment.

The problem, though, is that even we humans don’t make great use of these trajectories. Most of the time, they’re treated as logs, debug records, or at best stuffed back into context as retrieval fodder. An agent harness that’s been running for six months might have accumulated thousands of task trajectories — and yet the next time a similar problem comes up, the agent starts from scratch, exploring the same dead ends and stepping into the same traps.

This raises a core question:

How do you actually turn trajectories into skills?

Take a system like Hermes-agent. After enough mileage, the list of skills keeps growing. But how many of those skills are genuinely effective?

This article draws on a few recent pieces of work to discuss a question that’s becoming increasingly important: how to automatically mine, summarize, validate, and optimize agent skills from large collections of agent trajectories — skills that future tasks can actually reuse.

Hermes-agent as the Entry Point

Hermes-agent has an interesting pitch: an agent that gets better the more you use it. In practice, this means distilling past experience into external memory that can be reused in the future. That external memory falls roughly into two categories.

The first is declarative memory — factual knowledge about the user, the project, the environment, and preferences. Concretely, this takes the form of USER.md and MEMORY.md. This kind of memory answers the question “how can the agent know you better.”

The second is procedural memory — skills. Concretely, this means sessions and messages written into state.db, plus the skills that get distilled into markdown files.

More specifically, the automatic skill learning in Hermes-agent is closer to a prompt-driven self-sedimentation mechanism. Its prompt gives the agent a triggering rule: when you finish a complex task, go through multiple tool calls, fix a tricky error, or discover a non-trivial workflow, you should save that approach as a skill. And if you find a skill you are using is outdated, incomplete, or wrong, patch it immediately — don’t wait for the user to point it out.

SKILLS_GUIDANCE = (
    "After completing a complex task (5+ tool calls), fixing a tricky error, "
    "or discovering a non-trivial workflow, save the approach as a "
    "skill with skill_manage so you can reuse it next time.\n"
    "When using a skill and finding it outdated, incomplete, or wrong, "
    "patch it immediately with skill_manage(action='patch') — don't wait to be asked. "
    "Skills that aren't maintained become liabilities."
)

The actual persistence is handled by the skill_manage tool, which provides a few basic actions: create creates a new SKILL.md, patch does partial replacement of an existing skill, edit does a full rewrite, write_file adds supporting files, and delete removes skills that are no longer needed.

The upside is that it is simple, direct, and interpretable. But it is also rough around the edges.

  • It mostly relies on the model’s own judgment to decide what experience is worth turning into a skill.
  • It feels more like post-hoc sedimentation after a single task, rather than distilling common patterns across multiple successful and failed trajectories.
  • There is no evaluation. Does it actually improve task success rates? Are the skills genuinely effective?
  • Over time, the skill bank keeps growing, but which skills are still useful and which have gone stale is unclear.

It demonstrates the simplest possible loop — the agent discovers a workflow while working, writes it up as a skill, and closes the circle. It also exposes the real problems that follow-up research needs to solve: how to automatically induce skills from large numbers of trajectories, how to validate skills, how to merge them, how to optimize them, and how to keep the skill bank from turning into a junkyard.

Outside the core Hermes-agent, Nous has also spun out a separate project called hermes-agent-self-evolution, which pushes this skill-sedimentation idea toward an external optimizer style of self-evolution. The idea is to read existing skills, prompts, tool descriptions, or code, combine them with execution traces, use DSPy + GEPA to generate candidate variants, and then produce a better version through evaluation, constraint checking, and human review.

# https://github.com/NousResearch/hermes-agent-self-evolution/README.md
Read current skill/prompt/tool ──► Generate eval dataset
                                        │
                                        ▼
                                   GEPA Optimizer ◄── Execution traces
                                        │                    ▲
                                        ▼                    │
                                   Candidate variants ──► Evaluate
                                        │
                                   Constraint gates (tests, size limits, benchmarks)
                                        │
                                        ▼
                                   Best variant ──► PR against hermes-agent

Think of it as an external optimizer: it treats skills / prompts / tool descriptions as optimizable objects, uses execution traces to explain why something failed, and searches for better text variants. That said, this project is still in its early stages — most features are still listed as planned.

Trace2Skill: Batch Trajectory Induction

https://arxiv.org/abs/2603.25158

When we’re dealing not with a single skill or a single task, but with a large collection of successful and failed trajectories, how do we automatically induce transferable skills from them?

A single trajectory from a single task is easily colored by randomness. The lesson learned from one run isn’t necessarily a stable pattern. If every local lesson gets written straight into the skill bank, you end up with fragmentation, duplication, conflicting rules, and overfitting.

Trace2Skill starts from a different premise: collect a batch of trajectories first, then induce common patterns from the overall distribution.

The full Trace2Skill pipeline breaks down into three stages:

Stage 1: Trajectory Generation
  frozen agent + initial skill S0
  → roll out on evolution tasks
  → collect successful trajectories T+ and failed trajectories T-

Stage 2: Parallel Multi-Agent Patch Proposal
  dispatch one analyst sub-agent per trajectory
  → success analyzer A+ summarizes winning patterns
  → failure analyzer A- performs agentic root-cause analysis
  → each trajectory produces a trajectory-local skill patch

Stage 3: Conflict-Free Patch Consolidation
  merge all patches hierarchically
  → deduplicate, resolve conflicts, filter out spurious patterns
  → distill into a unified skill directory S*

In essence, this decomposes “from trajectories to skill” into three sub-problems:

  • Where does experience come from? From the agent’s real rollouts in the task environment.
  • How is local experience expressed? As trajectory-local patches, not written directly into the final skill.
  • How do multiple local experiences become a stable skill? Through hierarchical merging, conflict detection, and format validation — culminating in a unified skill directory.

That said, Trace2Skill isn’t a full-blown self-learning agent operating in a real environment. In the paper and codebase, it’s evaluated on SpreadsheetBench, where the error analyst can compare against ground truth and verify a failure cause with minimal repair. This works well for benchmarking.

But in real-world settings, most tasks don’t have a clear ground truth. Whether an analysis report is good enough, or whether a code change matches the user’s intent — these are hard to judge with a single evaluator.

SkillOpt: Treating Skills as Trainable External Parameters

https://arxiv.org/abs/2605.23904

Can a skill be further optimized? In other words, is a skill just a one-shot summary of experience, or can it be continuously trained through feedback — like model parameters? This is precisely where SkillOpt enters the picture.

SkillOpt asks: given a current skill, how well does it perform across a batch of tasks? Which rules should be added, deleted, or replaced? Did the change actually make the agent stronger? And if it didn’t, should the change be rejected?

The implication is that skills aren’t just experience-summary documents — they’re external state that can be optimized. SkillOpt borrows ideas from deep learning and applies them to the optimization of natural-language skills. The basic setup: the target model stays frozen; the execution harness stays frozen; the task environment stays frozen. The only thing being optimized is the external skill document — through text-space optimization, analogous to deep learning training.

The SkillOpt pipeline breaks down into several steps:

current skill S_t
  ↓
target agent runs a batch of tasks with S_t → scored trajectories
  ↓
optimizer model analyzes success/failure mini-batches
  ↓
proposes add / delete / replace edits
  ↓
caps each round's edits via a textual learning rate
  ↓
validates on a held-out selection split
  ↓
accept only if score improves; otherwise reject and log as negative feedback
  ↓
after multiple epochs, export best_skill.md

The optimizer model can propose changes, but whether they’re accepted isn’t determined by the model’s confidence — it’s determined by the held-out validation score.

How It Works

Forward pass: the target agent runs a rollout batch

The frozen target agent takes the current skill and executes a batch of tasks. The harness records task metadata, messages, tool calls, observations, command outputs, final answers, verifier feedback, and benchmark-specific context.

This mirrors the forward pass in deep learning — except the output isn’t a loss tensor. It’s:

scored trajectories:
  task
  current skill
  agent messages
  tool calls
  observations
  final answer
  verifier feedback
  score

This is critical: SkillOpt depends on scored rollouts. Without a score, the downstream validation gate has nothing to work with.

Backward pass: minibatch reflection

These trajectories tell the optimizer two kinds of things:

  • Successful trajectories: which rules in the current skill are likely working, and which behaviors should be preserved.
  • Failed trajectories: what guidance the current skill is missing, which rules aren’t precise enough, and which actions need to be prohibited or replaced.

SkillOpt doesn’t cram all trajectories into the optimizer model at once. Instead, it splits successes and failures into reflection mini-batches.

Failure mini-batches produce corrective edits. They answer:

What common problems do these failed trajectories expose?
What checking steps is the current skill missing?
Which existing rules are too vague?
Which behaviors should be explicitly banned?

Success mini-batches produce preservation edits. They answer:

Which behaviors led to success?
Which rules should be kept or emphasized?
Which effective workflows shouldn't be broken by future edits?

Add / Delete / Replace

The optimizer model’s output isn’t a full rewrite of the skill — it’s a set of structured edits. Local edits make each update more controllable and auditable, as opposed to rewriting the entire SKILL.md.

Textual Learning Rate

Skill updates are constrained as controlled textual operations. In model training there’s a learning rate; SkillOpt brings this idea to text-space: each round limits how many edits can be applied. This addresses the common problem where agent skills easily “learn into chaos.” The core insight: skill evolution should be continuous, not a tear-down-and-rebuild every time.

Validation Gate

After each round produces a candidate skill, SkillOpt has the same frozen target agent re-run tasks on the held-out selection split using that candidate. The modification is only accepted if the candidate skill’s score strictly exceeds the current skill’s score.

Rejected-Edit Buffer

If a candidate skill doesn’t pass the validation gate, SkillOpt doesn’t simply discard the attempt. It records the rejected edits, telling future optimizer runs:

Which modification directions were tried but failed;
Which rules seemed reasonable but hurt performance;
Which failure patterns still haven't been resolved;
Which edits caused regressions.

It’s essentially an optimizer memory, but more granular: which skill edits were attempted but failed validation.

Final output: best_skill.md

Limitations

SkillOpt has clear limitations.

First, it’s heavily dependent on scored trajectories. Tasks need a reasonably reliable scoring mechanism — whether exact match, unit tests, a benchmark evaluator, or LLM-as-judge. You need some way to produce comparable scores. This is relatively straightforward in benchmarks, but much harder in real-world environments.

AutoSkill: Online Lifelong Skill Lifecycle

https://arxiv.org/abs/2603.01145

How should skills be continuously generated, maintained, and reused?

As users interact with an agent over the long term, they keep expressing stable preferences, work habits, formatting requirements, task constraints, and operational patterns. AutoSkill attempts to turn these transient interaction experiences into explicit, reusable, maintainable, and retrievable skills.

Real user-agent interactions are very different from benchmark trajectories.

In a benchmark, task boundaries are clear, inputs and outputs are well-defined, and there’s often an evaluator. Whether a spreadsheet task is solved correctly, whether code passes tests, whether a web task is completed — these can all be scored fairly directly.

Real user interactions are rarely like that. Users repeatedly express needs that aren’t really “task-shaped.” These don’t have a single right answer like an exam question, but they matter a lot.

Traditional memory can preserve these preferences, but memory tends to be just facts or fragments. For example:

The user prefers a formal style.
The user dislikes excessive technical jargon.
The project uses pnpm.

This kind of memory handles “knowing what,” but doesn’t necessarily translate into “how to do it later.” AutoSkill’s goal is to further abstract these recurring interaction experiences into skills — procedural artifacts with trigger conditions, behavioral goals, constraints, and execution methods.

So AutoSkill’s core insight is this: user interaction experience shouldn’t just be saved as memory — it should be abstracted into executable, long-lived skills.

The overall closed loop:

User conversation / agent trajectory / conversation history
  ↓
Extract reusable skill candidate
  ↓
Retrieve similar existing skills
  ↓
Decision: discard / add / merge
  ↓
Write or update SKILL.md in SkillBank
  ↓
On future requests: query rewrite + hybrid retrieval
  ↓
Inject relevant skills into LLM prompt
  ↓
New feedback triggers another round of skill evolution

At the heart of it are two loops.

The first is the response generation loop. When a new request arrives, AutoSkill rewrites the retrieval query based on the current query and historical context, retrieves relevant skills from the SkillBank, injects them into the prompt, and helps the model generate a response.

The second is the skill evolution loop. After each interaction, the system tries to extract reusable skill candidates from user input and interaction traces, then decides whether the candidate should be discarded, added as a new skill, or merged into an existing skill.

The key design point in AutoSkill: new candidates don’t go straight into the SkillBank. They first retrieve the most similar existing skills, and only the candidate plus a small set of nearest-neighbor skills are sent to the management judge — not the entire SkillBank jammed into the prompt. This serves two purposes: first, it avoids skill duplication; second, it keeps the management decision scalable.

The management judge makes three types of decisions:

discard: The candidate is too vague, low-signal, non-transferable, one-off, or already covered by the existing bank.
add: The candidate represents a new durable capability.
merge: The candidate and an existing skill belong to the same capability; the existing skill should be updated.

The decision is based on four dimensions:

job-to-be-done
deliverable type
hard constraints / success criteria
required tools / workflow

Only merge when stripping away instance-level details still leaves the same capability; otherwise add; if it’s not worth keeping long-term, discard.

If the decision is merge, AutoSkill invokes the skill merging module. This isn’t simple text concatenation — it performs a semantic union:

Preserve the original skill's capability identity.
Incorporate the candidate's new reusable constraints, examples, and execution details.
Remove duplicate content.
Drop outdated or irrelevant topic constraints.
Avoid regression — keep the important checks from the original skill.
Don't introduce new standards the user never explicitly set.

After merging, the skill version number increments.

AutoSkill persists skills as regular files. The default layout:

SkillBank/
  Users/<user_id>/<skill-slug>/SKILL.md
  Common/<skill-slug>/SKILL.md
  vectors/
    <embedding-signature>.meta.json
    <embedding-signature>.ids.txt
    <embedding-signature>.vecs.f32
  index/
    skills-bm25.*
    skill_usage_stats.json

Users/<user_id> holds user-personalized skills, Common holds shared skills, vectors caches embeddings, and index stores the BM25 index and usage statistics.

That said, AutoSkill has limitations.

  • The main pipeline relies heavily on an LLM to judge whether a candidate is worth saving, whether it should be merged, and how to merge — rather than being based on scored validation.
  • It’s better suited for learning preferences, styles, workflows, and personalized constraints. For complex tool-using tasks, extracting skills purely from user interaction may not be enough.

SkillX: Hierarchical Skill Knowledge Base

https://arxiv.org/abs/2604.04804

The core idea behind SkillX is to organize agent experience into a hierarchical Skill Knowledge Base rather than a flat list of skills. Or put differently: experience should be decomposed into skills at different granularities that can be composed together.

Take a strong agent, have it roll out on training tasks, collect successful trajectories; distill those trajectories into three layers of skills — planning, functional, and atomic; then apply clustering, quality filtering, schema validation, and iterative refinement across multiple rounds. The end result is a Skill Knowledge Base that is retrievable, injectable, and transferable to weaker agents.

The overall pipeline looks like this:

Training tasks
  ↓
Strong backbone agent rollout
  ↓
Filter successful trajectories
  ↓
Extract three-layer skills:
    Planning Skills
    Functional Skills
    Atomic Skills
  ↓
Clustering / merging / splitting
  ↓
Two-stage filtering:
    general quality filter
    tool schema filter
  ↓
Skill Library update:
    add / modify / keep
  ↓
Explore under-used / failure-prone tools
  ↓
Synthesize new tasks, continue rollout, continue extracting skills
  ↓
Form a plug-and-play SkillKB

The framework breaks agent experience into three layers.

  • Planning Skill captures “how to organize a long-horizon task”

  • Functional Skill: reusable tool subroutines

  • Atomic Skill: tool-level usage constraints and failure patterns

And SkillX doesn’t stop after a single pass of extracting skills from trajectories. It introduces iterative skills refinement, letting the SkillKB keep improving across multiple rounds.

Starting from the current skill library, it rolls out on training tasks, extracts multi-level skills, then applies refinement operators — Skills Merge and Skills Filter — and finally updates the Skill Library via add / modify / keep.

The value of SkillX is that it turns agent experience from a single agent’s private experience into transferable public skills. Think of it as a form of non-parametric distillation: not distilling a strong model’s capabilities into a weaker model’s parameters, but distilling a strong model’s execution experience into an external SkillKB — and the weaker model gains capability by retrieving and injecting from that SkillKB.

Of course, SkillX has clear boundaries too. It suits scenarios where the tool environment is relatively stable, and it’s designed primarily for tool-using agents.

Rethinking Through the Lens of RL

If we re-examine Traj2Skill (as an umbrella for this family of methods) from an RL perspective, the essence may not be “distilling knowledge from trajectories” — it’s:

Extracting reusable policy improvement units from the distribution of historical behavior.

An agent’s behavioral process in an environment can be abstracted as a policy generating trajectories and maximizing expected return:

$$ J(\pi)=\mathbb{E}{\tau\sim\pi}\left[\sum{t=0}^T\gamma^t r_t\right] $$

Here, a trajectory records the complete interaction — state, observation, tool calls, and environment feedback.

So the goal of Traj2Skill isn’t just to extract human-readable lessons from a trajectory. It’s to find structured behavioral patterns in existing trajectories such that the new policy $\pi’$ achieves a higher expected return on future tasks than the original policy $\pi$:

$$ J(\pi’) > J(\pi) $$

The key difference is that Traj2Skill typically doesn’t update model parameters directly. Instead, it updates an external skill library — separate from the model itself.

trajectory → experience signal → update external skill library → guide future policy

Viewed this way, a skill is not ordinary memory, nor a compressed version of a task log. It’s more like an externalized policy prior.

If we bring Agentic-RL into the comparison, Agentic-RL is closer to end-to-end policy optimization — it optimizes the model itself. Traj2Skill, by contrast, is closer to modular policy improvement — it optimizes reusable behavioral modules external to the model.

From an engineering standpoint, this is also what makes the skill library appealing. An LLM-based policy is expensive to train, slow to update, weak in interpretability, and ill-suited for frequent fine-tuning. But a skill library is externalized, readable, editable, searchable, and versionable. It offers a lighter-weight path for policy improvement than directly updating model parameters. This is likely what motivates work like SkillOpt: bringing skills into a closed loop of “generate — evaluate — optimize.”

Looked at differently, the two approaches aren’t substitutes — they can complement each other. That’s exactly what SAGE / SkillRL are exploring: incorporating skills into reinforcement learning or policy optimization frameworks.

Zooming out further: trajectories are too long, too specific, too contingent. What’s truly valuable is extracting behavioral structures that transfer across tasks. So where does Traj2Skill sit — closer to offline RL, or behavior cloning?

A lot of current Traj2Skill work is actually closer to behavior cloning. The basic pipeline of trajectory mining is: summarize frequent patterns, compress, reuse. But there’s a problem here: a successful trajectory is not the same as an optimal trajectory.

This is the credit assignment problem. The final return of a long-horizon agent trajectory may come from only a handful of key decisions — not from every step along the way. What’s truly worth distilling into a skill is not necessarily the most frequently occurring behavioral snippets, but the behavioral structures that can consistently improve expected return on future tasks.

This perspective also exposes the hardest problem for Traj2Skill: where does the reward come from? Benchmarks have standard answers, but real-world environments are different — there’s often no simple ground truth. This means that the ceiling of a real-world Traj2Skill system depends heavily on the quality of its evaluator.

If we lay these pieces of work side by side, they each tackle different parts of the trajectory-to-skill pipeline:

  • Hermes-agent: self-sedimentation after a single trajectory
  • Trace2Skill: experience induction across multiple trajectories
  • SkillOpt: evaluation-driven skill optimization
  • AutoSkill: long-term skill lifecycle management
  • SkillX: hierarchical skill organization and strong-to-weak transfer

So the core of Traj2Skill-style work breaks down into two questions.

The first is SkillOps: how to create, merge, retrieve, inject, version, evaluate, and retire skills — without letting the skill bank degrade into a junkyard.

The second is the deeper question: how to perform credit assignment over trajectories to identify behavioral structures that genuinely improve expected return.

The former solves for long-term operation. The latter answers whether this is really about “summarizing experience” — or about policy improvement.