Memory for Agents That Forget the Right Things

Every team I have sat with this year has a memory slide. Vectors, graphs, “episodic” stores, a little brain icon. Then you watch the agent and it either remembers nothing useful or remembers a joke you made in March and applies it as policy. Both failures come from the same wish: that memory could be a pile instead of a decision.

People forget on purpose. We forget the wrong compile flag from a laptop we no longer own. We remember that this customer invoices in yen and hates surprise emails. Agent memory should be at least that opinionated.

Three drawers, not one soup

1. Working memory: the current trace. Cheap, noisy, disposable. This is the context window. Do not romanticize it.
2. Task memory: artifacts for this job — the spec, the failing test, the last patch. Lives for hours or days. Delete it when the ticket closes.
3. Durable memory: facts that should outlive the ticket. Preferences, invariants, scars. Small on purpose.

Most systems dump all three into one store and retrieve with cosine similarity. Similarity is a fine way to find a paragraph. It is a poor way to decide what is true. “The user likes concise answers” and “the user said ‘be concise’ while angry about a specific bug” will sit near each other and take turns steering the next run.

Write less than you retrieve

The write path is the product. I want a human or a strict schema to approve anything that enters durable memory. Free-form “notes to self” from the model are how you grow a folklore. Folklore is charming in a company handbook. It is dangerous when a loop can act on it.

type DurableFact = {
  claim: string;
  sourceRun: string;
  expiresAt?: string;
  confirmedBy: "user" | "eval" | "human-review";
};

If you cannot name who confirmed a fact, it does not belong in the durable drawer. Put it in the ticket and let it die with the ticket. Forgetting is not a defect in a system that can act. Forgetting is hygiene.

I would rather an agent ask me the same preference twice a year than silently obey a preference I outgrew. The first is slightly annoying. The second is how you get a system that feels haunted.