Four Ways to Remember, Ranked by Cost
"Make the agent remember our users" sounds like one feature. It is four different architectures, differing in cost, freshness, and blast radius when they go wrong. Teams that pick the wrong tier usually over-engineer: they ship a vector database where a text file would have been faster, cheaper, and more debuggable.
Tier 1: Working Notes in the File
The cheapest durable memory is a plain file the agent reads at session start and updates as it works: user preferences, project context, open TODOs, known pitfalls. No embeddings, no services, no sync. It works because most of what matters about a user or a project fits in a few kilobytes, and because the agent already knows how to read and write files.
Rules that keep it healthy: fixed structure (so the agent can update it surgically without rewriting the whole file), append-mostly history for provenance, and a size budget — when the file outgrows the budget, compress it deliberately rather than letting it bloat the context forever.
Tier 2: Structured Profiles
When other systems need the same facts (onboarding flows, support tooling), promote memory into a typed profile — a schema with fields like communication language, subscription state, long-running goals. Updates become validate-able writes instead of free text, and stale facts have a defined owner.
The design question is conflict policy: the user changed their mind twice — which statement is current? Time-stamped facts with "last write wins per field" plus an audit trail is usually enough. Temporal awareness matters: "lives in Moscow" and "moved to Tashkent in March" should coexist as facts with different validity, not overwrite each other blindly.
Tier 3: Retrieval Over Episodic History
Conversation logs and task outcomes are too bulky to keep in any always-loaded structure, but they contain the long tail: what was tried, what failed, what the user objected to. The pattern that works is episodic retrieval — write structured episode records (goal, steps, outcome, user feedback), embed them, and retrieve on demand when the current task looks similar.
This is RAG with one twist: the corpus is the agent's own behavior. Quality problems mirror RAG's — recall misses on paraphrase, stale episodes outranking fresh ones — so the same hybrid-retrieval and acceptance-metric discipline applies (see our RAG post for the pipeline).
Tier 4: Learned Parameters
Fine-tuning memory into model weights is the most expensive tier and the right one only when the "memory" is actually a skill — a format the agent should always produce, a workflow it should always follow. Facts about a specific user never belong here: they change too fast and too personally. A useful smell test: if losing this memory means a user complains, it is Tier 1-3; if losing it means every output degrades, it might be Tier 4.
Memory Hygiene: The Part Everyone Skips
Unmanaged memory rots. Three failure modes show up in every long-running system:
- Poisoning. A bad instruction captured once ("always cc this address") persists and acts in every future session. Memory writes need the same review discipline as knowledge-base writes: separate the proposer from the reviewer, gate updates.
- Contradiction pileup. Thousands of small facts, many now false. Periodic consolidation — an offline pass that merges, re-validates, and prunes — keeps retrieval precision from decaying.
- Silent staleness. Facts with no timestamps can't expire. Every stored fact should know when it was learned and from what source.
Key Takeaways
- Start with a file. It solves most "remember this" needs at zero infra cost.
- Promote to structured profiles only when multiple systems need the same facts.
- Episodic retrieval over your own logs is RAG — budget for the same pipeline discipline.
- Fine-tuning is for skills, not user facts.
- Memory needs governance: review on write, timestamps on facts, consolidation on schedule.
