I Replaced mem0 With a Self-Built Shared Memory System, and Measured It

Last week I asked Claude Code to open a pull request in a client repository, the same way I had asked Codex to do it dozens of times. Codex knew the routine: branch naming, who owns the PR action, what goes in the title. Claude Code opened the PR its own way, and I spent about ten minutes undoing steps and re-explaining a process I had already written down once.

Ten minutes is nothing. What bothered me was where the routine had been living: in Codex's global instruction file, with no equivalent for Claude Code at all. Each tool had its own private idea of how I work, and the one that had never been told anything looked exactly as confident as the one that had.

In March I published a comparison of Claude Code's file memory and mem0 and concluded I would keep both. Six months later I run neither for this job. This is what replaced them, and what I found when I measured it.

Three tools meant three versions of the truth

For anyone running more than one AI tool the stake is simple: a fact corrected in one place stays wrong everywhere else, and nothing tells you. I was running Codex, Claude Code and a self-hosted assistant, each with its own memory, and mem0 was supposed to be the shared layer between them.

In the March test, mem0 took ten paragraph-length memories and turned them into fifteen atomic facts, dropping deployment patterns and format specifications on the way. Its natural-language retrieval was genuinely good, scoring 0.9 on a question about writing voice, and I said so. My conclusion was that mem0 fit "cross-agent institutional memory" and Claude's files fit structured reference.

That was a fair conclusion about search and the wrong one about the job. The job was never "find the fact by meaning"; it was "make every tool believe the same things, and make a wrong belief withdrawable." Memory that can be wrong silently is a liability, so what matters in a memory system is who gets to say what is true, not how clever the search is. Neither option gave me that.

I moved the decision about what is true to a human

A memory an agent writes for itself is a memory nobody reviewed, so my setup treats that as the constraint. The server's most important feature is a tool it doesn't have. Every AI tool I use reaches the same small MCP server, which exposes search, read, a quality check, and propose_context. There is no approve, delete or retire tool. When an agent learns something durable, it proposes a record; the proposal lands in an inbox, and I accept, revise or reject it from an Obsidian dashboard mirrored to a ClickUp list so I can do it from my phone. The records are Markdown files in one folder, indexed locally by Basic Memory, an open-source tool that gives me full-text and vector search without a hosted service.

Each record carries a kind, a scope, a review date and a checksum of its approved text. When a tool searches, the server ranks with the index and then re-reads the actual files, withholding anything pending, overdue, edited since verification, disputed or retired. The ranking can go stale; the content an agent receives cannot.

In the nine days since it went live, agents proposed 64 records, I made 65 review decisions, 52 records are approved, and zero were approved by an agent. Judging from the activity log's timestamps rather than a stopwatch, review has cost under ninety minutes total, most of it one batch on the second day accepting records migrated from my old notes. Since then it is a few minutes on the days something arrives.

Adding local embeddings to keyword search rescued 12 queries and lost none

A memory that exists but is not found is the same as no memory, and I had no idea how often that was happening. The first version used keyword search only, fast and exact and right for 52 short records. The question was whether adding embeddings, run locally on the CPU with no API key, would change what agents actually received.

I built a query set from the records themselves: for each of the 52, three rarest keywords chosen deterministically, one natural question, and one paraphrase written to avoid the record's own vocabulary, 156 queries in all. Then I ran keyword-only, vector-only and Basic Memory's hybrid mode against the live index. An agent sees up to eight records per search, so the number I care about is whether the target is in the top eight.

Hybrid rescued 12 queries that keyword search missed inside the top eight and lost none. Every rescued query was a plain question whose words differed from the record, such as "What's the approach for tracking personal, non-billable to-dos in the task app?" Keyword-style queries stayed perfect in hybrid because the exact-match half still wins ties.

What the numbers do not prove: this is one run on 52 records, the questions were written by a language model from the records, and "relevant" means "the record the question was written from." Only 15 of the 52 paraphrases actually shared one or fewer content words with their target once I measured overlap instead of trusting the instruction. On those 15, keyword search found the target in the top eight 53% of the time, hybrid 73%, embeddings alone 87%.

Hybrid's first result is still the keyword result

The setting that reads best in the documentation was not the one that behaved best.

Basic Memory's hybrid mode scores the best keyword hit as 1.0 and adds vector similarity underneath it. Embedding similarities on this corpus sit around 0.5 to 0.7, so any record with a keyword match outranks any record found only by meaning. Hybrid's first result is therefore the keyword engine's first result; on paraphrased queries both put the target first 29% of the time. The embeddings improve ranks two through eight, which is worth having but is not what "hybrid" suggests.

Embeddings alone were the best single ranker here, putting the target first 60% of the time on the vocabulary-mismatch set against keyword's 13%. The cost is exact identifiers such as repository names and ticket prefixes, which embedding-only search put first 87% of the time instead of 100%. The local reranker fixed hybrid's ordering, 72% to 85% at rank one, but doubled search time and was worse than plain embeddings on the mismatch set.

I kept hybrid without the reranker: agents receive eight records, so first-position ordering is not the binding constraint at this size, and I would rather keep identifier queries perfect. The admission that goes with it: each search now takes about six seconds instead of one, and all six are the embedding model loading in a fresh process. The search itself takes 0.16 seconds once loaded. I have not fixed that yet.

Where each option actually fits

mem0 does something my setup will never do: it captures memory with no effort. A language model reads the conversation, extracts facts, deduplicates and stores them. For a team with nobody willing to review an inbox, that is the right trade. It lost here because the extraction dropped the reference detail I needed and nothing could expire a fact or replace one while keeping its history.

Claude Code's file memory is free, exact and always loaded, and I still use it for project-local notes. It lost as the shared layer because it is one tool's memory. The same went for Codex's instruction files; the fix there was one global file that Claude Code now imports with a single line, so both read the same rules. Basic Memory also sells a cloud product with sync and a web interface, which I did not evaluate; the local version answered my needs and keeps the files on my machine.

On price, as listed on the vendors' pages on September 16, 2026: mem0's Hobby tier is $0 with 10,000 add and 1,000 retrieval requests a month, Starter is $19 a month, Pro $249. Basic Memory Cloud is $15 to $30 per seat per month. My setup costs $0 in software and 65 MB of disk for the embedding model. My usage would fit inside mem0's free tier, so money was never the reason; control and one integration for every tool were.

What this setup still doesn't do

The records are Markdown on one Windows machine with local Git snapshots and no off-device backup yet, so the system whose purpose is not losing things could lose everything to a disk failure. The MCP server is a workflow boundary, not a security one: a tool with direct file access could edit a record without going through it, and my tools are instructed not to, which is the extent of the control.

The system's own source code sat uncommitted in a single directory for eight days before it reached a repository, which I only noticed while writing this. And review is real work that will grow with the corpus; 52 records took under ninety minutes, and I do not know what five hundred will take.

Before you add a memory product to your agents, write down who is allowed to change what they believe and how a wrong belief gets withdrawn. If the product has no answer to the second question, its search quality is not the thing to evaluate. And if you have solved the withdrawal problem some other way, I would like to know how.