ReMe Long-Term Memory: Recording, Retrieval, Consolidation, and Forgetting

Long-term memory sounds simple until an Agent has to correct an old fact, explain where a memory came from, or rebuild its index after a crash. ReMe handles these problems with files, small workflow steps, and derived indexes that can be rebuilt.

This article follows one memory through the source code. Suppose a user says, "Keep my release notes short and include migration risks." ReMe can record the conversation, turn it into a daily note, index that note, retrieve it later, and consolidate the useful preference into a digest file. If the file is deleted, the related search entries and graph node are removed too.

The four useful entry points are BaseStep, LocalFileStore, BM25Index, and the Auto Dream steps.

The path of one memory

ReMe separates source files from data built for search:

conversation
    -> session/*.jsonl
    -> daily/YYYY-MM-DD/*.md
    -> FileNode + FileChunk
    -> BM25 index, optional vectors, and file graph
    -> search results with file paths and line numbers
    -> digest/personal, digest/procedure, or digest/wiki

The files under session, daily, resource, and digest hold the memory a person can read and edit. The index, chunk cache, catalog, and graph live under metadata. ReMe treats that second group as derived data. If it becomes inconsistent, the system can clear it and scan the files again.

That choice explains most of the implementation. Search speed matters, but an opaque index is not the only copy of a memory.

Step is the unit of work

BaseStep is the common execution unit. Search, extraction, integration, and final catalog updates are all steps. A step receives a RuntimeContext, runs execute(), and returns a response.

The call path is short:

async def __call__(self, context=None, **kwargs):
    self.context = RuntimeContext.from_context(context, **kwargs)
    if self.input_mapping:
        self.context.apply_mapping(self.input_mapping)
    result = await self.execute()
    if self.output_mapping:
        self.context.apply_mapping(self.output_mapping)
    return result

The real detail sits in Ref. A step can ask for file_store, file_catalog, a model, or an Agent wrapper without constructing it. Ref resolves the component from step arguments, then the runtime context, then the application component registry. It caches the result for that call and clears the cache before the next call so a context override can take effect.

This keeps workflow code focused. SearchStep asks self.file_store for results. DreamExtractStep asks self.file_catalog which daily files changed. The step does not need to know how those components were created.

The separation also prevents a subtle state bug. A step instance should not carry changing task data from one run into another. Current task data belongs in RuntimeContext. Shared application data belongs in ApplicationContext.metadata. Durable memory belongs in files and storage components.

FileStore turns files into searchable units

BaseFileStore defines a small contract: upsert files, delete files, query graph links, run keyword or vector search, and optimize an index. LocalFileStore supplies the local implementation.

Each indexed file has two representations:

• FileNode describes the file path, modification time, links, and chunk IDs.
• FileChunk contains a piece of text, its source path, line range, optional embedding, and search scores.

Chunks are the search unit. Files remain the ownership unit. That difference matters when a file changes.

During upsert, LocalFileStore first looks up the old node. It removes the old chunks from the in-memory map, stages the new node and chunks, and reuses an embedding when both the chunk ID and text are unchanged. It then updates the file graph, creates any missing embeddings, marks old keyword documents as deleted, and adds the new chunk text to the keyword index.

The order gives each subsystem the same view of the file:

old FileNode
    -> remove old chunks
    -> store new FileNode and links
    -> embed changed chunks when embeddings are enabled
    -> retire old BM25 documents
    -> add new BM25 documents

Embedding failure does not make local keyword search unusable. LocalFileStore disables the failing embedding path and can continue with BM25. This is a useful failure mode for a local memory system because an unavailable model should not hide every saved file.

Deleting a file follows the same boundary. The store resolves its FileNode, removes every chunk ID, deletes the graph node, and tells the keyword index to delete those document IDs. The source file and the derived search data do not silently drift apart.

BM25 searches chunks without an embedding model

BM25Index is an inverted index. ReMe tokenizes each chunk, assigns integer IDs to new terms, and stores posting lists that connect a term to the chunks containing it. The BM25 document ID is the FileChunk.id, so the result can be mapped directly back to chunk text and its file location.

For each query term, the score uses term frequency, document length, average document length, and inverse document frequency. The defaults in this implementation are k1 = 1.5 and b = 0.75:

score = IDF * TF * (k1 + 1)
        / (TF + k1 * (1 - b + b * document_length / average_length))

A rare query term receives more weight than a term found in nearly every chunk. Repeating a term can increase the score, but BM25 limits that gain. The length factor keeps a large file chunk from winning only because it contains more words.

Updates use lazy deletion. When a chunk changes, _remove_doc() marks its old slot in _deleted and removes the live ID mapping. The posting list still contains the old slot, but scoring sets deleted rows to zero. optimize_index() later rebuilds the arrays, postings, and vocabulary without deleted entries.

Lazy deletion makes normal updates cheaper. It also means deletion has two moments: the chunk disappears from search immediately, while physical index space is reclaimed during maintenance.

SearchStep combines retrieval paths

SearchStep reads the query, limit, score threshold, filters, and vector weight from RuntimeContext. It asks for more candidates than the final limit, with a default multiplier of 5 and a hard cap of 200.

When keyword and vector search both return results, the step merges their rankings with Reciprocal Rank Fusion. The constant k is 60:

rrf contribution = branch_weight / (60 + rank)

The default vector weight is 0.7, so the keyword branch receives 0.3. RRF uses rank rather than comparing raw cosine and BM25 scores. Those score ranges mean different things and should not be added directly.

If embeddings are disabled or fail, vector search returns an empty list and the step keeps the BM25 ranking. If keyword search is disabled, it can return the vector ranking. The result is then filtered, truncated, and optionally deduplicated inside a tool context.

Each returned chunk includes its path and line range. Link expansion also loads inbound and outbound wiki links for the result files. An Agent can read the matching passage first, then inspect the nearby files when it needs more context. Retrieval therefore stays connected to the files a person can open.

Auto Dream consolidates daily memory

Daily notes are useful evidence, but they grow quickly and repeat the same facts. Auto Dream processes changed daily files and writes longer-lived digest nodes. Its workflow has four steps.

DreamExtractStep scans recent daily paths and compares file modification times with the file catalog. Unchanged files are skipped. Deleted paths are removed from the catalog. Changed files are packed into an LLM request that returns memory units and topic candidates.

Each unit has a name, summary, source paths, and one bucket:

• personal for user-specific facts and preferences
• procedure for repeatable methods
• wiki for general knowledge

DreamIntegrateStep handles each unit separately. Its Agent can search existing nodes, read files and frontmatter, and write or edit digest files. The structured result records one action: CREATE, CORROBORATE, REFINE, or CORRECT. This lets an existing memory gain evidence, become more precise, or replace an outdated statement without creating a duplicate file for every daily note.

DreamTopicsStep selects interest topics and avoids recent duplicates before writing interests.yaml. DreamFinishStep updates the catalog and records a checkpoint. Paths connected to failed units are excluded from that checkpoint, so the next run can try them again.

Auto Dream does not rewrite the daily notes. The daily layer remains evidence. The digest layer is the consolidated view, with links back to its sources. In this design, compression means semantic consolidation. It does not mean compressing a file archive or shortening every conversation to a fixed token count.

What forgetting means in the current code

ReMe does not currently apply automatic age decay, TTL expiry, or an importance score that silently erases old memory. Calling that behavior "forgetting" would describe a feature the source code does not implement.

Forgetting is explicit here. A person or Agent can correct, move, or delete a memory file. When a watched file disappears, the indexing path removes its chunks, keyword documents, and graph node. BM25 hides lazy-deleted entries immediately and later reclaims their storage during optimization.

This policy favors control and traceability. It also leaves a product decision to the application: when should old evidence be removed, and who is allowed to do it? A time-based retention rule can be added outside this core, but it should delete or update the source files through the same path so every derived index stays consistent.

Reading the design as an engineer

The source code draws a firm line between memory and acceleration data. Files contain the durable record. FileStore, BM25, embeddings, the graph, and the catalog make that record faster to find and easier to connect. BaseStep moves runtime data through the workflow. Auto Dream creates a smaller, maintained digest without destroying the daily evidence.

That design costs more filesystem and index coordination than saving messages in one database table. In return, a developer can inspect a memory, follow its source links, rebuild search data, and delete it through an ordinary file operation. For an Agent that needs explainable long-term memory, those properties are part of correctness.

Source files

• BaseStep
• BaseFileStore
• LocalFileStore
• BM25Index
• SearchStep
• Auto Dream steps