Stanford Research Open-Sources Memorilla, a Latent Memory Module for Frozen LLMs

Why it matters

Stanford's SNAP group, led by Jure Leskovec and working with co-authors from Toronto, Penn, and MIT, has released Memorilla, an open-source memory module that condenses an entire document collection into a fixed set of memory tokens handed to a frozen Qwen3-8B. Rather than stuffing every document into the context window or retrieving only the chunks most similar to a query, Memorilla learns, per question, what to extract from the whole store. A frozen Qwen3-Embedding-4B encodes each document once, learned slots attend over the full collection, and the resulting vectors are injected ahead of the question. Only the memory module itself is trained; the encoder and decoder stay frozen. The team reports the best average across its benchmark suite, with clear gains over full-context prompting and rival methods on most tasks, though it trails LLMLingua-2 on PubMedQA, where source texts are short and compression offers little benefit. Code, training recipes, and checkpoints are public under Apache 2.0, making the approach reproducible and deployable by others.