Short answer: It turns raw conversation or notes into discrete, well-formed memory objects instead of storing everything verbatim.
Raw dialogue buries useful facts in filler and makes later recall noisy. Extraction pulls durable points into structured memories, then compares them to existing entries as new, duplicate, or superseding updates. Extraction that is too broad floods the store; too narrow misses clinically or operationally important asides. Engram’s pipeline runs this end to end so searchable memory stays focused.
Everything covered so far in this Part has assumed memory already exists as tidy, well-formed objects with clean properties. Real memory never starts out that way. It begins as a conversation, a note, a stream of raw events, and something has to turn that unstructured starting material into the kind of discrete, well-shaped memory the rest of this Part has been describing. This chapter looks at how that extraction step actually works.
Why Can’t Raw Conversation Just Be Stored and Searched Directly, Without Any Extraction Step?
A raw conversation carries a lot of content that isn’t actually worth remembering on its own, small talk, clarifying questions, filler that served a purpose in the moment but adds nothing to a system’s longer-term understanding of what actually happened. Storing an entire conversation verbatim also buries the genuinely useful facts inside a much larger block of text, making it harder for a later search to isolate exactly the detail it needs. Extraction solves both problems at once: it pulls out only the pieces that carry lasting value and stores each one as its own discrete, individually searchable memory, rather than leaving useful facts scattered throughout an undifferentiated transcript.
What Does an Extraction Step Actually Do to a Piece of Raw Conversation?
An extraction step reads through incoming content and identifies the specific facts worth remembering, then writes each one out as its own standalone memory, rephrased into a clear, self-contained statement rather than left as a fragment of dialogue that would only make sense alongside its surrounding context. A single exchange might yield several separate extracted memories, or none at all, depending on how much of that exchange actually carried lasting, memory-worthy content. The goal isn’t summarizing the conversation as a whole, it’s identifying the individual, durable facts buried inside it and giving each one its own clean, independent form.
Does Extraction Only Work on Conversational, Dialogue-Shaped Content?
Not exclusively. Extraction can run just as naturally over plain, non-conversational text, a raw event log, a freeform note, anything that isn’t already shaped as back-and-forth dialogue between defined roles. Some systems also allow content to skip the extraction step entirely, accepting already-structured facts directly when a system has already done its own extraction work elsewhere and doesn’t need that step repeated. This flexibility matters because real systems bring memory-worthy content in from many different shapes, not just tidy chat transcripts, and an extraction pipeline that only handled one specific input shape would leave considerable value on the table.
What Actually Happens to a Newly Extracted Fact Before It Gets Permanently Stored?
A freshly extracted fact typically doesn’t get written to permanent storage immediately as-is. Instead, a follow-up step usually compares it against whatever related memories already exist, deciding whether the new fact is genuinely new information, a duplicate of something already stored, or an update that should supersede an existing memory entirely. This comparison step is exactly what keeps a growing memory store from accumulating duplicate or contradictory entries every time a similar topic comes up again in a later conversation, catching that overlap right at the point of extraction rather than leaving it to accumulate as unresolved clutter.
Why Does Keeping Extraction Focused and Narrow Actually Matter for Downstream Search Quality?
An extraction step that tries to capture too much, treating every passing detail as worth remembering, risks flooding a memory store with low-value clutter that dilutes genuinely important facts during later search. An extraction step that’s too narrow, in the opposite direction, risks losing real, useful information that never gets captured at all. The right balance depends on what a system is actually configured to care about, extraction should be deliberately scoped around specific categories of information genuinely worth remembering for that system’s purpose, rather than either indiscriminately capturing everything or leaving that judgment undefined.
How Does Weaviate Engram’s Extraction Pipeline Handle This Process End to End?
Weaviate Engram runs incoming content through an asynchronous pipeline, starting with an extraction step tailored to that content’s shape, followed by a transform step that reconciles new facts against existing memories before anything gets permanently committed. Consider an in-home elder-care coordination service, where caregivers report daily observations that need to become searchable, structured memory for the broader care team:
from engram import EngramClient
client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])
client.memories.add(
[
{"role": "user", "content": "Client seemed more fatigued than usual today, napped twice in the afternoon."},
{"role": "assistant", "content": "Thanks for the update. Any changes to appetite or medication response?"},
{"role": "user", "content": "Appetite was fine, ate a full lunch. Did mention some dizziness when standing up quickly though."},
],
properties={"client_id": "care-client-5521"},
)
From this exchange, Engram’s extraction step identifies the durable, care-relevant facts, increased fatigue with afternoon napping, normal appetite, and a new symptom of dizziness on standing, writing each as its own clear, standalone memory rather than leaving them buried inside the back-and-forth phrasing of the original exchange:
results = client.memories.search(
query="Has this client reported any dizziness or balance issues recently?",
properties={"client_id": "care-client-5521"},
)
A care coordinator reviewing this client’s recent history later benefits enormously from having that dizziness symptom captured as its own clean, searchable fact, rather than needing to reread an entire raw transcript hoping to spot it buried in context. This is exactly the value extraction delivers for a use case like elder-care coordination, where a single passing detail mentioned almost as an aside can carry real clinical significance, and where a system that only stored raw conversation verbatim would make that detail far harder to reliably surface when it actually matters.
Fact extraction turns raw, messy content into the kind of discrete, well-formed memory this Part has spent its earlier chapters describing how to structure and organize. One question extraction has to answer for every single fact it produces is exactly how small or how broad that fact should actually be. Our next chapter, What are atomic facts and why does granularity matter?, takes up exactly that question.