What input shapes can the memory pipeline accept?

The previous chapter focused on extraction from conversation specifically, reading dialogue turn by turn to pull out durable facts. Conversation is only one of several shapes raw data can actually take before it reaches a memory pipeline. This chapter looks at the full range of input shapes a system might need to handle, and how each one changes what extraction actually has to do.

Why Would a Memory System Need to Support More Than One Shape of Incoming Data in the First Place?

Not every source of information worth remembering arrives as a back-and-forth conversation. A system might receive a standalone note, an event logged automatically somewhere else in an application, or facts that some other process has already carefully extracted and simply needs stored and integrated. Forcing every one of these genuinely different shapes through a single, conversation-only interface would mean awkwardly wrapping non-conversational content in a fake dialogue structure it was never actually part of, adding friction rather than removing it.

When Does Plain, Freeform Text Serve as the Right Input Shape Rather Than a Structured Conversation?

Freeform text fits naturally whenever a piece of content doesn’t actually have any conversational structure to begin with, a standalone note, an observation logged by an automated process, or a system event that never involved any back-and-forth exchange at all. This shape still benefits from the same extraction intelligence a conversation would receive, an underlying model reads the text and identifies the durable facts worth remembering, it just doesn’t need to account for distinct speaker roles or conversational turns, since none genuinely exist in this kind of content.

What Does It Actually Mean to Submit Content That’s Already Been Pre-Extracted, and Why Would a System Choose That Path?

Pre-extracted input skips the extraction stage entirely, accepting already-structured facts directly, each one paired with the specific topic it belongs to, and passing them straight into the transform and commit stages that follow. This path exists for situations where a system already has its own process for deciding what counts as worth remembering, an agent equipped with its own tool-calling logic, deciding for itself when and what to commit to memory, rather than relying on Engram’s own extraction step to make that judgment call from raw text or conversation.

Does Bypassing Extraction Also Mean Bypassing the Deduplication and Reconciliation That Happens Downstream?

No, and this is exactly what makes pre-extracted input a genuinely useful middle ground rather than an all-or-nothing escape hatch. Even though the extraction stage itself gets skipped, pre-extracted facts still flow through the same transform and commit stages that handle deduplication, reconciliation, and everything else covered throughout this knowledge base’s discussion of keeping memory clean and coherent over time. A system choosing this path takes over responsibility for deciding what’s worth remembering, but doesn’t have to rebuild the machinery that keeps those remembered facts well-integrated once they arrive.

How Should a Team Actually Decide Which of These Input Shapes Fits a Given Piece of Data?

The right shape follows directly from where the data actually comes from and how much extraction judgment a system wants to delegate. Genuine back-and-forth dialogue, a chat transcript or an agent conversation, belongs in the conversation shape, since that’s exactly the structure conversation-aware extraction is built to understand. Standalone content with no conversational structure, an event, a note, an observation, belongs in the plain string shape. Content a system has already deliberately decided is worth remembering, bypassing the need for a model to make that judgment independently, belongs in the pre-extracted shape. Mixing these up, forcing a genuine conversation through the string shape or forcing an isolated event through the conversation shape, works technically but loses the specific handling each shape was actually designed to provide.

How Does Weaviate Engram Let a System Choose the Right Shape for Genuinely Different Kinds of Incoming Data?

Weaviate Engram accepts exactly these three input shapes through the same underlying API, routing each one to its own dedicated extraction step, or skipping extraction entirely for pre-extracted content, while still sharing the same downstream transform and commit stages. Consider a warehouse logistics platform’s inventory-event assistant, tracking both automated system events and an autonomous restocking agent’s own deliberate observations:

from engram import EngramClient
from engram import PreExtractedInput, PreExtractedItem

client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])

client.memories.add(
    "Automated system event: pallet scanner at dock 4 flagged a barcode mismatch on inbound shipment 88213, requiring manual verification before putaway.",
    properties={"warehouse_id": "warehouse-central-7"},
)

client.memories.add(
    PreExtractedInput(items=[
        PreExtractedItem(
            content="Restocking agent determined that SKU 44921 should be reordered ahead of schedule due to an unexpected demand spike detected in recent order patterns.",
            topic="RestockingDecisions",
        ),
    ]),
    properties={"warehouse_id": "warehouse-central-7"},
)

The first call uses the plain string shape for a standalone automated event with no conversational structure, letting Engram’s own extraction step decide what’s worth remembering from it. The second call uses the pre-extracted shape, since the restocking agent has already made its own deliberate judgment about what matters and simply needs that judgment persisted and integrated, deduplicated and reconciled just like any other memory, without asking Engram’s extraction step to second-guess a decision the agent already made carefully on its own. A warehouse coordinator searching this facility’s history later can retrieve both kinds of memory together, regardless of which input shape originally produced them.

Choosing the right input shape lets a system match how data actually enters the pipeline to how it should genuinely be processed. Underneath every one of these shapes, though, sits the same fundamental question: how does an underlying model actually decide what counts as a fact worth extracting in the first place, and could a simpler, rule-based approach handle that decision instead? Our next chapter, When should you use LLM vs rule-based extraction?, takes up exactly that question.