What are tiered memory architectures?

Pattern Three splits memory by speed and lifespan instead of stuffing everything into one flat retrieval bag. Short-term memory is the live context window: recent turns, tool outputs, and whatever you just paged in. Working memory holds task-local state for the current job, such as a destination, budget, or checklist that should die when the ticket closes. Long-term memory lives outside the window in a durable store. Weaviate Engram is a natural long-term tier. Bounded summaries and property-scoped notes can serve as mid-term. The recent message slice remains short-term. Tiering is how you keep Pattern Two’s vector recall without paying Pattern One’s full-history tax on every call.

This chapter defines the tiers, maps them onto Engram primitives, shows a promotion path from scratchpad to durable facts, and walks through a three-tier assemble loop. When relationships between entities matter more than tier speed, graph architectures become the next pattern.

Why does one flat store stop being enough for long-horizon agents?

After Pattern Two, agents can remember across sessions. Long jobs still hurt. A multi-hour repair thread needs the last tool error in-window now, a running plan for this ticket, and durable preferences from last quarter. If all three compete in the same top-k hybrid search, the plan gets crowded out by similar old anecdotes. Context engineering writing from Weaviate draws the same split. Short-term stays lean. Long-term lives externally. Working memory holds multi-step task state without pretending it is lifelong knowledge.

OS-inspired designs push the metaphor further. The context window is scarce RAM. External recall and archival stores are disk. Agents or harnesses page content in and out. You do not need a full virtual-memory product to benefit. You need explicit tiers with different write and read policies.

Tiering also clarifies forgetting. Evicting a working note is routine. Deleting a long-term safety rule is exceptional. Flat stores blur that difference until maintenance jobs invent it after the fact.

How can Engram play the long-term and mid-term tiers?

Once tiers exist as ideas, assign substrates. Short-term stays in process: the last two or three exchanges plus the current tool bundle. Mid-term can be a bounded ConversationSummary fetched with FetchRetrieval, or a small working object keyed by ticket_id that you overwrite each step. Long-term is Engram UserKnowledge and related topics searched with HybridRetrieval under user_id.

Engram’s dual-memory tutorial pattern is already two tiers. Recent messages handle “that” and “it.” Memory search handles durable personalization. Adding a conversation summary or a task scratchpad makes three. Buffers and daily rollups can promote mid-term activity into longer-term experience without keeping every episode in the prompt.

Promotion rules matter. When a working fact proves durable, call memories.add so transform can reconcile it into long-term. When a ticket closes, purposeful-forget the mid-term scope. Do not promote speculative scratch text. Write control still applies at every boundary.

What should each tier optimize for?

Knowing the substrates, set objectives. Short-term optimizes for conversational coherence and immediate tool fidelity. It must stay small enough that the model does not get lost in the middle. Mid-term optimizes for task continuity across many steps in one job. It should be complete for the active goal and irrelevant tomorrow. Long-term optimizes for stable personalization and transferable lessons. It should be maintained: deduplicated, superseded, and scoped.

Read paths differ. Short-term is always injected. Mid-term is fetched by known scope keys, not guessed by similarity. Long-term is searched by the current query, sometimes filtered by topic. Mixing those read styles is the point of tiering. A single hybrid search over everything collapses the architecture back to Pattern Two.

Token budgets should be explicit. Cap recent messages. Cap summary length via topic instructions. Cap long-term hits with retrieval limits. Measure total prompt tokens per turn so tiers do not silently inflate.

What does a three-tier assemble-and-promote loop look like?

Imagine a watchmaking agent on watch-escapement-timing-desk-5. The live turn needs the last timing reading. The ticket scratchpad holds the current beat-error target. Engram holds the watchmaker’s durable preferences.

import os
import uuid
from engram import EngramClient, HybridRetrieval, FetchRetrieval
from engram.exceptions import APIError

client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])
watchmaker = "watchmaker-jon-vale"
conversation_id = f"timing-{uuid.uuid4().hex[:8]}"
desk = {
    "desk_id": "watch-escapement-timing-desk-5",
    "conversation_id": conversation_id,
}

# Tier 1: short-term (in-process)
recent = []

# Tier 2: working memory for this ticket (app-owned, overwritten)
working = {
    "beat_error_target_ms": None,
    "last_amplitude": None,
    "status": "open",
}

def load_long_term(query: str):
    return client.memories.search(
        query,
        user_id=watchmaker,
        group="personalization",
        topics=["UserKnowledge"],
        properties={"desk_id": "watch-escapement-timing-desk-5"},
        retrieval_config=HybridRetrieval(limit=5),
    )

def load_mid_term_summary():
    try:
        hits = client.memories.search(
            query="conversation summary",
            user_id=watchmaker,
            group="personalization",
            topics=["ConversationSummary"],
            properties={"conversation_id": conversation_id},
            retrieval_config=FetchRetrieval(limit=1),
        )
        return hits[0].content if hits else ""
    except APIError:
        return ""

def assemble_prompt(user_text: str) -> str:
    long_term = load_long_term(user_text)
    summary = load_mid_term_summary()
    lt = "\n".join(f"- {m.content}" for m in long_term) or "- (none)"
    return (
        "You are a timing coach for watch-escapement-timing-desk-5.\n"
        f"Long-term memories:\n{lt}\n"
        f"Session summary:\n{summary or '(none yet)'}\n"
        f"Working scratchpad: {working}\n"
        "Use scratchpad for the active ticket. Use long-term only for durable prefs."
    )

def promote_if_durable(note: str):
    # Example gate: only explicit preference language graduates to long-term.
    if "prefer" in note.lower() or "always" in note.lower():
        run = client.memories.add(
            note,
            user_id=watchmaker,
            group="personalization",
            properties={"desk_id": "watch-escapement-timing-desk-5"},
        )
        return run.run_id
    return None

def turn(user_text: str, assistant_text: str):
    recent.append({"role": "user", "content": user_text})
    recent.append({"role": "assistant", "content": assistant_text})
    del recent[:-6]  # keep short-term tiny

    # Update working tier from structured cues in the user text.
    if "beat error" in user_text.lower():
        working["beat_error_target_ms"] = 0.4
    if "amplitude" in user_text.lower():
        working["last_amplitude"] = "270 deg"

    # Mid/long writes via Engram (summary + possible durable extract)
    client.memories.add(
        [recent[-2], recent[-1]],
        user_id=watchmaker,
        group="personalization",
        properties=desk,
    )
    promote_if_durable(user_text)
    return assemble_prompt(user_text)

print(
    turn(
        "On watch-escapement-timing-desk-5 I prefer timing at 20s lift. Beat error feels high.",
        "Aim near 0.4 ms beat error and recheck amplitude after the next regulate.",
    )
)

Three read styles appear in one prompt builder. Hybrid search for long-term. Fetch for the bounded session summary when enabled. Direct dict access for working state. Promotion is gated so not every complaint becomes lifelong doctrine.

When does tiering still fall short?

Tiered systems excel at time and attention management. They still struggle when the question is relational. Supplier A links to batch B links to failure mode C. Similarity across tiers may not reconstruct that path. At that point you keep Engram for personalization tiers and add a graph layer for entity structure.

Also avoid fake tiers. If mid-term and long-term are the same unscoped collection, you only renamed Pattern Two. Separate keys, separate read APIs, and separate retention clocks are what make Pattern Three real.

Our next chapter, What are graph-based memory architectures?, focuses on when memory needs explicit entities and edges instead of only tiered text retrieval.