Short answer: Use Engram extraction, transform merges, and bounded topics on purpose—compress redundant narrative, keep decisive identifiers and corrections.
Cost hits three places: extract/transform tokens, growing vector store, and prompt budget when too many lines are pasted. Engram already compresses architecturally—chat becomes discrete memories, transforms merge duplicates, bounded topics keep one summary or profile per scope. Retention preserves exact wording; consolidation packs coverage into fewer tokens. Under a tight budget, smart consolidation usually wins; under a loose budget, a few raw atoms can still be cheaper than a bad summary. Retain corrections so updates win. Frugal loops skip low-value chatter, limit hybrid search, and let transform merges do quiet work before building scrubbers. Cost-constrained does not mean memory-poor—every stored and retrieved token should have a job.
Cost-constrained agent deployments feel memory pressure in three places at once. Pipeline runs spend tokens extracting and transforming facts. The vector store grows with every turn. The prompt budget shrinks when you paste too many retrieved lines into the model. Compression is how you keep recall useful without paying for every raw utterance forever.
Weaviate Engram already compresses at the architecture level. Extraction turns chat into discrete memories. Transform steps merge and rewrite duplicates. Bounded topics keep one canonical summary or profile per scope. Your job is to use those levers on purpose. Research on agent memory keeps returning to the same tradeoff. Retention preserves exact wording. Consolidation packs more coverage into fewer tokens. Under a tight budget, smart consolidation usually wins. Under a loose budget, keeping a few raw facts can still be cheaper than a bad rewrite.
Where do memory costs actually accumulate?
Eager extraction on every message is expensive. Some systems now wait for recurrence before they invoke an LLM to consolidate, because repeated themes are worth summarizing and one-off noise is not. Engram still runs a pipeline when you call memories.add, so the application can choose when to add. Batch quiet periods. Skip pure acknowledgements. Send only turns that carry preferences, decisions, or task state.
Retrieval cost shows up in the prompt. Returning twenty memories feels thorough. It often wastes tokens on near-duplicates and dilutes the signal. Hybrid search with a small limit is a compression control as much as a relevance control. Topic filters shrink the candidate set further. You pay for what you inject, not for what sits unused in storage.
Storage cost grows more slowly than prompt cost, but it still matters at scale. Unbounded topics accumulate many memories per user. That is fine when facts stay atomic and searchable. It becomes wasteful when every paraphrase is stored as a new object. Engram’s transform path is meant to rewrite and drop duplicates. Topic descriptions that ask for consolidation make that path work harder for you.
How does Engram compress without a custom compressor?
Bounded topics are the clearest built-in compressor. A ConversationSummary scoped by conversation_id holds one memory per conversation. Each new add updates that summary in place. Token cost in the prompt stays roughly constant even as the chat grows. A bounded user profile works the same way for always-on personalization. Fetch retrieval returns that single object by topic and scope without ranking a long list.
Transform steps compress across time. When a user changes roles or preferences, Engram can rewrite an older memory and delete the redundant new extract. Buffers on enterprise pipeline configs can roll daily activity into one aggregate memory. Even without custom pipelines, you get free compression by letting extraction produce facts instead of storing full transcripts as the long-term store.
Application-side dual context finishes the pattern. Keep the last few live messages for deixis and tone. Pull a short Engram hit list for durable facts. That mix is the Engram context-window guidance for a reason. Recent text is cheap locally. Long history belongs in compressed memory.
When should you retain raw detail instead of consolidating?
Some fields hate lossy summaries. Order IDs, alloy codes, dosage strings, and exact error hashes should stay as discrete memories. Consolidation papers warn that merge and abstract operators can drop the one detail that answers the next query. Under a loose token budget, retention of those atoms is often correct. Under a tight budget, consolidate the narrative and retain the identifiers as separate short memories.
Also retain corrections. If a user says the earlier preference was wrong, you need that update to win. Engram’s rewrite path helps when the new content reaches the same topic and scope. Do not bury the correction only inside a long daily rollup that never gets retrieved for preference questions.
So compression policy is not “always summarize.” It is “compress what is redundant, keep what is decisive.” Cost constraints make that judgment sharper. They do not remove it.
What does a frugal Engram loop look like in practice?
Imagine a millinery shop agent on felt-hat-blocking-bench-6. The blocker and client trade many turns about crown height, brim curl, and steam time. You do not want every “mm-hmm” in long-term memory. You want a living session summary, a few hard constraints, and a tiny retrieval budget on the next turn.
import os
import uuid
from engram import EngramClient, HybridRetrieval, FetchRetrieval
client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])
hat_user = "client-nora-vale"
conversation_id = f"block-{uuid.uuid4().hex[:8]}"
scope = {"conversation_id": conversation_id}
def worth_remembering(user_text: str) -> bool:
keys = ("prefer", "must", "avoid", "size", "brim", "crown", "steam")
return any(k in user_text.lower() for k in keys)
user_turn = (
"For felt-hat-blocking-bench-6 I need a 58cm oval block, "
"moderate brim curl, and no more than ninety seconds of steam on the tip."
)
assistant_turn = "Noted. I will keep steam under ninety seconds on the tip."
# Skip empty acknowledgements; only spend pipeline runs on dense turns.
if worth_remembering(user_turn):
run = client.memories.add(
[
{"role": "user", "content": user_turn},
{"role": "assistant", "content": assistant_turn},
],
user_id=hat_user,
group="personalization",
properties=scope,
)
client.runs.wait(run.run_id)
# Constant-size continuity: fetch the bounded conversation summary when enabled.
try:
summary_hits = client.memories.search(
query="conversation summary",
user_id=hat_user,
group="personalization",
topics=["ConversationSummary"],
properties=scope,
retrieval_config=FetchRetrieval(limit=1),
)
except Exception:
summary_hits = []
# Tight hybrid budget for atomic facts — compression at retrieval time.
fact_hits = client.memories.search(
"What block size, brim curl, and steam limits apply?",
user_id=hat_user,
group="personalization",
topics=["UserKnowledge"],
properties=scope,
retrieval_config=HybridRetrieval(limit=3),
)
recent_messages = [
{"role": "user", "content": user_turn},
{"role": "assistant", "content": assistant_turn},
]
prompt_memory = []
if summary_hits:
prompt_memory.append(f"SUMMARY: {summary_hits[0].content}")
for m in fact_hits:
prompt_memory.append(f"FACT: {m.content}")
# Send recent_messages + prompt_memory to the LLM. Cap stays small on purpose.
The gate before memories.add cuts pipeline spend. The fetch of one summary caps narrative tokens. The hybrid limit of three caps fact tokens. Together they are compression without a separate vector database of embeddings for every utterance.
How do you tune compression as traffic grows?
Measure three numbers. Pipeline runs per active user. Average memories injected per turn. Answer quality on questions that need exact identifiers. If runs are high and quality is flat, raise the write gate. If prompts are fat and answers are vague, lower the search limit and tighten topics. If identifiers go missing after aggressive summarization, split them into short unbounded facts and keep summaries for story only.
Offline jobs can help at larger scale. Periodically list a user’s low-value memories and delete superseded ones after you confirm a rewrite exists. Engram delete is permanent, so be deliberate. Prefer letting transform merges do the quiet work during ordinary adds before you build a scrubber.
Cost-constrained does not mean memory-poor. It means every stored and retrieved token has a job. Engram’s extract-transform-commit path, bounded summaries, and limited hybrid search give you that job structure out of the box. Your product policy decides how aggressively to use it.
Our next chapter, What is the future of memory standards across agent frameworks?, looks past single-product compression tactics toward shared memory interfaces that different agent stacks can speak.