Short answer: Signal is memory that correctly changes what the agent should do; noise is near-duplicates, superseded facts, and distractors that still occupy top retrieval slots.
Growth monitors count how many memories you keep; signal-to-noise asks how many still help. Long-lived stores accumulate paraphrases, expired seasonal notes, and polite chatter; similarity search then retrieves distractors that sit close to the query while precision falls even if average similarity looks healthy. This chapter defines a practical SNR for Weaviate Engram stores, how to measure useful hits versus junk in scoped searches without boiling the ocean, and how topics, transforms, and deletes raise the ratio. Prefer modest hybrid limits and topic filters at read time; let transform reconcile corrections instead of outvoting old text with near-duplicates. Delete explicitly for leaked cross-scope facts, unsafe speculation stored as fact, and obsolete setpoints. Do not chase a perfect 1.0 SNR—chase noise occupying the ranks where signal should sit.
Growth monitors count how many memories you keep. Signal-to-noise asks how many of those memories still help. Long-lived stores accumulate paraphrases, expired seasonal notes, and polite chatter that once looked harmless. Similarity search then retrieves distractors that sit close to the query in embedding space. Precision falls even while average similarity looks healthy. This chapter defines a practical SNR for Weaviate Engram memory stores, shows how to measure useful hits versus junk in scoped searches, and how topics, transforms, and deletes raise the ratio as the yard ages.
What Does Signal-to-Noise Mean for an Agent Memory Store?
Signal is a memory that correctly changes what the agent should do or say for the current task. Noise is everything else that still occupies retrieval slots: near-duplicates, superseded facts, off-topic personal asides, and speculative notes stored as certainties. SNR is the share of retrieved items that are signal under a fixed probe set, not a mystical purity score for the whole database.
Research on long-horizon agents shows why this ratio matters. Under noisy writes, unbounded memory can keep high embedding similarity while Precision@k collapses. Failed distractors sit close to the query. Insertion-order eviction is blind to that failure mode. Selective retention that discounts redundancy and failed entries preserves precision at smaller capacity. Engram faces the same geometry. Your job is to keep the active set skewed toward load-bearing facts.
SNR is not the same as storage cost. A small store can be mostly noise. A large store can be mostly signal if write control and reconciliation work. Growth alerts and SNR alerts should both exist. One without the other misleads ops.
How Do You Measure SNR Without Boiling the Ocean?
After the definition is clear, pick a probe pack per group. Each probe is a realistic question with labeled must-have phrases and forbidden junk patterns. Run the same hybrid search the agent uses. Score each returned memory as signal, neutral, or noise using a rubric humans already trust from personalization eval. SNR for a probe is signal count divided by hit count. Roll up median SNR across the pack weekly.
Complement retrieval SNR with write-side ratios. Among completed runs, what share of committed operations are updates or deletes versus creates? Chronically create-only preference topics usually mean paraphrases are stacking. Transform-with-context is supposed to rewrite and drop duplicates. When that stops showing up in operations, noise is winning at write time.
Keep probes scoped. A hive yard’s wintering rules should not be scored with a retail gift-shop question. Property filters keep the denominator honest. Unscoped searches mix tenants of meaning and invent a fake SNR crisis.
How Can Weaviate Engram Raise Signal Before You Delete Anything?
Measurement without levers is theater. Start with write control. Tighten topic descriptions so acknowledgments and one-off logistics do not become durable UserKnowledge. Use bounded topics for canonical profiles and summaries so each scope holds one living document instead of a paraphrase pile. Route distinct concerns into separate topics so a search for mite treatment does not drown in picnic scheduling trivia.
Let the pipeline reconcile. When keepers correct a fact, add the correction conversation and allow transform steps to update or delete. That raises SNR by amending the active set instead of outvoting old text with newer near-duplicates. Buffers that consolidate intermediate extracts before commit also keep half-baked fragments out of searchable storage.
At read time, prefer modest hybrid limits and topic filters. Retrieving eight mixed memories when three constraint memories suffice injects noise into the prompt even if the store itself is decent. Dual-memory patterns that keep recent chat separate from Engram search reduce the temptation to stuff every old paraphrase into context.
When Should You Delete Noise Explicitly?
Some noise never reconciles cleanly. Leaked cross-scope facts, unsafe speculation stored as fact, and obsolete setpoints after a confirmed supersession deserve memories.delete. Deletion is permanent. Use it after review, with the memory id from a probe hit list. Prefer correction adds when history should be rewritten in place through transform.
Here is an apiary desk probe that scores SNR on a hive-yard search and quarantines an explicitly bad memory id during review:
import os
from engram import EngramClient, HybridRetrieval
client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])
keeper = "keeper-mira"
group = "apiary_ops"
yard = "hive-yard-west"
probe = {
"query": "What mite treatment and sugar schedule apply for yard west this March?",
"signal_markers": ["oxalic", "1:1 syrup", "march"],
"noise_markers": ["wedding catering", "thanks!", "maybe try"],
}
hits = client.memories.search(
query=probe["query"],
user_id=keeper,
group=group,
properties={"yard_id": yard},
retrieval_config=HybridRetrieval(limit=6),
)
signal = 0
noise = 0
labeled = []
for m in hits:
text = m.content.lower()
is_signal = any(s in text for s in probe["signal_markers"])
is_noise = any(n in text for n in probe["noise_markers"])
if is_signal and not is_noise:
signal += 1
label = "signal"
elif is_noise and not is_signal:
noise += 1
label = "noise"
else:
label = "neutral"
labeled.append({"id": m.id, "label": label, "preview": m.content[:140]})
snr = signal / len(hits) if hits else 0.0
print({
"yard_id": yard,
"hit_count": len(hits),
"signal": signal,
"noise": noise,
"snr": round(snr, 3),
"labeled": labeled,
})
# After human review confirms a noise id:
# client.memories.delete(bad_id, user_id=keeper, group=group)
# Prefer reconciliation when the real schedule changed:
client.memories.add(
[
{
"role": "user",
"content": "Correction for yard west: March mite pass is vaporized oxalic only. Stop mentioning the old trickle method.",
},
{
"role": "assistant",
"content": "Updated yard west March mite protocol to vaporized oxalic only.",
},
],
user_id=keeper,
group=group,
properties={"yard_id": yard, "memory_class": "treatment_protocol"},
)
The marker lists are deliberately simple for a weekly job. Replace them with human or calibrated judge labels as the probe pack matures. The Engram calls stay the same.
How Do You Keep SNR Healthy as the Store Ages?
Schedule SNR alongside growth monitoring. Alert when median probe SNR drops while create volume stays high. Promote production noise incidents into regression fixtures so pipeline edits cannot reintroduce the same junk extract. Revisit topic descriptions quarterly. Seasonal apiary knowledge needs explicit supersession paths every spring and fall.
Do not chase a perfect 1.0 SNR. Some neutrals are cheap context. Chase the failure mode that hurts: noise occupying the top ranks where signal should sit. That is the precision collapse long-horizon studies warn about.
Signal-to-noise is the quality of a long-lived Engram store expressed as a ratio you can trend. Probe scoped searches. Tighten topics and transforms. Delete only what review condemns. Then growth stays worth paying for because retrieval still earns its place in the prompt.
Our next chapter, How do you build dashboards for memory system health?, gathers growth, SNR, latency, and run health into views operators can watch without reading raw probe scripts.