How do you detect memory pollution and drift in production?

Successful runs can still leave you with wrong memory. Pollution is active junk that should never have been stored, or should have been revoked. Drift is once-true fact that the world has outgrown while retrieval still ranks it highly. Agents then sound confident and act on stale humidity setpoints, outdated allergen notes, or duplicated preferences that disagree. This chapter shows how to detect those failure modes in production with Weaviate Engram search and lifecycle tools, how transform and bounded topics reduce conflict by design, and how to quarantine or delete polluted memories before they steer the next shift.

How Is Pollution Different From Ordinary Retrieval Misses?

A miss means the right memory was absent or poorly ranked. Pollution means a wrong memory was present and influential. Drift is pollution’s slow cousin. The fact was right when written. A later state change invalidated it. Similarity search still retrieves both the old and new values with nearly equal scores. Research on evolving knowledge shows that cosine similarity alone cannot tell a contradiction from a paraphrase. That is a structural gap, not a tuning mistake.

TEPA-style work names the failure clearly. Stale active memories that newer evidence has superseded keep polluting the prompt. Append-only stores make this worse under reversal. Systems that revoke or rewrite the old key stay useful. Engram’s transform steps exist for the same custodial job. They deduplicate, merge, and reconcile when pipelines are configured for it. Detection still matters because configuration, noisy writes, and cross-talk can defeat the custodian.

Operationally, treat pollution as a quality incident with a memory id, not as a vague “the model forgot.” You need the retrieved contents, their topics, timestamps, and whether a later correction run updated or deleted anything. Run-level debugging tells you the write succeeded. Pollution detection asks whether what succeeded deserved to stay active.

What Signals Reveal Drift Before Users File Tickets?

After you separate pollution from misses, watch for early signals. Contradictory hits in the same scoped search are the loudest. Two memories that disagree on the same subject should not both sit in the top five without a review. Sudden growth in unbounded topics without matching update or delete volume suggests near-duplicate pileup. Correction phrases in chat that do not produce updates in later runs suggest write control is too loose or topics are too broad.

Outcome metrics help too. Rising user corrections on preference questions, rising “actually we changed that” replies, and eval drops on consistency suites all hint at drift. Pair those with Engram search samples on a fixed probe set. Ask the same cave-condition questions daily. Store the top hits. Diff content across days for scopes that should be stable.

Bounded topics shrink one class of drift by force. A single profile or summary per scope updates in place instead of accumulating rival versions. Use them for canonical state. Keep unbounded topics for many distinct facts. Mixing those shapes without intent is how creamery notes become a contradictory stack.

How Do You Probe a Scope for Conflicting Memories With Engram?

Signals need a concrete probe. Pick a high-stakes question. Search with the same user_id, group, and properties your agent uses in production. Inspect every hit, not only the first. Look for numeric conflicts, opposite preferences, and outdated absolute claims. When you find a bad id, fetch it, then delete it or write a correcting conversation that the pipeline can reconcile.

Prefer correction through memories.add when transform-with-context should rewrite history. Prefer memories.delete when the content is junk, leaked across scopes, or unsafe to leave searchable even briefly. Deletion is permanent. Use it when quarantine is the right call.

Here is a creamery aging-cave monitor that searches for humidity guidance, flags contradictory hits, and removes an explicitly bad memory id during review:

import os
import re
from engram import EngramClient, HybridRetrieval

client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])
affineur = "affineur-elise"
group = "creamery_aging"
cave = "cheese-cave-n3"

probe = "What humidity should cave N3 hold for the washed-rind wheels this week?"

hits = client.memories.search(
    query=probe,
    user_id=affineur,
    group=group,
    properties={"cave_id": cave},
    retrieval_config=HybridRetrieval(limit=8),
)

humidity_vals = []
for m in hits:
    found = re.findall(r"(\d{2,3})\s*%", m.content)
    humidity_vals.extend(int(v) for v in found)

unique_vals = sorted(set(humidity_vals))
pollution_suspected = len(unique_vals) >= 2

print({
    "cave_id": cave,
    "hit_count": len(hits),
    "humidity_values_found": unique_vals,
    "pollution_suspected": pollution_suspected,
    "samples": [{"id": m.id, "content": m.content[:160]} for m in hits[:5]],
})

# Example remediation when review confirms a polluted id (permanent)
# bad_id = hits[0].id
# client.memories.delete(bad_id, user_id=affineur, group=group)

# Prefer pipeline reconciliation when the new state is real:
client.memories.add(
    [
        {
            "role": "user",
            "content": "Correction: cave N3 washed-rind humidity is now 92%, not 85%. Update the standing note.",
        },
        {
            "role": "assistant",
            "content": "Updated cave N3 washed-rind humidity to 92%.",
        },
    ],
    user_id=affineur,
    group=group,
    properties={"cave_id": cave, "memory_class": "climate_setpoint"},
)

The probe does not replace human review on food-safety notes. It makes contradiction visible on a schedule. The follow-up add gives Engram a chance to rewrite through transform instead of leaving two active setpoints forever.

Which Production Controls Limit How Fast Pollution Spreads?

Detection without write control is endless cleanup. Tighten topics so chitchat and speculation do not become facts. Scope by cave, batch, or conversation so one room cannot overwrite another. Use groups to isolate aging ops from retail chat. Log run_id values on every add so a polluted create can be traced to an input transcript.

Sample committed operations on canary traffic. A healthy correction stream should show updates and deletes, not only creates. Create-only growth on preference topics is a drift smell. Empty commits on explicit corrections are another smell. Both deserve topic and transform review.

Keep a quarantine path in the product. When trust is low, exclude flagged ids from the agent prompt even before delete finishes. Wrong humidity guidance is costlier than a brief “memory unavailable” fallback for that probe.

How Should Teams Review Drift Without Freezing the Memory System?

Run pollution checks as batch jobs, not as blockers on every turn. Daily probes on critical scopes are enough for many products. Page humans when contradiction rate crosses a threshold. Feed confirmed bad cases into your evaluation suite so regressions get caught in CI, not only in the cave at 5 a.m.

Do not equate “more memory” with “better memory.” Maintenance is the product. Engram’s extract-transform-commit path is built to reconcile. Your observability and probes are how you verify that reconciliation is actually winning in production.

Pollution and drift are the quiet failures after runs go green. Probe scoped searches for contradictions. Correct through the pipeline when you can. Delete when you must. Keep topics and scopes tight so the next write cannot reseed the same mess.

Our next chapter, How do you A/B test memory configurations?, turns these quality instincts into controlled experiments, and shows how to compare memory setups without guessing from a single bad week.