Short answer: Flag memories that still retrieve well and look authoritative but no longer match today’s environment or ground truth.
Age alone is not staleness; a young fact can be wrong after a migration. Use Engram timestamps plus outside checks, then gate retrieval or refresh scoped writes. Never refresh across the wrong user or property. Measure stale-fact errors separately from prune volume. The next design choice is how aggressive refresh should be versus stability.
Staleness is the failure mode where a memory still retrieves cleanly and still looks authoritative, yet the world has moved on. Maintenance jobs merge duplicates and expire obvious junk. Staleness detection catches the survivors that are age-appropriate, well ranked, and wrong for today’s environment. Long-running agent fleets hit this early. They write faster than humans review. An old library pin or API base path can sit in Weaviate Engram for months, score high on hybrid search, and steer the next deploy into a dead end.
This chapter defines operational staleness versus mere age, shows signals you can compute from Engram timestamps and outside ground truth, explains how to gate retrieval when a fact is suspect, and walks through a detector that flags and refreshes scoped memories. The tension afterward is how aggressive to be. Stability versus plasticity is the design choice once you can see what is stale.
What counts as stale in a long-running deployment?
After nightly cleanup, many old memories are intentionally gone. What remains can still be stale. Age alone is a weak label. A safety constraint from last year may still be correct. A “use endpoint /v1/widgets” note from last week may already be false after a rename. Staleness means the memory’s claim no longer matches current external state or current policy, regardless of how freshly it embeds.
Research on evolving agent state separates retrieval failure from state drift. The fact is in context, and the agent still acts on an outdated version. Similarity search makes that worse. Old and new values often sit near each other in embedding space. Engram transform and contradiction scans help when both values are stored. Staleness detectors are needed when only the old value remains, still unchallenged inside the memory store.
Classify by topic. Configuration and tool-routing memories go stale when systems change. Preference memories go stale when users drift. Episodic notes go stale when the project phase ends. Use different thresholds and refresh strategies per class instead of one global TTL fantasy.
Which signals reveal staleness before users complain?
Once you know the definition, collect evidence. Engram exposes created_at and updated_at on every memory. Those stamps power age and “time since last amend” features. Pair them with application signals. Tool error rates after the agent cited a memory. Version manifests from your deploy system. Schema digests from the APIs the agent calls. A memory that names package 1.4 while production runs 2.1 is stale even if updated_at is recent because someone rephrased the same wrong pin.
Ground-truth probes beat vibes. For high-value scopes, store a small checklist of facts the agent must get right, and re-ask them on a schedule with tools disabled from using memory, then with memory enabled. If tool-backed truth and memory-backed answers diverge, flag the retrieved ids. Track a staleness rate metric: fraction of evaluated questions where the top memory asserts a superseded value.
Weaviate’s object TTL features on the database side show the same instinct for session logs and caches. Engram memories are curated facts, so blind TTL is often too blunt. Prefer detect-then-refresh over delete-by-birthday for durable personalization, and reserve hard expiry for clearly temporary topics.
How should a stale hit change retrieval and write-back?
Detection without a response still harms users. When a memory is flagged stale at answer time, demote it in the prompt, attach an uncertainty note, or exclude it until refreshed. Ask the agent to verify with a tool when the claim is operational. After verification, write the corrected fact through memories.add so Engram transform can rewrite the old neighbor. That is reconsolidation triggered by fleet health checks rather than by a user’s “actually.”
Batch detectors feed maintenance jobs. Nightly scans mark candidates. Jobs attempt refresh from ground truth. Failures open tickets instead of silent deletes. Successes leave a short history clause in the rewritten memory so operators can see what changed.
Never refresh across the wrong scope. A stale API tip for one service property must not rewrite another team’s user_id or project group. Keep searches and deletes inside the same group and properties the living agent uses.
What does a deployment-scoped staleness check look like?
Imagine a radio observatory agent on radio-telescope-pointing-desk-7. It still remembers an old pointing model coefficient after the array switched catalogs. The detector compares memory text to a live config digest and refreshes on mismatch.
import os
import re
from datetime import datetime, timezone, timedelta
from engram import EngramClient, HybridRetrieval
client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])
operator = "ops-mira-sol"
desk = {"desk_id": "radio-telescope-pointing-desk-7"}
# Ground truth from the deployment control plane (mocked here).
LIVE_CATALOG = "ICRF3"
LIVE_MODEL = "pointing-model-2026.08"
def parse_ts(value: str) -> datetime:
return datetime.fromisoformat(value.replace("Z", "+00:00"))
def extract_catalog(text: str) -> str | None:
m = re.search(r"catalog\s+([A-Z0-9]+)", text, re.I)
return m.group(1).upper() if m else None
def extract_model(text: str) -> str | None:
m = re.search(r"pointing-model-[\w.]+", text, re.I)
return m.group(0).lower() if m else None
def seed_stale():
run = client.memories.add(
"On radio-telescope-pointing-desk-7 use catalog ICRF2 with pointing-model-2024.11.",
user_id=operator,
group="personalization",
properties=desk,
)
client.runs.wait(run.run_id)
def detect_staleness(query: str = "pointing catalog model"):
now = datetime.now(timezone.utc)
hits = client.memories.search(
query,
user_id=operator,
group="personalization",
properties=desk,
retrieval_config=HybridRetrieval(limit=10),
)
flags = []
for m in hits:
catalog = extract_catalog(m.content)
model = extract_model(m.content)
age = now - parse_ts(m.updated_at)
reasons = []
if catalog and catalog != LIVE_CATALOG:
reasons.append(f"catalog {catalog} != {LIVE_CATALOG}")
if model and model != LIVE_MODEL:
reasons.append(f"model {model} != {LIVE_MODEL}")
if age > timedelta(days=180) and ("pointing" in m.content.lower()):
reasons.append("aged pointing guidance")
if reasons:
flags.append({"memory": m, "reasons": reasons})
return flags
def refresh_stale(flags):
if not flags:
return []
correction = (
f"On radio-telescope-pointing-desk-7 use catalog {LIVE_CATALOG} "
f"with {LIVE_MODEL}. Prior ICRF2 / 2024.11 guidance is retired."
)
run = client.memories.add(
correction,
user_id=operator,
group="personalization",
properties=desk,
)
client.runs.wait(run.run_id)
# Remove absolute old rows if they still retrieve beside the rewrite.
for item in detect_staleness():
m = item["memory"]
if LIVE_CATALOG in m.content and LIVE_MODEL in m.content:
continue
client.memories.delete(m.id, user_id=operator, group="personalization")
return client.memories.search(
"pointing catalog model",
user_id=operator,
group="personalization",
properties=desk,
retrieval_config=HybridRetrieval(limit=5),
)
seed_stale()
flagged = detect_staleness()
print("flagged", [(f["memory"].id, f["reasons"]) for f in flagged])
live = refresh_stale(flagged)
for m in live:
print(m.content)
The live catalog and model strings are the source of truth. Engram remains the agent-facing memory. The detector only decides when the two disagree. That split keeps embeddings from being asked to judge temporal validity alone.
How do you run staleness detection across a fleet without thrashing?
Sample continuously, refresh carefully. Probe critical scopes daily. Probe the long tail weekly. Cap automatic rewrites per scope per day. Require human approval when the ground-truth source is itself uncertain. Measure stale-fact errors on held-out operational questions before and after refreshes.
Separate dashboards for age-based prune volume and staleness flags. A store can be young and still stale after a big platform migration. A store can be old and still correct for durable preferences. Treat those curves independently.
Seeing staleness clearly forces a product choice. Refresh everything aggressively and you get brittle thrash. Freeze too much and agents lecture from the past. Balancing stability and plasticity is how you set the knobs this detector exposes.
Our next chapter, How do you balance stability and plasticity in memory?, turns those knobs into an explicit design trade-off for Engram-backed agents.