How does preference learning work over time?

Preferences are not static labels you write once and retrieve forever. Users change diets, tools, schedules, and taste. An agent that keeps every historical preference equally active eventually answers with yesterday’s truth. Research on long-term agent memory calls this pollution: stale items remain retrievable after newer conflicting evidence arrives. Preference learning over time is the discipline of noticing change, updating the right Engram objects, and keeping revoked history from winning hybrid search.

This chapter separates standing profile updates from accumulating preference facts, shows how Engram’s extract/transform path and explicit supersede/delete flows handle drift, outlines ranking and write-gate habits that respect short-term experiments versus long-term shifts, and walks a cooperage bung-boring desk through a reversal. Next we narrow into updates driven by explicit user feedback signals.

What kinds of preference change should memory treat differently?

Not every new like is a regime change. A potter may try a new glaze for one commission without abandoning their standing palette. A traveler may accept a layover once without becoming a “layover person.” Short-term fluctuations belong in session state or clearly dated unbounded notes. Long-term shifts belong in the bounded UserProfile and in preference topics that transform against prior context so contradictions resolve instead of stacking.

A practical split helps. Standing constraints (“never recommend glitter thread,” “left-handed winding”) update the bounded profile in place. Category preferences (“prefers email,” “likes linen 60/2”) live on unbounded or category topics but should supersede older values under the same key when the user reverses them. Episodic experiments (“tried brand X this week”) stay labeled as trials so they do not outrank a standing rule after the trial ends.

If you treat every utterance as a permanent preference write, you teach the agent to be whiplashed by noise. If you never write reversals, you teach it to be stubborn. Preference learning is knowing which of those errors you are making this turn. Logging whether a write was marked standing, trial, or ephemeral makes that judgment reviewable later.

How does Engram absorb drift without keeping stale actives in the prompt?

On write, send the new conversation into the personalization group with user_id. Topic descriptions should tell the pipeline what counts as a preference versus a one-off. Transform steps such as TransformWithContext reconcile against existing memories: dedupe, merge, and resolve conflicts rather than blindly appending clones. For bounded profile topics, that reconciliation updates the single per-user memory. For unbounded preference topics, you still need application policy when two actives disagree: delete or supersede the old memory, or demote it in ranking so hybrid search stops surfacing it as current.

Append-only preference stores fail hard under full reversal. Studies of revocable evidence memory show append-only and naive last-write-wins can underperform even having no memory when stale conflicts remain visible. Engram’s pipeline is built to maintain clean state—deduplication and preference changes are part of the managed lifecycle—but your orchestrator must still avoid re-injecting archived text and must pass scopes so one user’s new taste cannot rewrite another’s.

On read, prefer the bounded profile plus a small hybrid search over preference topics. If your product keeps explicit “current vs superseded” metadata, filter to current. Soft-forgetting can demote aged notes, but a clear reversal should not wait for decay; it should revoke the old active preference. Decay is for neglect. Revocation is for contradiction.

How do you tell a lasting shift from a temporary exception?

Write gates need a notion of strength. Explicit universal language—“from now on,” “I don’t like X anymore,” “always/never”—is strong evidence for profile or preference supersession. Single-session hedges—“just for this job,” “this once”—should stay in session or job-scoped properties, not rewrite the standing profile. Repeated consistent choices across sessions can promote a trial into a standing preference even without slogan language; one counterexample should not instantly erase a year of signal unless the user is explicit.

Some teams track short-window versus long-window preference estimates and update when they diverge. You do not need a research module to apply the spirit: require more evidence to change long-standing profile clauses than to add a dated like. When unsure, ask a confirmation turn before committing a destructive supersede. Memory updates that surprise the user feel like identity theft. A one-line confirmation is cheaper than a week of baffling personalization.

Keep organizational policy out of preference drift. “I wish overnight shipping were free” is not permission to invent free overnight. Preferences choose among allowed options; SOPs still live on the organizational plane.

What does preference reversal look like at a cooperage bung desk?

A workshop assistant helps coopers at a bung-boring desk. Cal used to want wider bung allowances; now he wants tighter fits on export barrels. Scenario id: cooperage-bung-boring-bench-7.

from weaviate.engram import EngramClient
from weaviate.engram.retrieval import HybridRetrieval, FetchRetrieval

engram = EngramClient()
user_id = "cooper-cal"
group = "personalization"
scenario = "cooperage-bung-boring-bench-7"

# Strong reversal language -> update standing preference / profile
reversal = [
    {
        "role": "user",
        "content": "From now on, stop suggesting wide bung allowances on export barrels. I want a tighter fit.",
    }
]
run = engram.memories.add(
    content=reversal,
    group=group,
    scopes={"user_id": user_id, "properties": {"scenario": scenario}},
)
# Pipeline transform reconciles with prior bung-preference memories;
# bounded UserProfile updates in place if standing constraints are in profile.

# If an old unbounded memory remains active, revoke explicitly
# engram.memories.delete(memory_id="mem_cal_wide_bung_v1")

# Temporary exception -> session/job scope, not standing rewrite
exception = [
    {
        "role": "user",
        "content": "Just for repair ticket R-229, use the wider allowance once.",
    }
]
engram.memories.add(
    content=exception,
    group=group,
    scopes={
        "user_id": user_id,
        "properties": {"scenario": scenario, "ticket_id": "R-229", "ephemeral": "true"},
    },
)

current = engram.memories.search(
    query="bung allowance export barrels preference",
    group=group,
    topics=["bung_preferences", "UserProfile"],
    retrieval=HybridRetrieval(alpha=0.5, limit=5),
    scopes={"user_id": user_id},
)

After the “from now on” turn, retrieval should surface the tighter-fit preference—and the wide-allowance note should be transformed away, deleted, or otherwise inactive. The R-229 exception lives under ticket scope so it cannot redefine Cal’s standing taste next month. That is preference learning over time: detect strength, choose the right topic and scope, let Engram reconcile, and refuse to keep stale actives in the prompt.

Learning preferences across sessions means treating drift as a first-class write problem: bounded profiles for standing identity, reconciled preference topics for category tastes, ephemeral scopes for one-offs, and revocation so stale likes stop polluting search. Our next chapter, What are feedback-driven memory updates?, focuses on updates triggered by explicit thumbs, corrections, and rated outcomes rather than inferred drift alone.