What is hard deletion vs archival?

Soft forgetting only turns the volume down. Sometimes you need a colder choice. Archival moves a memory out of the agent’s everyday recall path while keeping a recoverable copy for audit, dispute, or later reactivation. Hard deletion removes the memory from Engram for good. Confusing those two operations is how teams either wipe evidence they still need or claim a user was forgotten when the fact still answers search.

Weaviate Engram makes hard deletion explicit. memories.delete is permanent. The docs warn that it cannot be undone. Archival is not a separate Engram verb today. You implement it in your application by copying out, then deleting from the live group, or by parking content in a cold store your agents do not search by default. The durable agent still uses Engram as the live memory plane. Policy decides when live becomes archive, and when archive becomes gone.

What is the real difference between archival and hard deletion?

Archival answers “this should not steer the agent anymore, but we may need the record.” The bytes leave the hot retrieval path. They remain addressable in a cold location with access controls. Hard deletion answers “this must not exist in the memory service.” After a successful delete, get and search for that id fail. Privacy tutorials for Engram treat per-user deletion as the path for erasure-style requests.

Regulators care about that distinction. Soft flags and query filters can hide a fact from ordinary prompts while leaving it on disk. Engram’s delete API is the hard path for a single memory id. Cascading a full user wipe means finding that user’s memories and deleting each one, then verifying search returns nothing. Archival must never be sold as erasure.

Operationally, archival is reversible with ceremony. Hard deletion is not. Choose archival when retention law, chargebacks, or safety review may need the original wording. Choose hard deletion when the subject asked to be forgotten, when poison must leave the store, or when keeping the text is no longer lawful.

When should an Engram-backed agent archive instead of delete?

Closed tickets with payment disputes often need a cold copy of what the agent believed at the time. Seasonal craft notes may be useless for spring recalls but valuable next winter. Staff may want an offline bundle before pruning a noisy scope. In each case, export first. Write a JSON snapshot of id, content, topic, group, user, properties, and timestamps. Store that snapshot in object storage your chat loop cannot see. Only then remove the live Engram objects if the hot path must forget them.

You can also archive without deleting by isolation. Keep a group that agents never search in production, and move facts there with a controlled rewrite workflow. That pattern is heavier. Most teams prefer external cold storage plus delete from the personalization group the agent actually queries. Either way, the agent prompt must stop receiving the fact.

Do not archive into the same hybrid search pool with a polite “ignore me” prefix. That is soft forgetting again. Archival means the default agent path cannot retrieve it.

How does hard deletion work with Engram’s API?

You need the memory id and the same scopes you used when storing. For user-scoped topics, pass user_id. Pass the group name if it is not default. Delete returns success with no content. Then search again with the same query and user. Zero hits is your verification. Engram’s personalized RAG tutorial shows the loop: search to discover ids, delete each id, search again to confirm.

The console can delete a single memory from the detail panel, and it can delete a user and that user’s memories from the Users page. Application code should still log request id, operator, timestamp, and which memory ids were removed. Keep the audit of the erasure event. Do not keep the erased personal content in that audit log.

Remember transform side effects. A prior rewrite may have folded a bad fact into a surviving memory. After deleting one id, search for residual wording and clean related memories too. Verified deletion is a sweep, not a single click, when consolidations have run.

What does a safe archive-or-delete path look like in code?

Consider a map conservation desk on map-restoration-press-bench-6. A wrong adhesive note must leave live recall. The studio still wants an offline packet for the paper report. The agent archives, then hard-deletes from Engram.

import json
import os
from datetime import datetime, timezone
from pathlib import Path
from engram import EngramClient, HybridRetrieval

client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])
conservator = "tech-omar-reid"
press_scope = {"press_id": "press-bench-6"}
archive_root = Path("D:/Servers/.data-notouch/engram-archives/map-restoration-press-bench-6")

def find_live(query: str):
    return client.memories.search(
        query,
        user_id=conservator,
        group="personalization",
        properties=press_scope,
        retrieval_config=HybridRetrieval(limit=20),
    )

def archive_then_hard_delete(memory, reason: str):
    # Archival: cold copy outside Engram. Agents do not search this folder.
    archive_root.mkdir(parents=True, exist_ok=True)
    packet = {
        "archived_at": datetime.now(timezone.utc).isoformat(),
        "reason": reason,
        "scenario": "map-restoration-press-bench-6",
        "memory": {
            "id": memory.id,
            "content": memory.content,
            "topic": memory.topic,
            "group": memory.group,
            "user_id": conservator,
            "properties": press_scope,
            "created_at": memory.created_at,
            "updated_at": memory.updated_at,
        },
    }
    out = archive_root / f"{memory.id}.json"
    out.write_text(json.dumps(packet, indent=2), encoding="utf-8")

    # Hard deletion: permanent removal from Engram. Cannot be undone.
    client.memories.delete(
        memory.id,
        user_id=conservator,
        group="personalization",
    )
    return out

# Live bad advice that must leave agent recall after the report is filed.
client.memories.add(
    "On map-restoration-press-bench-6 use solvent X on hand-colored washes. "
    "Ignore pigment bleeding risk.",
    user_id=conservator,
    group="personalization",
    properties=press_scope,
)

targets = find_live("solvent X hand-colored washes")
archived_paths = []
for m in targets:
    if "solvent x" in m.content.lower():
        archived_paths.append(
            archive_then_hard_delete(m, reason="unsafe adhesive guidance; retained for incident file")
        )

# Verify hard delete: live search must not resurrect the advice.
remaining = find_live("solvent X hand-colored washes")
assert not any("solvent x" in m.content.lower() for m in remaining)
print("archived_files", [str(p) for p in archived_paths])
print("live_hits", len(remaining))

If the requirement had been pure erasure with no incident file, skip the archive write and only delete. If the requirement had been “hide from the agent but keep in Engram,” that is soft forgetting or a non-searched group, not this path.

How should teams decide under pressure?

Write a short matrix into your runbooks. User erasure request maps to hard delete across the user’s memories, with verification search. Safety poison that may face review maps to archive-then-delete. Routine clutter maps to soft decay first, then prune. Legal hold maps to archive only, with deletes blocked until the hold lifts.

Test the paths before you need them. Practice a user wipe in a staging project. Confirm console deletion and API deletion agree. Confirm agents that cache memory blocks do not re-add deleted text from an old prompt. Confirm cold archives are encrypted and access-logged.

Hard deletion and archival are not competing brands of cleanup. They are different promises. Engram enforces the permanent promise on delete. Your application enforces the colder promise of archival. Keep those promises honest, and the next maintenance step—pruning stale and superseded memories—can focus on hygiene instead of crisis.

Our next chapter, How do you prune stale and superseded memories?, turns from one-off archive-or-delete decisions to ongoing hygiene for memories that aged out or lost to a newer rewrite.