How do you back up and restore persistent memory?

Scaling keeps a memory cluster alive under load. Backup and restore keep that memory recoverable when something still goes wrong. Persistent agent memory is not a disposable cache. It is the accumulated identity of users, projects, and procedures your product depends on. This chapter covers what a durable backup must capture for vector-native memory, how Weaviate’s cloud-native backup modules create consistent copies without downtime, how managed Engram and self-hosted paths differ on restore ownership, and how to verify that Weaviate Engram still answers correctly after a restore—not merely that files landed on disk.

Why Is Persistent Memory Harder to Protect Than Ordinary App State?

Ordinary databases back up rows. Memory systems also back up meaning. Vectors, inverted indexes, object payloads, and scope metadata must stay aligned. A restore that returns objects without their vectors forces expensive re-embedding. A restore that returns vectors without the right filters leaks or blanks scoped recall. Agent behavior depends on that alignment. If the store is wrong, the agent does not merely lose data. It loses continuity.

Memory also changes continuously through asynchronous pipelines. Engram accepts writes quickly and commits extracted memories in the background. A naive disk image taken mid-flight can miss in-flight runs or capture a half-reconciled fact. Good backup design pairs a durable store snapshot with an application canary. You need a known memory you can search after restore to prove the product still works.

That raises a practical question. What exactly should enter the backup set before you trust a restore drill?

What Belongs in the Consistency Group for Agent Memory?

Treat recovery as a consistency group, not a single volume. The vector store holds memories and indexes. Configuration holds groups, topics, and retrieval defaults. Secrets and API keys are not backup payloads, but the runbook must know how to reattach them. Embedding model version pins matter too. Restoring vectors computed under one model into a stack that searches with another silently degrades recall.

Weaviate backups are built for this class of store. A backup can cover the whole instance or selected collections. The snapshot includes objects, vectors, and indexes, so restore does not require rebuilding HNSW from scratch. Backups can run while the cluster stays available for reads and writes. That property matters for memory products that cannot schedule a nightly outage window.

For production, store backups on an external backend such as S3, GCS, or Azure Storage. Filesystem backups suit development on a single node. Multi-node clusters need an external provider so the backup survives the loss of any one machine. Incremental backups reduce transfer size by storing changed data, which keeps frequent schedules affordable as memory volume grows.

How Do Managed Engram and Self-Hosted Paths Differ on Restore?

Once the consistency group is clear, ownership of the restore button decides your recovery time. On Weaviate Cloud, daily backups are part of the service. Retention follows the cluster plan. Additional backups can be triggered through the API. Restore is not a casual self-serve click for every plan. Contact support and bake that handoff into your recovery time objective. Managed Weaviate Engram inherits that operational story for the memory collections it persists.

Self-hosted Weaviate puts the schedule, retention, and restore drills on your team. You enable a backup module, point it at a bucket, and automate backup.create with unique IDs. You poll status asynchronously. You restore onto a target that meets restore requirements, including compatible topology and collections that do not already exist on the destination. Partial include lists help when you only need the memory collections in a larger cluster.

Either path still needs an application check. Infrastructure success is necessary. It is not sufficient. The next step is proving Engram recall after the bytes are back.

How Do You Prove Engram Still Remembers After a Restore?

Before a maintenance window or as a standing canary, write a small set of scoped memories that only your restore drill knows. After restore, search those scopes with the same user, group, and properties. Confirm content, hit counts, and hybrid ranking still look right. If the canary fails, do not reopen traffic. Investigate embedding pins, collection selection, and whether inactive tenants were included for multi-tenant setups.

Also decide what to do with writes that arrived after the backup timestamp. Your recovery point objective is the gap you accept. Some teams pause nonessential memories.add traffic briefly around a coordinated checkpoint. Others accept a short gap and re-seed critical facts from an upstream system of record. Document the choice. Guessing during an outage creates a second incident.

Here is a clocktower workshop desk that seeds canary memories in Weaviate Engram and verifies them after a restore window. The backup itself runs through Weaviate’s backup API on the underlying store. The Engram checks prove the product layer still works.

import os
from engram import EngramClient, HybridRetrieval

client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])
group = "horology_lab"
curator = "curator-jonas"
movement = "clock-tower-west"

# Seed canary facts before backup (wait for commit in the drill, not on every chat turn)
seed = client.memories.add(
    [
        {
            "role": "user",
            "content": (
                "Clock Tower West: the gravity escapement needs a two-degree "
                "advance on cold mornings below 4C, and visitors must not "
                "touch the pendulum bob during tours."
            ),
        },
        {
            "role": "assistant",
            "content": "Logged cold-weather escapement offset and tour safety rule.",
        },
    ],
    user_id=curator,
    group=group,
    properties={"movement_id": movement, "cost_center": "tower_ops"},
)
client.runs.wait(seed.run_id)

# ... Weaviate backup.create / restore completes on the underlying store ...

def verify_after_restore() -> dict:
    hits = client.memories.search(
        query="What cold-weather escapement offset applies at Clock Tower West?",
        user_id=curator,
        group=group,
        properties={"movement_id": movement},
        retrieval_config=HybridRetrieval(limit=3),
    )
    joined = " ".join(m.content for m in hits).lower()
    ok = ("two-degree" in joined) and ("pendulum" in joined)
    return {
        "movement_id": movement,
        "hit_count": len(hits),
        "canary_ok": ok,
        "previews": [m.content[:120] for m in hits],
    }

print(verify_after_restore())

Keep the canary narrow and boring. Wide synthetic dumps hide failures. A single scoped truth that must survive restore is enough to catch the common mistakes.

Which Operational Habits Keep Restores Trustworthy?

Schedule restores into staging on a calendar, not only after a real failure. Measure wall-clock restore time against your recovery time objective. Measure data lag against your recovery point objective. Alert when the last successful backup is older than your policy. Track backup storage growth beside memory growth so cost surprises do not force you to thin retention below compliance needs.

Separate backup storage from the primary region when you can. Cross-cloud backup is supported in Weaviate’s design. A cluster on one provider can land backups on another. That separation is what makes a backup useful when the primary environment is the thing that failed.

Version the runbook with the stack. Note Engram group names, topic templates, and embedding pins beside backup IDs. After each drill, record whether the canary search passed. A green backup job with a red canary is still a failed drill.

Persistent memory earns trust when you can lose a node and still recover identity on purpose. Capture objects, vectors, and indexes together. Prefer external backup backends in production. Know who owns restore on managed versus self-hosted paths. Prove Engram recall with scoped canaries after every restore. Our next chapter, How do you plan disaster recovery for memory infrastructure?, widens the lens from a single restore drill to full outage playbooks, failover order, and how to keep agents honest while the memory tier comes back.