What is hard isolation vs soft isolation?

Earlier chapters in this Part have used the terms hard isolation and soft isolation without pausing to define them precisely. This chapter draws that distinction out directly, since understanding exactly what separates the two clarifies why a memory system should never treat them as interchangeable, even though both genuinely count as real isolation.

What Actually Defines Hard Isolation as Categorically Different From Any Other Kind of Separation?

Hard isolation means a boundary that cannot be crossed under any circumstance a caller controls, enforced at the storage layer itself rather than through a filter a query happens to include. A search running under one identity has no path whatsoever to data belonging to a different identity, not because nobody thought to ask for it, but because the underlying storage never made that data reachable from this identity’s query in the first place. There’s no parameter to omit, no filter to forget, that would ever expose the other side of a hard boundary, because the separation doesn’t depend on any parameter being supplied correctly.

What Actually Defines Soft Isolation, and Why Does It Still Deserve to Be Called Isolation at All?

Soft isolation means a boundary that’s genuinely enforced when a caller chooses to apply it, but that the caller can also deliberately choose not to apply, searching more broadly across everything that boundary would otherwise separate. This is still real isolation in every sense that matters when it’s actually in use, memories tagged to one conversation are kept genuinely distinct from memories tagged to a different one whenever a search filters by that distinction. What makes it soft is that crossing the boundary is a legitimate, intended option rather than something the system structurally forbids.

Why Does a Memory System Actually Need Both Kinds of Isolation Rather Than Just Standardizing on the Stricter One?

Standardizing on hard isolation everywhere would solve privacy concerns thoroughly but would make some genuinely useful searches impossible, since a caller could never deliberately search across multiple conversations, sessions, or categories that a hard boundary would keep permanently separate. Standardizing on soft isolation everywhere would preserve that flexibility but would leave user privacy dependent on every caller remembering to apply the right filter correctly every time, exactly the fragile pattern this Part has argued against since its opening chapter. The two kinds of isolation solve different problems, and a well-designed system applies each one to the specific boundary it’s actually suited for.

How Should a Team Actually Decide Which Boundaries in Their Own System Deserve Hard Isolation Versus Soft Isolation?

The decisive question is whether crossing a given boundary is ever legitimately something a caller should be able to do deliberately. A boundary between two different users’ private data should never be crossed under any circumstance, no legitimate feature should ever need to search across users, which makes it a clear candidate for hard isolation. A boundary between two different conversations belonging to the same user is a different matter entirely, since a system might genuinely benefit from searching across all of that user’s conversations at once in certain contexts, which makes it the right kind of boundary for soft isolation instead.

What Actually Went Wrong Before Systems Like Weaviate Built Genuine Hard Isolation Directly Into Their Storage Architecture?

Before storage-level multi-tenancy existed as a first-class capability, isolating tenants generally meant one of two flawed approaches, either giving each tenant an entirely separate schema, which became unmanageable past a modest number of tenants, or relying on a single shared index filtered by a tenant identifier at query time, which meant a giant, mostly-irrelevant index had to be searched and filtered down for every single query. That filter-based approach is exactly what soft isolation looks like when it’s mistakenly applied to a boundary that actually needed to be hard, functional most of the time, but resting on every query correctly including the right filter rather than on a structural guarantee that could never be bypassed by mistake.

How Does Weaviate Engram Apply Hard and Soft Isolation to the Different Boundaries a Real System Actually Needs?

Weaviate Engram applies hard isolation to user boundaries through Weaviate’s native multi-tenancy, and soft isolation to custom property boundaries through optional, caller-controlled filters, matching each mechanism to the kind of separation it’s actually built for. Consider a freelance marketplace platform where a contractor manages several distinct client engagements at once, each with its own project notes, while the platform must guarantee one contractor’s data structurally can never reach a different contractor’s search:

from engram import EngramClient

client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])

client.memories.add(
    "Client requested the homepage redesign prioritize mobile load speed over animated visual effects.",
    user_id="contractor-freelance-6690",
    properties={"engagement_id": "engagement-acme-redesign"},
)

this_engagement_only = client.memories.search(
    query="What did this client prioritize for the homepage redesign?",
    user_id="contractor-freelance-6690",
    properties={"engagement_id": "engagement-acme-redesign"},
)

all_engagements = client.memories.search(
    query="What design priorities have come up across my current client work?",
    user_id="contractor-freelance-6690",
)

The contractor’s own user scope enforces hard isolation, structurally guaranteeing a different contractor’s search could never reach this data no matter what parameters they supply, while the engagement identifier enforces soft isolation, letting this contractor narrow down to one specific client relationship or deliberately broaden across every current engagement depending on what a given moment actually calls for. This is exactly the value applying each kind of isolation to its proper boundary delivers for a use case like a freelance marketplace, where cross-contractor privacy needs to be absolute while a single contractor’s own organization of their own work needs room to flex.

Hard and soft isolation aren’t competing approaches, they’re complementary tools, each suited to a different kind of boundary a real memory system needs to draw. Understanding the distinction sets up the practical question this Part has been building toward from its very first chapter: what it actually takes, concretely, to prevent cross-user leakage from ever happening in the first place. Our next chapter, How do you prevent cross-user data leakage by design?, takes up exactly that question.