Short answer: They drop weak matches by score, not just by top-N count, so unrelated nearest neighbors stay out of context.
Top-k alone always returns something even when nothing is relevant. A fixed similarity cutoff requires genuine closeness. Adaptive cutoffs find a natural score jump between clustered strong hits and the rest. That distinction protects responses from context pollution. Engram’s infrastructure can apply precision control at the right level per search.
Several chapters throughout this Part have mentioned filtering out weak matches before they reach a final response. This chapter looks at that control directly, the different ways a search can actually be told how strict or how lenient to be about what counts as a genuine match, and why a fixed number of results is often the wrong tool for that job.
Why Doesn’t Simply Limiting a Search to Its Top Few Results Actually Solve the Precision Problem on Its Own?
A search returning its closest matches will always return something, since every stored memory has some measurable distance from any given query, even when every single one of those memories is actually unrelated to what the query was asking about. Limiting a search to its top three or five results guarantees a manageable number of results, but it says nothing about whether those results are actually good matches or simply the least-bad ones among an otherwise irrelevant pool. A limit controls quantity, not quality, and those are genuinely different problems that call for genuinely different controls.
What Does a Fixed Similarity Threshold Actually Add That a Simple Result Count Doesn’t?
A threshold sets a specific similarity requirement, a maximum distance or a minimum certainty, that a result has to clear before it’s included at all, regardless of how many results happen to meet that bar. This directly addresses what a limit alone can’t, ensuring that whatever gets returned is actually similar enough to be worth including, rather than simply whatever happened to be closest among an entirely unrelated set. A well-chosen threshold means a search can genuinely return nothing at all when nothing actually qualifies, which is a more honest, more useful outcome than padding a response with weak matches just to hit some fixed count.
Why Might a Single, Fixed Threshold Number Still Not Be the Ideal Way to Set This Kind of Cutoff?
A fixed threshold has to be chosen in advance, without knowing exactly what a specific query’s own result distribution will actually look like, and the right cutoff for one query’s results doesn’t always transfer cleanly to another query where the underlying content is naturally clustered differently. A threshold set to accommodate a query with results spread thinly across a wide range of similarity might let through weak matches on a different query where results cluster in a very tight, high-similarity band immediately followed by a sharp drop into genuinely irrelevant territory.
What Does an Adaptive Cutoff Actually Do Differently from a Fixed Threshold Chosen in Advance?
An adaptive cutoff examines the actual pattern of similarity scores a specific search produced and looks for a natural discontinuity, a noticeable jump in distance between one result and the next, treating that jump as the genuine, natural boundary between results that actually belong together and results that don’t. This approach doesn’t require guessing a fixed number in advance at all, it lets each individual search’s own result distribution determine where the meaningful cutoff actually falls, which can differ considerably from one query to the next depending on how tightly or loosely that query’s genuinely relevant content happens to cluster.
Why Does This Distinction Between Fixed and Adaptive Cutoffs Actually Matter for a Real System’s Output Quality?
Weak, marginally related content injected into a response dilutes and can actively mislead whatever consumes that response, exactly the context pollution concern raised throughout this knowledge base’s earlier discussion of context engineering. A fixed threshold, chosen once and applied uniformly, risks being simultaneously too loose for some queries and too strict for others, while an adaptive cutoff tailors itself to each individual search’s own actual result pattern, catching genuine discontinuities a single, uniform number could easily miss in either direction.
How Does Weaviate Engram’s Underlying Infrastructure Let a System Apply Precision Control at the Right Level for a Given Search?
Weaviate Engram’s search results carry the same similarity scores that Weaviate’s own precision controls, fixed thresholds and adaptive, discontinuity-based cutoffs, are built to operate on, letting a system choose the right level of strictness for a given use case. Consider a museum’s collections-research assistant, helping curators find artifacts genuinely related to a specific piece under study, where padding results with only loosely related items would actively waste a curator’s limited research time:
from engram import EngramClient
client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])
results = client.memories.search(
query="Ceramic glazing techniques similar to this early Ming dynasty vase",
properties={"collection_id": "collection-east-asian-ceramics"},
retrieval_config="hybrid",
)
genuine_matches = [m for m in results if m.score >= 0.65]
A curator researching a specific vase benefits from seeing only artifacts that genuinely share meaningful glazing or technique characteristics, rather than a padded list that includes tangentially related ceramics simply because a fixed result count demanded five items regardless of how few actually qualified. On a different search, where the collection happens to hold several very closely related pieces clustered tightly together in similarity, an adaptive, discontinuity-based cutoff would naturally include all of them, recognizing that they belong together as a genuine group rather than arbitrarily cutting the list off at whatever fixed count was configured in advance. This is exactly the value precision control delivers for a use case like museum research, where a curator’s trust in a search result depends entirely on every returned item genuinely earning its place there.
Similarity thresholds and adaptive cutoffs both work by filtering candidates before they’re returned. A related concern shapes something equally important once results have already passed that filter: how those surviving candidates actually get ordered relative to each other. Our next chapter, How are retrieved memories ranked and scored?, takes up exactly that ordering.