How should you design retrieval APIs for agent frameworks?

An earlier chapter in this Part described tool-based retrieval as an agent deciding for itself when to search memory rather than a search running automatically before every turn. This chapter looks at what actually goes into designing that tool well, the interface an agent framework sees, the description that shapes when an agent reaches for it, and the arguments that let a caller actually control what comes back.

Why Does the Interface Around a Retrieval Function Matter as Much as the Retrieval Logic Behind It?

Most modern agent frameworks can accept a plain function directly as a tool, using that function’s own name and description to decide when and how an agent should call it. This means the interface a developer designs isn’t just internal plumbing, it’s the actual surface an agent’s own reasoning interacts with, and a poorly designed interface can undermine even a technically excellent underlying search. A function that’s hard for an agent to reason about correctly produces bad tool calls regardless of how well the retrieval itself is implemented underneath.

What Actually Makes a Retrieval Tool’s Description Effective at Guiding an Agent’s Decision to Use It?

A tool’s description is arguably the single most important piece of information a calling agent has when deciding whether a given moment actually calls for a search. A vague description like “searches the database” tells an agent almost nothing concrete about when reaching for this specific tool would actually help, while a description naming the specific kind of information the tool surfaces, and the specific situations where searching it pays off, gives an agent a genuine, legible basis for that decision. This is exactly the same guidance covered in the earlier chapter on tool-based retrieval, applied here at the level of how that tool actually gets defined and exposed to a framework.

What Arguments Should a Retrieval Tool Actually Expose to the Agent Calling It?

At minimum, a retrieval tool needs a query argument, since an agent can only usefully search for something if it can express what it’s searching for in its own words. Beyond that minimum, a tool can expose additional arguments a caller might reasonably need to constrain, a specific scope, a specific category, a maximum number of results, but each additional argument also adds something the calling agent now has to correctly understand and fill in. A tool with too many exposed parameters risks an agent misusing or simply ignoring some of them, while a tool with too few risks a caller having no way to express a constraint that genuinely mattered for a specific request.

Should a Retrieval Tool Return Raw, Structured Data or a Ready-to-Use Natural-Language Summary?

Both are legitimate choices, and the right one depends on what happens to the tool’s output next. Returning raw, structured results gives a calling agent maximum flexibility to reason over exactly what came back, filtering, comparing, or citing specific pieces individually, at the cost of a slightly more complex response the agent has to parse and interpret correctly. Returning a single, pre-composed natural-language string is simpler for a calling agent to consume directly, at the cost of losing some of that fine-grained structure the agent might have wanted to reason over on its own. A tool designed for a single, general-purpose framework often favors the simpler string return, since a plain string is the most broadly portable response format across different frameworks and calling conventions.

Does the Underlying Retrieval Behavior Actually Need to Change Depending on Which Agent Framework Is Calling the Tool?

No, and this is one of the genuinely convenient properties of designing retrieval as a plain, well-documented function in the first place. The actual search logic, whatever retrieval type, threshold, and scoping a use case actually calls for, stays exactly the same regardless of whether the calling framework happens to be one that wraps functions with a decorator, one that accepts them directly into a workflow, or one that expects a manually written schema. Only the thin wrapping layer around that same underlying function needs to adapt to a specific framework’s own conventions, keeping the actual retrieval behavior consistent no matter which framework ends up calling it.

How Does Weaviate Engram Support Designing a Retrieval Tool That Works Cleanly Across Different Agent Frameworks?

Weaviate Engram’s search method wraps cleanly into a plain Python function with a clear docstring, giving a calling framework exactly the name, description, and argument signature it needs to decide when and how to invoke it. Consider a veterinary clinic’s scheduling assistant, helping front-desk staff quickly check a pet’s prior visit history before booking a new appointment, where the assistant needs to decide on its own whether a given request actually calls for that lookup:

from engram import EngramClient

client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])

def search_patient_history(query: str, patient_id: str) -> str:
    """
    Searches a specific pet patient's prior visit history and medical notes.

    Use this whenever scheduling a new appointment or answering a question
    that depends on a pet's past visits, treatments, or known conditions.

    Args:
        query: What to look for in the pet's history, in natural language.
        patient_id: The unique identifier for this specific pet patient.

    Returns:
        A natural-language summary of the relevant visit history.
    """
    results = client.memories.search(
        query=query,
        properties={"patient_id": patient_id},
        retrieval_config="hybrid",
    )
    return "\n".join(f"- {m.content}" for m in results)

Because this function’s docstring clearly states what it searches and exactly when to reach for it, any agent framework that accepts plain functions as tools, whether it wraps them with its own decorator or passes them directly into a workflow, can correctly decide to call this function the moment a front-desk request actually depends on a pet’s prior history, without the underlying Engram search logic itself needing to change at all. This is exactly the value a well-designed retrieval interface delivers for a use case like veterinary scheduling, where front-desk staff benefit from an assistant that reliably reaches for a patient’s history exactly when it’s actually needed, regardless of which specific agent framework happens to be running underneath.

Designing a retrieval tool well means the same underlying search works cleanly no matter which framework ends up calling it. A related question sits just beneath this one: once a tool call is actually made, how a system should treat that call within the agent’s own broader reasoning process. Our next chapter, What does it mean to treat retrieval as a first-class tool call?, takes up exactly that question.