Short answer: Because the model can only use tools it can see described in the current prompt, and those definitions consume budget.
Tools extend action beyond recall, but their schemas and instructions occupy the same window as memories and chat. Context engineering has to treat tool definitions as first-class context: necessary, costly, and easy to overgrow.
Retrieval bridges stored memory into a live context window, letting a model reason with facts it couldn’t have known on its own. Tools extend that same idea into action rather than pure recall, letting a model reach outside its own reasoning to check something live, change something in the real world, or trigger a process no amount of retrieved text could substitute for. But before a model can call any tool at all, it needs to know that tool exists and understand what it does, and that knowledge itself has to live somewhere: inside the context window, consuming exactly the same scarce budget as everything else covered in this Part.
What Does a Tool Definition Actually Consist Of, and Why Does It Cost Context Space?
A tool definition is made up of a name, a description explaining what the tool does and when to use it, and a specification of what inputs it expects. All three pieces have to be present in the context window before a model can ever decide to call that tool, because the model has no other way of knowing the tool exists or how to use it correctly. This means a long list of available tools, each with a verbose description, adds up to a real, non-trivial chunk of the token budget before a single word of actual conversation or retrieved memory has even entered the picture.
This cost is easy to overlook because tool definitions feel like infrastructure rather than content, something that’s just “always there” rather than something actively competing for space. But from the model’s perspective, and from the token budget’s perspective, a tool definition is exactly as real a consumer of context as a retrieved memory or a line of conversation history.
Why Does the Quality of a Tool’s Description Matter as Much as Its Existence?
The description is arguably the single most important piece of a tool definition, because it’s what the model actually reasons over when deciding whether a given tool is the right one for the current situation. A vague or generic description, something like “searches for information,” gives the model little to work with when trying to decide whether this tool, specifically, is the right choice among several similarly vague options. A precise description that states exactly what the tool is useful for and when it should be used lets the model make that decision correctly far more often.
This means the same context-budget discipline applied everywhere else in this Part applies here too: a longer, more thorough description costs more tokens, but a description that’s too terse to actually guide correct tool selection wastes whatever tokens it does spend, since a poorly chosen tool call produces wasted effort and a worse outcome regardless of how few tokens the description itself consumed.
What Happens When a System Provides Too Many Tools at Once?
A long list of available tools, each competing for the model’s attention alongside everything else in context, creates exactly the kind of confusion already covered as a general context-engineering failure mode: irrelevant options crowding the window and increasing the odds the model picks the wrong one, or hesitates between several plausible-looking choices when only one was actually appropriate for the current task. This problem gets worse as tool count grows, since each additional tool both consumes more context space and adds one more plausible-but-wrong option the model has to correctly rule out.
This is why many well-designed agent systems don’t expose every possible tool on every single call. Instead, they narrow the available set based on what the current task actually seems to require, keeping the model’s tool-selection decision focused on a small, genuinely relevant set rather than an exhaustive catalog of everything the system could theoretically do.
How Does Standardization Change How Tool Definitions Get Assembled?
Emerging standards for describing tools, like the Model Context Protocol, reduce the burden of hand-crafting a custom description for every single integration by giving tools a consistent, predictable shape that any compatible system can consume directly. This doesn’t eliminate the token cost of including a tool’s definition in context, that cost is inherent to the model needing to see what a tool does, but it does reduce the engineering overhead of assembling and maintaining those definitions across many different integrations, which indirectly makes it easier to keep descriptions consistently well-written rather than varying wildly in quality across a system’s tool catalog.
How Does Exposing Weaviate Engram’s Search as a Tool Fit Into This Picture?
Weaviate Engram’s `memories.search()` call can itself be exposed to a model as a tool, with its own name, description, and expected arguments, letting an agent decide for itself when a memory lookup is actually needed rather than having that decision made rigidly before every single call. Consider a municipal 311 service-request triage assistant helping residents report issues like potholes, missed trash pickups, or broken streetlights, where the assistant benefits from deciding for itself when a resident’s history is actually relevant:
from engram import EngramClient
client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])
def search_resident_history(query: str, resident_id: str) -> str:
"""
Search this resident's past service requests and any notes about
recurring issues at their address. Use this when a new request
might be related to something previously reported, not for
unrelated first-time reports.
Args:
query: What to look for in the resident's history.
resident_id: The resident's unique identifier.
Returns:
Relevant past service-request history, if any exists.
"""
results = client.memories.search(
query=query,
user_id=resident_id,
retrieval_config=HybridRetrieval(limit=5),
)
return "\n".join(f"- {m.content}" for m in results)
The description here is doing real work, telling the model exactly when this tool is worth calling, a possible recurring issue, and explicitly when it isn’t, a first-time unrelated report, which reduces the odds the model calls it unnecessarily for every single request regardless of relevance. If a resident reports a brand-new pothole with no reason to suspect any history matters, the model can reasonably skip this tool entirely rather than spending a retrieval call and its accompanying tokens on a lookup unlikely to surface anything useful. If a resident reports what sounds like a fourth complaint about the same streetlight, the tool’s description makes clear this is exactly the situation it was built for. Getting that description right is what lets the model make good, selective tool-use decisions rather than either ignoring memory entirely or querying it reflexively regardless of whether the current situation calls for it.
Tool definitions describe what an agent can do, but system prompts describe something more foundational: the standing identity, instructions, and constraints that shape everything an agent does across every single call, in a way that behaves surprisingly like a form of memory in its own right. Our next chapter, Are system prompts a form of long-lived memory?, takes up exactly that idea.