Short answer: Because they re-solve problems they already figured out, spending fresh reasoning and tool calls instead of recalling prior results.
Resending chat history is one cost. Re-deriving the same conclusions is another. Without memory, each task rebuilds context from scratch. Storing useful outcomes cuts repeated work and wasted tokens.
There’s a cost to resending conversation history on every call, and it’s already been shown to add up fast. There’s a second, quieter cost that has nothing to do with resending text at all: the cost of an agent spending real reasoning effort, tool calls, and multiple back-and-forth steps re-figuring out something it has, in a meaningful sense, already figured out before. Without memory, every task starts that reconstruction process from nothing, and for an agent doing real work rather than answering a single question, that reconstruction isn’t cheap. It’s paid for in tokens, in time, and often in mistakes that a system with the right context wouldn’t have made in the first place.
What Does “Repeated Reasoning” Actually Look Like in Practice?
Picture an agent picking up a task cold: reviewing a codebase it’s touched before, or continuing a multi-week project it has no record of having worked on. Without memory, it has to reconstruct everything from scratch, reading through files, inferring why a particular decision was made, piecing together intent from whatever artifacts happen to still be lying around. None of that reconstruction is free. Each exploratory step, each file read, each moment spent inferring “why did we do it this way” instead of simply knowing, costs tokens and often an entire extra round of tool calls and model reasoning before the agent even gets to the actual task in front of it.
This is a fundamentally different kind of waste than resending a long conversation transcript. Resending history costs tokens for content that was already fully formed and just needs to be repeated. Reconstructing reasoning costs tokens for work that has to happen all over again, work that produces the same conclusion it already produced once, just recomputed at full price instead of recalled.
Why Does This Cost Compound Faster for Agents Than for Simple Chat?
A simple back-and-forth chat only reconstructs context once per conversation, however long that turns out to be. An agent doing real, multi-step work reconstructs it far more often, because agents operate in tight loops of reasoning, acting, and observing results, often many times within a single task, and they take on many separate tasks in a single day, each one potentially starting from the same cold, memoryless state.
Multiply the reconstruction cost of one task by how many times an agent repeats a similar task, across sessions, across days, across an entire deployment, and the picture changes from “a bit of wasted effort here and there” to a recurring tax that gets paid over and over on work that’s already effectively been done. Agents also produce and consume information far faster than people do, which means this waste compounds on a much shorter timescale too. What might look like a minor inefficiency in an isolated demo becomes a serious, continuous drag once the same agent is handling dozens or hundreds of similar tasks.
Is There Concrete Evidence This Waste Actually Shows Up?
This isn’t just a theoretical worry. In an internal comparison at Weaviate, two identical coding-assistant sessions were given the exact same task, picking up a multi-week product writeup and working out where things had been left off, with everything else about the setup held constant. The only difference was whether the session had access to grounded, persistent memory of earlier work on the same project.
The session without memory got to the right answer. It read through the available files, reconstructed the reasoning correctly, and produced a solid summary of where things stood. But it had to build that understanding from raw material every step of the way, and it took meaningfully longer to do it, close to thirty percent longer on the very first exchange, purely because of the reconstruction work involved. The session with grounded memory didn’t have to reconstruct anything; it recalled the actual reasoning chain behind an earlier decision directly, arriving at the same understanding faster because the work had already been done and simply needed to be retrieved. A related finding made the risk of reconstruction even more concrete: when context was incomplete, the version without grounded memory filled the resulting gap by fabricating a plausible-sounding detail, a specific URL that didn’t actually exist, and it did so consistently across separate runs. The version with grounded memory had enough real context available that it didn’t need to guess, and it avoided the fabrication both times.
Isn’t This the Same Fix as Reducing Token Costs From Resending History?
It’s tempting to assume that whatever solves the history-resending problem automatically solves this one too, since both involve avoiding repeated tokens, but they’re solving genuinely different problems. Cutting down resent history means not paying twice for conversation text that was already sent once. That doesn’t automatically capture anything about why a decision was made, what an agent concluded after investigating something, or what reasoning path led to a particular outcome, unless something deliberately extracts that reasoning and stores it as its own piece of memory, separate from the raw conversation it came from.
A system that only trims or searches over conversational facts will still leave an agent reconstructing its own conclusions from scratch every time, because a conclusion isn’t the same kind of content as a preference or a fact about a user. It’s the outcome of work, and treating it as worth remembering, on purpose, is a separate design decision from simply managing how much raw conversation history gets resent.
How Does Weaviate Engram Target This Specific Kind of Waste?
Capturing an agent’s own conclusions as memory, not just facts about users, is exactly the kind of thing Engram’s topic system is built to support. Consider a data-analytics agent that regularly gets asked to investigate sales anomalies for a retail company. Every time it tracks down a root cause, that diagnosis is worth keeping, so the next similar anomaly doesn’t require redoing the same investigation from zero:
from engram import EngramClient
client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])
client.memories.add(
[
{
"role": "assistant",
"content": "Investigated the March dip in the Midwest region — traced it to a shipping carrier outage, not a demand drop. Regional dips that align with carrier-status incidents should be checked against the carrier feed before assuming a demand issue.",
}
],
user_id="sales-ops-team",
)
Weeks later, when a similar-looking anomaly appears in a different region, the agent can search for whether it’s already worked through something like this before starting its investigation from nothing:
results = client.memories.search(
query="Have we seen a regional sales dip like this before, and what turned out to cause it?",
user_id="sales-ops-team",
)
If the pattern matches, the agent starts from an actual prior conclusion rather than re-running the same multi-step investigation it already completed once. That’s the direct target of this kind of memory: not just cutting down what gets resent as conversation history, but making sure the actual work an agent has already done doesn’t have to be paid for a second time just because nothing was built to remember it happened.
Everything discussed in this Part so far has treated an agent’s interactions as though they all take the same basic shape, one continuous back-and-forth that either does or doesn’t get remembered. In practice, agents operate across meaningfully different shapes of interaction, from a single isolated question to a conversation that spans many turns to a relationship that spans many separate sessions over time, and each shape changes what memory actually needs to do. Our next chapter, What is the difference between single-turn, multi-turn, and multi-session systems?, breaks down exactly that distinction.