What is working memory in an agent system?

Every agent has working memory, whether anyone designed it deliberately or not, because it’s simply whatever the model can see while generating its current response. It’s worth defining precisely, on its own terms, before moving through the other kinds of memory that build on top of it, because working memory isn’t just “the context window” restated with a different name. It has a specific, bounded job, specific things that do and don’t belong inside it, and a particular relationship to every other kind of memory that follows: nothing else matters at all unless it eventually passes through this one narrow space.

What Actually Counts as Part of an Agent’s Working Memory?

Working memory is everything present in a single call to the model, in full. That includes the obvious parts, the system instructions and the user’s message, but it also includes the model’s own intermediate reasoning from earlier in the same task, the outputs of any tools that have already run, and anything pulled in from long-term storage specifically for this step. If it’s part of what the model actually reads before producing its next output, it’s working memory, regardless of where it originally came from.

This is a wider category than it first sounds. A retrieved memory, the moment it gets inserted into a request, is functioning as working memory for that call, even though a second ago it was sitting in long-term storage instead. The defining trait isn’t origin, it’s presence: whatever’s actually in front of the model right now, all of it, counts.

Why Call It Memory at All, if It Disappears the Instant the Call Ends?

Calling something this transient “memory” can feel like a stretch, given that none of it survives past the one call it belongs to. Within the boundary of that single call, though, it does exactly what memory is supposed to do: it lets later reasoning build on earlier reasoning within the same response, lets a tool’s result actually inform the next decision the model makes, and keeps the model’s own output internally consistent across many sentences rather than contradicting itself paragraph to paragraph.

The word “memory” here describes a function, not a duration. It’s information the current reasoning process genuinely depends on having access to, and for the length of that one process, it behaves exactly the way memory is supposed to: available, load-bearing, and necessary for coherent output. Whether it survives another five seconds afterward is a completely separate question from whether it’s doing memory’s job right now.

What Specifically Falls Outside of Working Memory?

Anything not actually present in the current call’s input sits outside this category, no matter how relevant it might be. A fact stored days ago but not yet retrieved for this particular request isn’t working memory. A stable preference sitting in long-term storage, waiting for a search that hasn’t happened yet, isn’t working memory either. Neither is anything belonging to a different user’s session, even one running through the exact same agent at the exact same moment.

Any of these things can become working memory the instant they’re retrieved and inserted into a request, but until that happens, they belong to a different category entirely, something stored and waiting rather than something present and in use. The line is strict and mechanical: it’s about what’s actually included in this specific call, not about what could plausibly be relevant to it.

Does Everything Competing for a Spot in Working Memory Get Treated Equally?

Not remotely evenly, even once something has made it into working memory. Where a piece of information sits within that space, and how much else is competing for attention alongside it, both affect how reliably the model actually uses it, a pattern already covered in detail elsewhere. What’s worth stating plainly here is that working memory itself has no built-in mechanism for prioritizing any of this. It doesn’t rank, filter, or weigh what it’s handed; it simply holds whatever arrives, in whatever order it arrives in.

This means deciding what actually deserves a spot in working memory for a given step is entirely someone else’s job, handled upstream by whatever assembles the request in the first place, not something working memory sorts out for itself once the material shows up. Working memory is a passive space, not an active curator, and treating it as though it does its own prioritization is exactly how irrelevant material ends up crowding out what actually mattered.

How Does Working Memory Relate to Every Other Kind of Memory Covered From Here On?

Given all of this, working memory’s actual role becomes clear: it’s the single funnel every other kind of memory has to pass through to have any effect at all. Whatever gets covered in the chapters ahead, memory of specific past events, memory of general facts, memory of learned procedures, none of it can shape a response unless it first gets pulled into this one bounded space for this one particular step. Long-term memory can hold an enormous amount, permanently and reliably, and still contribute nothing to a given response if nothing routes it into working memory at the right moment.

Picture an insurance claims processing assistant handling a new claim. The specifics of this particular claim obviously belong in working memory, but so does anything retrieved about this claimant’s history, the moment it’s pulled in:

from engram import EngramClient

client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])

history = client.memories.search(
    query="Has this claimant filed similar claims before, and how were they resolved?",
    user_id="claimant-70142",
    retrieval_config=HybridRetrieval(limit=5),
)

The retrieved results don’t remain long-term memory once they come back from this call. Assembled together with the current claim’s details into a single request, they become, for the duration of that one step, exactly the same kind of working memory as everything else the model is looking at:

context = "\n".join(m.content for m in history)
prompt = f"Current claim: {claim_details}\n\nClaimant history:\n{context}"

Nothing about the claimant’s history changed by being retrieved. What changed is that it now occupies the one space that actually matters for producing this specific response, alongside everything else competing for the model’s limited attention in that same request. That’s the whole relationship in practice: long-term memory decides what’s worth keeping, working memory decides nothing on its own, and retrieval is the bridge that moves something from one category into the other, exactly when it’s needed.

Working memory only ever holds what’s already been decided as relevant for this one step. It says nothing about how an agent captures and stores something worth remembering for later, based on a specific event that happened once and won’t happen again in quite the same way. Our next chapter, What is episodic memory?, picks up exactly that question.