How should you design memory architecture for autonomous research agents?

Autonomous research agents fail in a distinctive way. They rediscover the same dead end every morning. They forget why a source was discarded. They mix one investigation’s hypotheses into another’s outline. Long context windows do not fix that. Dense tool traces bury the causal link between an early assumption and a late decision. Weaviate Engram is built for this class of long-horizon work: extract durable facts from noisy agent streams, reconcile contradictions, and keep investigations scoped while shared research craft can still improve. This chapter explains why transcript dumps are a weak research memory, how Engram groups separate investigation state from methodological skill, how multi-agent buffers assemble lessons that no single context window ever held, how to search and write across planner and tool-worker turns, and how corpora stay separate from memory.

The aim is continuity without amnesia and without cross-thread contamination. Remember the rejected method. Keep the open questions honest.

Why do long-horizon research agents need more than a bigger prompt?

Research loops span hours or days. They include planner messages, tool outputs, paper snippets, failed queries, and human course corrections. Paying to replay all of that on every step is expensive. Attention still drifts. Similarity search over raw logs often misses causal links that do not look alike in embedding space. “We avoided radiocarbon for these varves because of reservoir effects” may not rank next to a later question about dating strategy if the wording drifted.

Engram’s product case for agent memory starts from that limit-in-the-loop problem. Agents without durable memory re-solve the same intermediate puzzles and lose the insight. The remedy is not one ever-growing pile. It is actively maintained memories with topics that magnetize what matters: active hypotheses, discarded approaches, source judgments, and open questions. Pipelines run asynchronously so the agent can keep working while extraction and reconciliation finish.

The next design question is structural. Which memories belong to one inquiry, and which memories should teach the agent how to research better next time?

How should Engram groups split investigation memory from research craft?

Use a personalization-style or investigation group for thread-local state. Topics might capture hypotheses, rejected methods, trusted sources, and unresolved questions. Scope hard with a researcher or tenant user_id when multiple clients share a deployment. Add property scopes such as investigation_id so two concurrent studies cannot bleed. On search, include the active investigation. Omit it only when a deliberate cross-study review is authorized.

Use a continual-learning group for methodological playbooks. Examples include “prefer stratigraphic cross-checks before single-proxy dating claims” or “when a subagent overuses near-text search, switch to structured filters.” Those topics can be project-wide so craft compounds across trusted teams. Engram’s own multi-agent RAG story shows why this split matters. Task goals, actions taken, and later feedback may live in different agent windows. A buffered pipeline can hold intermediate extracts until a transform combines them into one experience memory, then commit only the useful lesson. Intermediate scraps should not pollute retrieval.

Keep literature corpora in a normal knowledge index or warehouse. Engram is not a paper database. Merge document hits with Engram investigation memory at answer time. That is the same personalized RAG pattern applied to science: shared evidence in one store, per-investigation judgment in another.

How do multi-agent research traces become durable Engram memories?

Planner agents and tool workers rarely share one context. Engram accepts that. Add conversation turns from each agent as they happen. Configure topics that separately catch goals, actions, and feedback. Let a buffer gather those pieces. Transform steps combine them into denser experience only when the batch is complete. Commit steps persist the result. Because pipelines are durable and ordered by scope, rapid fire-and-forget adds still process in a coherent sequence for that investigation.

Pre-extracted inputs help when a critic agent already decided the memory text, such as “Hypothesis H2 falsified: varve couplets are turbidite repeats.” String inputs fit non-chat events like “Downloaded core log set B.” Conversation inputs fit human steering. Hybrid search remains the default for open questions about what was tried. Fetch retrieval fits bounded objects such as a single running investigation brief when you configured a bounded summary topic.

Do not block the research loop on runs.wait unless you are debugging. The latest tool output is already in working memory. Cross-session value arrives on the next cold start or the next subagent spawn.

What does an investigation turn look like in code?

Before the planner chooses the next experiment, search investigation memory for hypotheses and dead ends. Search continual-learning playbooks for method guidance. After a meaningful step, add the exchange or event under the investigation scope.

Here is a glacial lake chronology desk. The inquiry id isolates this study. Method lessons stay project-wide and de-identified.

import os
from engram import EngramClient
from engram.types import HybridRetrieval

client = EngramClient(api_key=os.environ["ENGRAM_API_KEY"])

researcher = "lab.north.archive"
inquiry = "glacier-varve-inquiry-6"

case_hits = client.memories.search(
    query="Rejected dating methods, open questions about turbidite repeats",
    user_id=researcher,
    group="personalization",
    retrieval_config=HybridRetrieval(limit=6),
    properties={"investigation_id": inquiry},
)

methods = client.memories.search(
    query="When to prefer stratigraphic cross-checks over single-proxy dating",
    group="continual_learning",
    retrieval_config=HybridRetrieval(limit=4),
)

turn = [
    {
        "role": "user",
        "content": (
            "For glacier-varve-inquiry-6, mark radiocarbon as low priority. "
            "Reservoir effects look severe. Test whether couplets are turbidite "
            "repeats before any Bayesian age model."
        ),
    },
    {
        "role": "assistant",
        "content": (
            "I will park radiocarbon for now, prioritize turbidite-repeat checks, "
            "and keep the age model blocked until that falsification test finishes."
        ),
    },
]

run = client.memories.add(
    turn,
    user_id=researcher,
    group="personalization",
    properties={"investigation_id": inquiry},
)
print(run.run_id, run.status)
print([m.content for m in case_hits])
print([m.content for m in methods])

The personalization search returns prior dead ends so the planner does not reopen radiocarbon casually. The continual-learning search returns craft without carrying another lab’s private notes. The add call captures the new priority shift under the inquiry property. Reconciliation can later rewrite related hypotheses when the turbidite test lands, instead of leaving contradictory claims fighting in retrieval.

When a search subagent and a critic agent contribute separately, add each stream with the same investigation_id. Let the continual-learning pipeline buffer and fuse only the de-identified method lesson. Keep named core IDs and embargoed site details in the investigation group.

Which safeguards keep research memory honest?

Test isolation between inquiries. Store a distinctive rejected method under inquiry A. Search as inquiry B. Assert it does not appear. Confirm playbooks still return. Cap retrieved memories so planning prompts stay short. Prefer topic filters when the turn is only about open questions or only about sources.

Treat provenance as a product concern beside Engram. Store citations and document ids in your corpus layer. Let Engram remember judgments about those sources. Do not let a fluent summary erase uncertainty. Topic descriptions should ask for discarded approaches and confidence notes, not for invented certainty. Human steering remains a first-class input. When a researcher corrects the agent, that correction should reconcile prior memories rather than sit as a ignored chat line.

Autonomous research memory with Engram is therefore a disciplined long-horizon filing system. Scope investigations hard. Keep literature in a knowledge index. Use buffers and transforms to learn from multi-agent traces without retrieving half-baked scraps. Search before the next experiment, write asynchronously after, and prove cross-inquiry isolation in CI. Our next chapter, How should you design memory architecture for voice assistants?, moves from long written investigations to spoken assistants, where latency, turn brevity, and ambient context change how the same Engram layers should be searched and updated.