Part 6 — The Memory Lifecycle: Pipelines from Raw Data to Stored Memory

This section walks the pipeline from raw conversation to durable, queryable memory: extract, transform, commit — plus the operational concerns that keep writes correct under load.

What you will learn

  • The extract → transform → commit pipeline and why each stage exists
  • Deduplication, conflict reconciliation, and merging during transforms
  • Async processing, durable execution, ordering, and idempotency
  • How pipeline output becomes a queryable memory store

Chapters in this section

There are 20 chapters in this part. Open any chapter to read it on its own, or work through them in order.

  1. 1 The Memory Pipeline: Extract, Transform, Commit
  2. 2 Extraction: Pulling Facts from Raw Conversation
  3. 3 Input Shapes: Conversations, Strings, and Pre-Extracted Facts
  4. 4 LLM-Powered Extraction vs Rule-Based Extraction
  5. 5 Transformation: Integrating New Information with Existing Memory
  6. 6 Deduplication During the Transform Stage
  7. 7 Reconciling Conflicting Memories
  8. 8 Merging Related Memories Into Higher-Order Knowledge
  9. 9 Commit Semantics: Why Atomic Writes Matter
  10. 10 Asynchronous Memory Processing and Why It Matters for Latency
  11. 11 Durable Execution for Memory Pipelines
  12. 12 Ordered Processing: Why Sequence Matters in Memory Updates
  13. 13 Buffering and Batching Raw Memory Input
  14. 14 Pipeline Steps as a Composable Graph
  15. 15 Idempotency in Memory Writes
  16. 16 Run Tracking and Observability of Pipeline Execution
  17. 17 Designing Pipelines for Different Use Cases
  18. 18 Continual Learning Pipelines: Memory That Improves Agent Behavior
  19. 19 LLM-as-Judge Patterns in Memory Pipelines
  20. 20 From Pipeline Output to Queryable Memory Store