CrewAI Memory & Knowledge

Memory and knowledge are what transform an agent from a one‑shot prompt‑responder into a context‑aware, continuously learning system. CrewAI provides built‑in mechanisms for agents to retain past interactions (memory) and to ground their reasoning in external information sources (knowledge). This handbook article is your practical, implementation‑focused guide to configuring memory backends, designing knowledge bases, choosing retrieval strategies, and building agents that remember.

What Is Memory in CrewAI

In CrewAI, memory is the ability of an agent to store and retrieve information across multiple tasks, and even across separate executions of the same crew. It’s not just a chat history—it’s a persistent, searchable context layer that the agent can query to improve its decisions.

CrewAI’s memory system operates at two main levels:

• Short‑term memory – the context of the current task, including outputs from preceding tasks and the agent’s own reasoning steps. This is automatically managed by the framework via the task context parameter.
• Long‑term memory – information that survives the current run and persists across future sessions. When enabled, the agent stores key facts, user preferences, and earlier decisions in a vector store. It retrieves relevant chunks before processing a new task.

Beyond memory, CrewAI also supports knowledge—structured external information sources like documents, databases, and APIs—that agents can consult for factual grounding.

A simple example: a customer support agent that remembers a user’s previous orders (long‑term memory) and looks up the return policy from a knowledge base (knowledge).

agent = Agent(
    role="Support Agent",
    goal="Resolve customer issues",
    memory=True,
    knowledge_sources=["./docs/faq.pdf", "./docs/returns.md"],
    verbose=True
)

Why Memory Matters

• Continuity across tasks – An agent that researched a topic in step one can recall the key findings in step three without re‑computing.
• Reducing redundant computation – Long‑term memory caches the results of expensive tool calls or LLM reasoning.
• Improving output quality – Agents that remember user preferences or past interactions give more personalised, consistent responses.
• Enabling personalization – A personal assistant agent that remembers your style, preferred tone, or frequent tasks.
• Supporting long‑running workflows – Memory allows workflows that span hours or days (and survive restarts) to retain context.

Without memory, every interaction starts from scratch. With memory, your crew becomes a team with institutional knowledge.

Types of Memory in CrewAI

Type	Scope	Lifetime	Storage
Short‑Term Memory	Current task and its immediate predecessors	Duration of the crew run	In‑memory, via context
Long‑Term Memory	Key facts, decisions, user preferences	Across multiple runs	Configurable vector store (Chroma, Pinecone, etc.)
Shared Crew Memory	Information shared among all agents in the crew	Per‑crew run or persistent	Same as long‑term, but accessible by all agents

Short‑Term Memory

Short‑term memory is the current execution context. When you define a task with context=[task_a, task_b], the outputs of those tasks are automatically injected into the agent’s prompt. Additionally, within a single task, the agent’s internal reasoning steps (the “agent loop”) are held in memory until the task completes. This requires no configuration; it’s the default behaviour.

Long‑Term Memory

When you set memory=True on an agent, CrewAI instantiates a memory store (by default an in‑memory vector database, which you should replace with a persistent backend for production). After each task, the agent extracts important pieces of information—facts, decisions, user data—and stores them as embedding vectors. Before the next task, the agent queries this store with the current task description and injects the most relevant memories into its prompt.

Configuring long‑term memory with a persistent vector store:

from crewai import Agent

agent = Agent(
    role="Financial Advisor",
    goal="Provide personalised investment advice",
    memory=True,
    memory_config={
        "provider": "chroma",          # or "pinecone", "weaviate", etc.
        "config": {
            "collection_name": "advisor_memory",
            "persist_directory": "./memory_db"
        }
    }
)

CrewAI supports multiple backends via the memory_config dictionary. The exact keys depend on the provider, but generally you specify provider and any connection details.

Shared Crew Memory

Shared memory extends the long‑term memory concept to the entire crew. When multiple agents have access to the same memory store, they can build on each other’s discoveries. To enable shared memory, you can configure all agents with the same store or use the crew’s memory parameter (if available in your version). In practice, a common pattern is to set memory=True on a coordinator agent and have other agents query its memory via a tool.

Knowledge Systems in CrewAI

While memory stores what happened, knowledge provides what is true. Knowledge systems supply agents with external, structured information—product manuals, internal wikis, real‑time APIs, or entire databases.

Knowledge Sources

CrewAI can ingest knowledge from:

• Documents – PDFs, Markdown files, DOCX, text files.
• Databases – SQL databases queried via tools.
• APIs – Real‑time data fetched on demand.
• Vector stores – Pre‑embedded knowledge bases (e.g., a Pinecone index with company documents).
• Web content – Scraped pages or RSS feeds.

Knowledge sources can be loaded at the agent level or the crew level. When loaded at the agent level, the agent can directly search them; at the crew level, they become available to all agents.

Example: loading knowledge from local files and a vector database:

from crewai import Agent, Task, Crew
from crewai.knowledge.source import StringKnowledgeSource, PDFKnowledgeSource

# Document-based knowledge
pdf_source = PDFKnowledgeSource(file_paths=["manual.pdf", "policy.pdf"])

# A pre-populated vector database
vector_source = StringKnowledgeSource(
    content=["Customer FAQ entry: ...", "..."]
)

agent = Agent(
    role="Helpdesk Agent",
    goal="Answer user questions accurately",
    knowledge_sources=[pdf_source, vector_source]
)

Knowledge Retrieval

When an agent has knowledge sources, CrewAI automatically performs retrieval before the agent begins a task. The retrieval pipeline typically uses:

• Embedding‑based search – The agent’s task description and any user query are embedded; the most similar knowledge chunks are fetched.
• Keyword retrieval – Fast but less semantic; good for exact matches.
• Hybrid retrieval – A combination that boosts accuracy.

The retrieval results are inserted into the agent’s system prompt or as a separate context block. The developer can control the number of retrieved chunks (k) and the retrieval method via the knowledge source configuration.

How Memory Works in Execution Flow

When a task runs with memory enabled, the following steps occur under the hood:

Input (Task) → Agent → Memory Retrieval (query relevant past facts) → Knowledge Retrieval (search external sources) → Task Execution → Output → Memory Update (store new facts)

This integration ensures that every decision is informed both by what the agent has learned and by authoritative external data.

Memory Lifecycle

A single memory unit goes through a clear lifecycle:

1. Context initialization – The agent receives the task, previous task outputs, and short‑term context.
2. Memory retrieval – The agent queries long‑term memory for related facts.
3. Task execution – The agent reasons, uses tools, and produces an output.
4. Output generation – The final result is produced.
5. Memory update – CrewAI automatically extracts structured facts from the interaction (e.g., “user_preference=dark_mode”) and upserts them into the vector store.
6. Persistence – If a persistent backend is configured, the facts are written to disk or a cloud database.

The extraction and update are handled by CrewAI’s internal memory processor; you can customise it by providing a memory_config with a custom embedding model or by overriding the fact‑extraction prompt.

Memory vs Knowledge

It’s crucial to distinguish these two concepts. They serve different purposes but are often used together.

Concept	Memory	Knowledge
Purpose	Store past interactions & decisions	Provide external, authoritative facts
Source	Agent’s own experience	Documents, databases, APIs
Lifetime	Accumulates over time	Static or slowly changing
Query	“What did this user prefer last time?”	“What is the return policy?”
Update	Automatic (after each task)	Manual or scheduled ingestion
Storage	Vector store (embedding‑based)	Vector store, file system, database

Think of memory as the agent’s personal diary; knowledge as its reference library.

Memory Update Strategies

You can control how new information is integrated into memory to prevent bloat and irrelevance.

Strategy	Description	When to use
Append	Always add new facts, never remove old ones.	Simple, but memory grows unbounded.
Replace	Overwrite existing facts if they conflict (e.g., user email changed).	Maintaining a single source of truth.
Summarization	Periodically summarize long memory into concise “core facts”.	Reducing token usage in large memory.
Selective Storage	Only store facts that meet a confidence threshold or relevance filter.	High‑noise environments.

CrewAI’s default behavior is append with a built‑in deduplication mechanism (facts that are too similar aren’t added). You can tune the memory processor to adopt summarization or selective storage.

Knowledge Retrieval Strategies

How you fetch knowledge determines both accuracy and latency.

Strategy	Mechanism	Best for
Full Retrieval	Load the entire document.	Very small knowledge bases (a few paragraphs).
Top‑K Retrieval	Embed the query and return the K most similar chunks.	Most common; balances recall and cost.
Semantic Search	Similar to Top‑K but with re‑ranking.	High‑quality, nuanced answers.
Hybrid Retrieval	Combine keyword (BM25) and embedding search.	When exact terms matter (e.g., product codes).

CrewAI’s KnowledgeSource objects allow you to specify the chunk_size, chunk_overlap, and retrieval method. For example, using PDFKnowledgeSource:

source = PDFKnowledgeSource(
    file_paths=["manual.pdf"],
    chunk_size=500,
    chunk_overlap=50,
    retrieval_method="hybrid"
)

Memory in Multi‑Agent CrewAI Systems

When multiple agents share a memory store, coordination becomes important.

• Shared memory coordination – All agents read from and write to the same vector store. This means an agent can act on information discovered by another agent, even across different tasks.
• Conflict resolution – If two agents store contradictory facts (e.g., “user wants PDF” vs “user wants CSV”), the last write wins by default. For consistency, use a dedicated coordinator agent that validates facts before storage.
• Context synchronization – In sequential crews, short‑term context flows naturally via task outputs. Long‑term shared memory gives the crew a collective memory that outlasts the current run.

Best practice: create a dedicated memory store per crew or per user session. Prefix memory keys with the user ID to isolate different users.

Memory in Flows and Tasks

CrewAI Flows offer fine‑grained control over memory injection.

• Task‑level memory injection – Each task can have its own memory configuration. A sensitive task may choose to disable memory to avoid storing confidential data.
• Flow‑level context propagation – The Flow’s state object acts as a super‑short‑term memory; it persists across all tasks in the flow. You can manually write flow state into agent memory at the end of the flow.
• Memory updates after execution – By default, memory updates happen after the agent finishes its task. In a Flow, you might want to update memory only after a series of steps succeed; you can do this by calling a custom “memory commit” task at the end.

Memory Persistence Mechanisms

Storage Type	Use case	Configuration Example
In‑Memory	Development, testing	Default (no config needed)
File‑Based (Chroma)	Single‑machine persistence	provider: "chroma", persist_directory: "./memory"
Database (Pinecone, Weaviate)	Production, cloud scaling	provider: "pinecone", api_key: "..."
Custom Vector Store	Existing infrastructure	Wrap your store in a CrewAI‑compatible adapter

For production, use a cloud vector database. This ensures that memory survives server restarts and can be shared across multiple replicas.

Common Beginner Mistakes

1. Storing too much irrelevant data – Memory bloat increases latency and dilutes relevance. Be selective.
2. No memory cleanup strategy – Old facts become stale. Implement periodic summarization or expiration.
3. Over‑reliance on memory instead of tools – Memory is for experiential knowledge; real‑time data should come from tools or APIs.
4. Poor knowledge structuring – Dumping raw, un‑chunked documents leads to poor retrieval. Invest time in chunking and metadata tagging.
5. Missing retrieval optimization – Not tuning k or not using re‑ranking can result in the agent seeing irrelevant context.
6. Forgetting to use persistent storage – Losing all user memories on server restart is catastrophic.

Best Practices

• Keep memory structured – Use the built‑in fact extraction; it stores data as key‑value pairs, which is ideal for personalization.
• Store only relevant information – Review the memory store periodically and prune.
• Use summarization for long contexts – Instruct the memory processor to summarise before storage.
• Separate memory from knowledge – Memory for user‑specific data; knowledge for company‑wide facts.
• Optimize retrieval strategies – Choose k (number of chunks) based on token limits; use hybrid search for code/identifiers.
• Monitor memory growth – Track the number of facts and retrieval latency.
• Test with real data – Simulate multi‑turn conversations to ensure memory retrieval is accurate.

Practical Example: Customer Support Agent with Memory & Knowledge

Let’s build an agent that remembers returning customers and uses a knowledge base for product questions.

1. Set up knowledge sources

from crewai.knowledge.source import PDFKnowledgeSource

policy_knowledge = PDFKnowledgeSource(
    file_paths=["returns_policy.pdf"],
    chunk_size=300,
    chunk_overlap=50
)

2. Create the agent with memory and knowledge

from crewai import Agent

support_agent = Agent(
    role="Customer Support Specialist",
    goal="Resolve customer inquiries quickly and accurately",
    backstory="You are an expert in our products and policies. You remember customers' preferences.",
    memory=True,
    memory_config={
        "provider": "chroma",
        "config": {"persist_directory": "./support_memory"}
    },
    knowledge_sources=[policy_knowledge],
    verbose=True
)

3. Define tasks

We simulate a returning user session.

from crewai import Task, Crew, Process

task1 = Task(
    description="Customer (user_id: 123) asks: 'What is the return window for electronics?'",
    expected_output="A clear answer citing the policy.",
    agent=support_agent
)

task2 = Task(
    description="Same customer asks: 'I want to return my headphones. I bought them 20 days ago.' "
                "Use memory to recall their previous interaction and the policy to answer.",
    expected_output="Personalised answer that references the earlier question.",
    agent=support_agent
)

crew = Crew(
    agents=[support_agent],
    tasks=[task1, task2],
    process=Process.sequential
)

4. Execution

result = crew.kickoff()

What happens internally:

• Task 1 – Agent queries memory (empty initially), searches the knowledge base for “return window electronics”, finds the relevant chunk, and answers. After the task, it stores a fact: user_123_last_query: "return window electronics".
• Task 2 – Agent retrieves memory: sees the user’s previous query, knows they are a returning customer. Searches knowledge base for “return headphones”. Because the memory has context, it can say “Last time you asked about the return window for electronics—it’s 30 days, so your headphones (20 days ago) are within the window.” It updates memory with the new interaction.

This demonstrates a memory‑augmented, knowledge‑grounded agent that becomes smarter with each interaction.

Memory in Real‑World Systems

• Customer Support Systems – Remember customer tier, past tickets, and preferences to personalise responses.
• Research Assistants – Store previously gathered sources, hypotheses, and decisions to avoid repetitive work.
• Personal AI Assistants – Recall user’s schedule, contacts, and writing style across days.
• Enterprise Knowledge Systems – Let agents search across thousands of internal documents, learning which documents are most useful.
• Document Processing Pipelines – Retain extraction patterns and validation rules to improve accuracy over time.

In all these cases, memory and knowledge transform a reactive tool into a proactive, learning system.

Memory & Other CrewAI Concepts

Memory and knowledge are deeply integrated with the rest of CrewAI.

• Core Concepts – Memory is set on agents, tasks, or crews; knowledge is loaded as agent resources.
• Tools & Delegation – Custom tools can be used to write to or read from memory stores, or to query external knowledge APIs.
• Flows – Flows can use their persistent state as short‑term memory and call agents with memory enabled.
• Production – Persistent memory backends, monitoring memory size, and knowledge refresh pipelines are production concerns.

For cross‑framework memory patterns, see the LangGraph comparison or the framework comparison guide. To explore standardised tool ecosystems, read the MCP Guide.

FAQ

1. What is memory in CrewAI?

It’s the ability for agents to store and retrieve information across tasks and sessions, using a vector store to remember past interactions and decisions.

2. How does CrewAI store memory?

When memory=True is set, CrewAI uses a configurable vector database (Chroma, Pinecone, etc.) to store facts as embeddings. You can configure the backend via memory_config.

3. What is the difference between short‑term and long‑term memory?

Short‑term memory is the current task context and previous task outputs (automatic). Long‑term memory persists across runs and stores facts that the agent can recall later.

4. What is shared memory?

A memory store accessible by multiple agents in the same crew, allowing them to build a collective knowledge base.

5. How does knowledge differ from memory?

Knowledge is external, structured information (documents, APIs). Memory is the agent’s own experience. Knowledge answers “what is the policy?”; memory answers “what did this user do last time?”

6. Can CrewAI use vector databases?

Yes. CrewAI supports Chroma (default), Pinecone, Weaviate, and custom vector stores.

7. How is memory updated?

After a task completes, the memory processor automatically extracts facts (using an LLM) and upserts them into the vector store.

8. Can I control what gets stored in memory?

You can customise the memory extraction prompt or implement a filter. The default behaviour stores key‑value facts like user preferences and decisions.

9. Is knowledge retrieval automatic?

Yes. When knowledge sources are attached to an agent, the framework automatically retrieves relevant chunks before each task.

10. How do I add a knowledge base from a website?

You can scrape the website into a text file or use a tool that retrieves live content. Alternatively, use a StringKnowledgeSource with the scraped content.

11. Can memory be disabled for sensitive tasks?

Yes, you can set memory=False on a specific agent or use a separate agent instance for sensitive work. Also, you can clear memory after a task.

12. How do I avoid memory bloat?

Use summarization strategies, set a maximum number of stored facts, and periodically prune old or irrelevant entries.

13. Does CrewAI support multi‑user memory isolation?

Not out‑of‑the‑box, but you can achieve isolation by using a separate memory store per user or by prefixing facts with a user ID and filtering queries.

14. How is memory handled in hierarchical crews?

The manager agent can have memory enabled to recall which workers performed best for certain tasks. Worker agents can have their own memory for domain‑specific facts.

15. What happens if the vector database goes down?

Memory retrieval will fail gracefully; the agent will proceed without prior context. In production, use a highly available vector database.

16. Can I use both memory and tools in the same agent?

Absolutely. Tools provide real‑time capabilities; memory provides historical context. They complement each other perfectly.

Conclusion

CrewAI’s memory and knowledge systems turn a collection of agents into a continuously improving, context‑aware workforce. With a few lines of configuration, agents can remember user preferences, learn from past successes, and ground their answers in authoritative documents.

Key takeaways:

• Memory (short‑term and long‑term) provides continuity and personalisation.
• Knowledge sources give agents access to the information they need to be accurate.
• Both are implemented through vector stores and automatic retrieval, with extensive customisation options.
• Proper memory management—cleanup, persistence, and retrieval tuning—is essential for production.

Deepen your expertise with these handbook articles:

• CrewAI Core Concepts – the foundation of agents and tasks.
• CrewAI Tools & Delegation – equip agents with tools to complement memory.
• CrewAI Flows – orchestrate complex workflows with memory‑aware steps.
• Taking CrewAI to Production – deploy persistent memory and knowledge pipelines.

For a broader view, see the CrewAI framework overview, the LangGraph comparison, or the framework comparison guide. To expand your tool ecosystem, visit the MCP Guide.

Now, give your agents the gift of memory—and watch them grow smarter with every interaction.