The 'memory' of an AI agent determines its coherence in multiple rounds of tasks. The current mainstream implementation can be divided into three levels:
1. In context window memory: directly concatenate the conversation history and intermediate results into prompt words. The advantage is simple implementation, but the disadvantage is that the cost increases with length and early information is truncated beyond the window.
2. Retrieval based memory: Vectorize historical conversations and knowledge base content and store them in a vector database, recalling them based on relevance when needed. RAG (Retrieval Enhanced Generations) is a typical representative of such schemes, and frameworks such as LangChain have provided mature memory and retrieval components.
3. Structured long-term memory: Drawing on the hierarchical thinking of operating systems, memory is divided into "working memory" and "archived memory", with agents autonomously deciding when to archive and retrieve it. The MemGPT paper (No. 2310.08560) published in arXiv in October 2023 systematically elaborated on this idea and evolved into the open-source Letta framework.
A common choice in engineering practice is to use a contextual window with a small amount of summary for short tasks; Long cycle tasks (such as customer service, research assistants) combined with vector retrieval; For scenarios that require high memory consistency, introduce structured memory layers and regular "memory organization" tasks to avoid infinite memory expansion.
It should be noted that the memory mechanism is only a part of the Agent's ability, and the actual effect depends on the coordination between tool invocation and task planning. It is recommended to first clarify how much memory is needed before selecting a plan when the enterprise is launched, in order to avoid excessive design.
[Reference source]
- MemGPT: Towards LLMs as Operating Systems（arXiv:2310.08560）
-LangChain Memory Concept Document: https://python.langchain.com/docs/concepts/memory/
-Comprehensive compilation of industry information that has been publicly released