added memory documentation #102

pritansh-sahsani · 2024-01-06T12:05:13Z

Summary by CodeRabbit

Documentation
- Introduced documentation for memory management in conversational AI systems, detailing various memory classes and their integration with core components.

coderabbitai · 2024-01-06T12:05:19Z

Walkthrough

The update introduces a comprehensive framework for memory management in AI conversational systems. It outlines various memory classes, each tailored to handle different aspects of memory storage and retrieval, ensuring efficient and scalable conversation handling. The document also discusses how these memory components interact with other system elements and hints at possible enhancements.

Changes

File Path	Change Summary
`docs/ai/03_memory.md`	Introduced classes for AI memory management, detailed their functionalities, and discussed integration with system components.

🐇✨
To store and recall, we've spun a new tale,
Of memories kept in a digital veil.
In classes they dwell, in bytes they parade,
Our AI's remembrance, in silicon made. 🌟
🐇✨

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit's AI:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>.
- Generate unit-tests for this file.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit tests for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai generate interesting stats about this repository from git and render them as a table.
- @coderabbitai show all the console.log statements in this repository.
- @coderabbitai read src/utils.ts and generate unit tests.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
The JSON schema for the configuration file is available here.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

coderabbitai

Review Status

Actionable comments generated: 9

Configuration used: CodeRabbit UI

Commits

Files that changed from the base of the PR and between edf3965 and 8852e9f.

Files selected for processing (1)

docs/ai/03_memory.md (1 hunks)

Additional comments: 1

docs/ai/03_memory.md (1)

2-4: The function extract_text is described as a standalone function, but it's not clear from the documentation where this function is defined or how it's related to the memory classes. Clarify the context in which extract_text is used within the memory management system.

coderabbitai · 2024-01-06T12:06:10Z

docs/ai/03_memory.md

+class SimpleMemory(BaseMemory, BaseModel): - Defines the SimpleMemory 
+
+class that inherits from BaseMemory.


The SimpleMemory class is introduced without a clear description of its purpose or how it differs from the other memory classes. Provide a brief description of the class's role in memory management.

coderabbitai · 2024-01-06T12:06:10Z

docs/ai/03_memory.md

+def add\_memory(self, prompt: str, llm\_response: Any) -> None: - Adds a conversation memory to the store.
+
+Parameters:
+
+prompt (str): The user's prompt or input.
+
+llm\_response (Any): The AI's response to the prompt.
+
+Functionality:
+
+This method checks if the given (prompt, llm\_response) pair already exists in memory.
+
+If not, it appends a dictionary containing the prompt and response to the list of self.messages.


The add_memory method in SimpleMemory is well-documented, but it's important to specify what type of store is being used. Is it an in-memory list, a database, or a file system? This information is crucial for understanding the method's implementation and potential limitations.

coderabbitai · 2024-01-06T12:06:10Z

docs/ai/03_memory.md

+def get\_memory(self, \*\*kwargs) -> str: - Retrieves all memories from the store.
+
+Parameters:
+
+kwargs (dict): Additional keyword arguments (not used in this method).
+
+Functionality:
+
+It constructs a formatted string containing all conversations stored in self.messages.
+
+For each conversation, it appends the user's prompt and AI's response to the formatted string.
+
+Returns the formatted string containing all memories.


The get_memory method's description does not specify the format of the returned string. For better clarity, include an example of the formatted string or describe the formatting in more detail.

coderabbitai · 2024-01-06T12:06:10Z

docs/ai/03_memory.md

+def remove\_memory(self, prompt: str) -> None: - Removes a specific memory from the store.
+
+Parameters:
+
+prompt (str): The user's prompt to be removed.
+
+Functionality:
+
+Iterates through each conversation in self.messages. If a conversation's prompt matches the provided prompt, it is removed from the list.


The remove_memory method description should clarify what happens if the prompt is not found in the memory. Does it fail silently, or is an error raised? This information is important for error handling and expected behavior.

coderabbitai · 2024-01-06T12:06:10Z

docs/ai/03_memory.md

+class SummaryMemory(BaseMemory, BaseModel): - Defines the SummaryMemory class that inherits from BaseMemory.
+
+Attributes:
+
+current\_summary (str): Keeps track of the current conversation summary.
+
+messages\_in\_summary (List[Dict[BaseMessage, Any]]): Stores a list of messages that have been included in the summary.
+
+Methods
+
+Method add\_memory:
+
+This method adds a new conversation memory to the store, avoiding duplicates.
+
+Method get\_memory:
+
+It creates an instance of the llms.OpenAI class using the compiler module. It filters out messages that are already included in the current summary. For each new message, it constructs a string with the user prompt and AI response and updates messages\_in\_summary.
+
+It creates a summarizer using the compiler module with the provided template, llm instance, and new lines. The summary is updated with the extracted text from the summarized memory.
+
+The final summarized memory is returned.
+
+Method remove\_memory:
+
+This method removes a specified memory (user prompt) from the store. It removes the memory from messages, updates current\_summary based on the remaining messages, and clears messages\_in\_summary.
+
+Method clear:
+
+Clears all stored memories, the list of messages in the summary, and resets the current summary.
+
+The SummaryMemory class builds upon the concept of storing and managing conversation memories. It adds the capability to create and update summaries of conversations using a summarization process. The class utilizes the compiler module to interact with OpenAI's language model and performs summarization based on a predefined template. It keeps track of which messages are included in the summary to prevent duplication. The SummaryMemory class seems to be designed for a more advanced conversational AI system that can generate summaries of interactions.


The SummaryMemory class description mentions the use of the compiler module and interaction with OpenAI's language model. Ensure that the documentation includes links to the relevant sections or external documentation for these components, as they are critical to understanding how SummaryMemory operates.

coderabbitai · 2024-01-06T12:06:11Z

docs/ai/03_memory.md

+class BufferSummaryMemory(BaseMemory, BaseModel): - Defines the BufferSummaryMemory class that inherits from BaseMemory.
+
+Attributes:
+
+current\_summary (str): Tracks the current conversation summary.
+
+current\_buffer (str): Stores recent messages in a buffer.
+
+messages\_in\_summary (List[Dict[BaseMessage, Any]]): Stores messages included in the summary.
+
+messages\_in\_buffer (List[Dict[BaseMessage, Any]]): Stores messages included in the buffer.
+
+Methods
+
+Method add\_memory:
+
+Adds a new conversation memory to the store while avoiding duplicates.
+
+Method get\_memory:
+
+It deals with both the summary and the buffer. Creates an instance of the llms.OpenAI class using the compiler module. Retrieves the memory\_threshold from kwargs or defaults to 1. Separates recent messages into the buffer and older ones into the summary based on the threshold. Constructs the buffer as a string. Generates a summary only for the new additions to the summary and updates the current\_summary. Returns the combined summary and buffer as the summarized memory.
+
+Method remove\_memory:
+
+Removes a specific memory (user prompt) from the store and updates related variables.
+
+Recalculates the buffer and summary based on the remaining messages and threshold, and updates the current\_buffer and current\_summary accordingly.
+
+Method clear:
+
+Clears all stored memories, the lists of messages in the summary and buffer, and resets the current summary and buffer.
+
+The BufferSummaryMemory class extends the functionality of SummaryMemory by introducing a buffer to store recent messages separately from the summary. This design allows the class to manage recent conversations in a buffer and maintain a more concise summary of older interactions. It utilizes a threshold to decide which messages are included in the buffer and which in the summary. The class's methods manage both the buffer and summary, offering a more sophisticated approach to memory management for conversational AI systems that require both a concise summary and recent context.


The BufferSummaryMemory class extends SummaryMemory, but it's not clear how the buffer mechanism is implemented or what the threshold for separating recent messages is. Provide more details on the buffer mechanism and how the threshold value is determined or can be configured.

coderabbitai · 2024-01-06T12:06:11Z

docs/ai/03_memory.md

+
+Method add\_memory:
+
+def add\_memory(self, program\_name: str, session\_id: str, prompt: str, llm\_response: str) -> None: - Adds a memory entry to the store for a specific session of an application.
+
+Parameters:
+
+program\_name (str): Name of the application.
+
+session\_id (str): Identifier for the specific session.
+
+prompt (str): User's input prompt.
+
+llm\_response (str): AI's response to the prompt.
+
+Functionality:
+
+Creates a directory for the specific program if it doesn't exist. Writes the given prompt and response as a JSON entry to the session file. 
+
+Method get\_memory:
+
+def get\_memory(self, program\_name: str, session\_id: str, max\_len: int = 2040 \* 1024, limit: int = 1000): - Retrieves memories from a specific session of an application.
+
+Parameters:
+
+program\_name (str): Name of the application.
+
+session\_id (str): Identifier for the specific session.
+
+max\_len (int): Maximum length of data to read (default is 2040 KB).
+
+limit (int): Maximum number of memories to retrieve (default is 1000).
+
+Functionality:
+
+Reads the last max\_len bytes of the session file. Extracts the last limit JSON entries as memories. Returns the retrieved memories.
+
+Method remove\_memory:
+
+def remove\_memory(self, program\_name: str, session\_id: str, prompt: str = ''): - Removes memories from a specific session of an application.
+
+Parameters:
+
+program\_name (str): Name of the application.
+
+session\_id (str): Identifier for the specific session.
+
+prompt (str): If specified, removes only memories with the given prompt.
+
+Functionality:
+
+If prompt is not specified, it removes the entire session file. If prompt is specified, it removes only the memory entries with the specified prompt.
+
+Method clear:
+
+def clear(self, program\_name: str = ''): - Clears all memories of a specific program or all programs.
+
+Parameters:
+
+program\_name (str): Name of the application to clear memories for (optional).
+
+Functionality:
+
+If program\_name is specified, it clears all memories for that program.
+
+If program\_name is not specified, it iterates through all programs and clears their memories.
+
+Helper functions
+
+def read(self, program\_name: str, session\_id: str, n: int, max\_len: int) -> List[Dict]: - Reads and returns specific memories from a session of an application.
+
+Parameters:
+
+program\_name (str): Name of the application.
+
+session\_id (str): Identifier for the specific session.
+
+n (int): Number of memories to retrieve.
+
+max\_len (int): Maximum length of data to read.
+
+Functionality:
+
+Reads the last max\_len bytes of the session file. Extracts the last n JSON entries as memories. Returns the extracted memories.
+
+def sessions(self, program\_name: str) -> List[Dict]: - Retrieves a list of sessions for a specific program.
+
+Parameters:
+
+program\_name (str): Name of the application.
+
+Functionality:
+
+Searches for session files in the program's directory and retrieves information about each session, including the last modified time and latest memory entry.
+
+Method \_memories\_dirs:
+
+A private method that creates the directory for storing memory data if it doesn't exist and returns the directory path.
+
+The InFileMemory class provides a memory management system that saves conversation data in individual session files on the local filesystem. It's designed to persistently store conversations for different applications and sessions. It offers methods to add memories, retrieve memories, remove memories, clear memories, read specific memories, and retrieve session information. This class could be useful for maintaining a history of interactions in a more permanent storage location.


The InFileMemory class documentation should include information about the file format used for storing sessions and how concurrency is handled. Are there any locking mechanisms or transactional guarantees when multiple instances try to write to the same session file?

coderabbitai · 2024-01-06T12:06:11Z

docs/ai/03_memory.md

+class ReadOnlyMemory(BaseMemory): - Defines the ReadOnlyMemory class that inherits from BaseMemory.
+
+Attributes:
+
+memory: BaseMemory - Holds the reference to the original memory instance.
+
+Constructor:
+
+def \_\_init\_\_(self, memory: BaseMemory): - Initializes the ReadOnlyMemory instance with a provided memory instance.
+
+Parameters:
+
+memory (BaseMemory): The memory instance that this read-only memory will wrap.
+
+Methods
+
+Method add\_memory:
+
+def add\_memory(self, prompt: str, llm\_response: Any) -> None: - Prevents adding memory in a read-only context.
+
+Functionality:
+
+This method is implemented with an empty body, preventing any addition to the underlying memory instance.
+
+Method get\_memory:
+
+def get\_memory(self, \*\*kwargs) -> Any: - Retrieves entire memory from the store.
+
+Parameters:
+
+kwargs (dict): Additional keyword arguments that may be passed to the underlying memory's get\_memory method.
+
+Functionality:
+
+Calls the get\_memory method of the underlying memory instance and returns its result.
+
+Method remove\_memory:
+
+def remove\_memory(self, prompt: str) -> None: - Prevents memory removal in a read-only context.
+
+Functionality:
+
+This method is implemented with an empty body, preventing any removal from the underlying memory instance.
+
+Method clear:
+
+def clear(self) -> None: - Prevents clearing the memory in a read-only context.
+
+Functionality:
+
+This method is implemented with an empty body, preventing any clearing of the underlying memory instance.
+
+Property memory\_keys:
+
+@property def memory\_keys(self) -> List[str]: - Returns a list of prompts for all memories in the underlying memory.
+
+Functionality:
+
+Iterates through each conversation in the underlying memory and retrieves their prompts, creating a list of prompts.
+
+The ReadOnlyMemory class is a memory wrapper that provides a read-only view of an existing memory instance. It prevents modifications to the underlying memory while allowing data retrieval operations like getting all memories and extracting memory keys (prompts). This class is useful in scenarios where you want to ensure that memory data remains unmodified during certain operations or contexts.


The ReadOnlyMemory class is described as a wrapper that prevents modifications. However, it would be beneficial to explain the use cases for such a class. When would one prefer to use ReadOnlyMemory over the other memory classes?

coderabbitai · 2024-01-06T12:06:11Z

docs/ai/03_memory.md

+Implementation with Compiler
+
+Compiler \_\_init\_\_.py
+
+Initializing a compiler takes an optional parameter ‘memory’, which should be a instance of one of the memory classes explained above. If memory is passed, the compiler passes it to the Program class. Additionally, if the user doesn’t prefer the conversation history to be displayed in the system prompt (in case the user doesn’t want a system prompt or wants to pass it with instructions, etc.), they can add a ConversationHistory variable to the prompt template.
+
+\_program.py
+
+1. Firstly, after all the variables are initialized, the class init function checks if memory has been passed. If yes, it further checks if the prompt template has a ConversationHistory variable, and if there isn’t, the ‘add\_variable’ function, declared in the \_program.py file itself, is called to add it to the prompt template at the end of the system prompt.
+
+1. Next, the value for ConversationHistory is added to the keyword arguments, (kwargs), by getting it from the Memory.get\_memory() function explained above. 
+
+1. The memory variable is then passed to all Program class instances created for further use.
+
+1. After the response has been completely generated, the program again checks if memory exists. If yes, then it calls the extract\_text() function, declared the \_program.py file itself to get the user prompt and llm response key-value pair, which is then passed to Memory.add\_memory() to save it into the memory. 
+
+Implementation with Agent
+
+The usage is same for agent as for the compiler. Simply create a memory instance and pass into the agent constructor. The agent \_\_init\_\_() function passes it into the compiler when the object is created, and the implementation continues as explained above.
+
+Future implementation
+
+Here are some suggestions on what could be improved on memory in the future.
+
+1. Manual saving of memory: Through a parameter, the user can disable the adding of prompt and llm responses by the Program instance, as explained in implementation with compiler, step 4. Instead, the user can call the memory.add\_memory() themselves if they wish to add only certain variables. This can be called either after the compiler/agent is run, or in the prompt 
+
+template as well.
+
+1. Fixing infile memory: Unfortunately, while the infile memory has been created and works, it has not been integrated into the compiler, but it only requires the parameters to be taken from the user and to be passed into the add\_memory function, similar to how the memory\_threshold variable has been passed for BufferSummaryMemory. 


The section on implementation with the Compiler and Agent is quite dense and could benefit from code snippets or pseudo-code to illustrate the described processes. Additionally, the future implementation suggestions should be clearly separated into their own subsection for better readability.

added memory documentation

8852e9f

coderabbitai bot reviewed Jan 6, 2024

View reviewed changes

anubrag force-pushed the main branch from 6cd38b2 to ad16fec Compare January 11, 2024 04:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

added memory documentation #102

added memory documentation #102

pritansh-sahsani commented Jan 6, 2024 •

edited by coderabbitai bot

coderabbitai bot commented Jan 6, 2024 •

edited

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (`.coderabbit.yaml`)

CodeRabbit Discord Community

coderabbitai bot left a comment

coderabbitai bot Jan 6, 2024

coderabbitai bot Jan 6, 2024

coderabbitai bot Jan 6, 2024

coderabbitai bot Jan 6, 2024

coderabbitai bot Jan 6, 2024

coderabbitai bot Jan 6, 2024

coderabbitai bot Jan 6, 2024

coderabbitai bot Jan 6, 2024

coderabbitai bot Jan 6, 2024

		class SimpleMemory(BaseMemory, BaseModel): - Defines the SimpleMemory

		class that inherits from BaseMemory.

added memory documentation #102

Are you sure you want to change the base?

added memory documentation #102

Conversation

pritansh-sahsani commented Jan 6, 2024 • edited by coderabbitai bot

Summary by CodeRabbit

coderabbitai bot commented Jan 6, 2024 • edited

Walkthrough

Changes

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (.coderabbit.yaml)

CodeRabbit Discord Community

coderabbitai bot left a comment

Choose a reason for hiding this comment

coderabbitai bot Jan 6, 2024

Choose a reason for hiding this comment

coderabbitai bot Jan 6, 2024

Choose a reason for hiding this comment

coderabbitai bot Jan 6, 2024

Choose a reason for hiding this comment

coderabbitai bot Jan 6, 2024

Choose a reason for hiding this comment

coderabbitai bot Jan 6, 2024

Choose a reason for hiding this comment

coderabbitai bot Jan 6, 2024

Choose a reason for hiding this comment

coderabbitai bot Jan 6, 2024

Choose a reason for hiding this comment

coderabbitai bot Jan 6, 2024

Choose a reason for hiding this comment

coderabbitai bot Jan 6, 2024

Choose a reason for hiding this comment

pritansh-sahsani commented Jan 6, 2024 •

edited by coderabbitai bot

coderabbitai bot commented Jan 6, 2024 •

edited

CodeRabbit Configration File (`.coderabbit.yaml`)