>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3528730. --> Reported by: [marcus_johansson](https://www.drupal.org/user/385947) Related to !874 >>> <p>--- AI TRACKER METADATA ---<br> <strong>Update Summary: </strong>ShortTermMemory. Will start this week.<br> <strong>Check-in Date: </strong>MM/DD/YYYY (US format) [When we should see progress/get an update]<br> <strong>Due Date:</strong> MM/DD/YYYY (US format) [When the issue should be fully completed]<br> <strong>Blocked by:</strong> [#XXXXXX] (New issues on new lines)<br> <strong>Additional Collaborators:</strong> @username1, @username2<br> AI Tracker found here: <a href="https://www.drupalstarforge.ai/" title="AI Tracker">https://www.drupalstarforge.ai/</a><br> --- END METADATA ---</p> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>Currently we do not cut off any long running agent reasoning at all, if it gets to long it would run out of tokens and die. This means that for real research issues our agents are working quite poorly at the moment.</p> <p>For assistants we currently just cut of after X amount of sessions if you would like that enabled (or no memory).</p> <p>There are a lot of different ways to handle this:</p> <ul> <li>No memory - it will not even remember the last thing you said.</li> <li>Just keep X amount of the newest messages, the problem is that the agent might repeat using a tool it already used or repeating answers it already answered.</li> <li>Tokenize and just keep X amount of tokens - can be process heavy.</li> <li>Just keep the latest X amount and summarize anything before that into X amount of tokens.</li> <li>Embeddings - you can store the memory in temporary embeddings and get the information based on that.</li> <li>Structured memory template - this is like summary, but it structures more into clear paths like goals, observations, actions, reasoning etc.)</li> <li>And probably 50 other things</li> </ul> <p>We need to make this into something that you can extend any chatbot, agent or other thing that actually works with threads.</p> <p>Note that ShortTermMemoryPlugin, could be used for storing threads over longer time. The memory definition is if its a memory that has been built over longer time or not: <a href="https://superagi.com/introduction-to-agent-summary-improving-agent-output-by-using-lts-stm/">https://superagi.com/introduction-to-agent-summary-improving-agent-output-by-using-lts-stm/</a></p> <p>We will also introduce LongTermMemoryPlugin, but this is a little bit more complex, since its a learning algorithm and from privacy point of view, since you are profiling users - where if you for instance have bought blue dresses before, and ask for a t-shirt it will have a profile of you wanting blue things and give back blue t-shirts.</p> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <ul> <li>Create a ShortTermMemoryPlugin with a plugin manager, attribute and interface.</li> <li>The attribute only needs label and description.</li> <li>The interface should extend configinterface</li> <li>Make an abstract class.</li> <li>Interface should have a method called setThreadId, getThreadId, setRequestId, getRequestId, setCallerId, getCallerId that is also in the abstract that does have an empy identifier.</li> <li>Interface should have a method called, getChatHistory where a plugin can be used to load potential history kept somewhere. Returns an array of ChatMessage.</li> <li>Interface should have a method called, setChatHistory where a consumer can modify the history. Takes array of ChatMessage object.</li> <li>Interface should have a method called, getOriginalChatHistory where a plugin can be used the actual untouched chat history.</li> <li>Interface should have a method called, getOriginalSystemPrompt where a plugin can get the original system prompt.</li> <li>Interface should have a method called, setSystemPrompt where a consumer can modify the system prompt. Takes a string.</li> <li>Interface should have a method called, getSystemPrompt where a consumer can get the system prompt. Returns a string.</li> <li>Interface should have a method called, setTools where a consumer can modify the tools. Takes an array of tools.</li> <li>Interface should have a method called, getTools where a consumer can get the tools. Returns an array of tools.</li> <li>Interface should have a method called transform where a plugin can transform the history, system prompt and tools.</li> <li>Create a first plugin for remember N amount of threads</li> <li>In the AiAgentEntityWrapper, add an object called originalChatHistory, that keeps the original chat history.</li> <li>In the AiAgentEntityWrapper, add an object called originalSystemPrompt, that keeps the original system prompt.</li> </ul> <h3>How it works for a consumer</h3> <p>If the AI Assistants API for instance wants to use this, it will</p> <ol> <li>Add the different plugins as a configuration item on the assistant form.</li> <li>Expose any subform that has detail settings for the plugin.</li> <li>Save those in its own configuration</li> <li>When the end user writes a new message it will (for now) before the request is sent, check if a ShortTermMemoryPlugin is set, create an instance with the configuration, set all the setters and then run transform and load all the changes into the request.</li> <li> </ol> <p>Note, the </p>