Drupal AI Contribution meetup 27 Feb 2025 (#3508193) · Issues · project / ai

Drupal AI Contribution meetup 27 Feb 2025

>>> [!note] Migrated issue   Reported by: [valthebald](https://www.drupal.org/user/239562) >>> Drupal AI Contribution meetup from 27 Feb, 2025 at 13:00 UTC to 28 Feb 2025 13:00 UTC (Check your local time at <a href="https://www.timebie.com/std/universal.php">https://www.timebie.com/std/universal.php</a>) <h2>Where</h2> #ai-contrib channel in <a href="https://www.drupal.org/slack/">Drupal Slack</a> <h2>Attendees</h2> (in the order of appearance in the chat) <h2>Agenda</h2> <h3>Follow-up to topics of <a href="https://www.drupal.org/project/ai/issues/3503762" title="Status: Closed (fixed)">#3503762: Drupal AI Contribution meetup 2025.3</a></h3> <h3>Current progress in AI core and the ecosystem</h3> <h3>Suggest your topic in comments</h3> <ul> <li>Hello everyone, and welcome to the Drupal AI contribution meeting!This meeting:➤ Happens every other week in #ai-contribute channel➤ Is text only!➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously.➤ Refers to various issues and plans tracking in d.o project:<a href="https://www.drupal.org/project/ai/issues/3508193" title="Status: Closed (fixed)">#3508193: Drupal AI Contribution meetup 27 Feb 2025</a>. Please comment there so we can grant you credit for attending/contributing! Please note, we will only give attributions for active engagement. (edited)  <ul> </ul></li> </ul> <li>0️⃣ Roll call! Who's here? Please specify your <a href="http://drupal.org">drupal.org</a> username. What's your favourite submodule of AI module in Drupal? <ul> <li>valthebald: Val "valthebald" from Sofia, Bulgaria. And my favourite submodule is ai_translate 🙂</li> <li>sudofly: Alex “sudofly” from Fort Lauderdale, Florida USAai_testing … my module, that is just an idea as of now but I can’t wait to check in some code</li> <li>valthebald: I like the sound of it!</li> <li>valthebald: testing as in phpunit test?</li> <li>sudofly: My background is Behat/Selenium end-to-end testing</li> <li>sudofly: so UI testing</li> <li>valthebald: that's nice! Behat is one of my favorite tools 🙂</li> <li>sudofly: Yeah I’ve put it through its paces, running Selenium in Lando docker containers and inside GitLab CI/CD pipelines.. using aggressive Javascript to click into cross-site iFrames, all sorts of fun stuff</li> <li>Vivek Panicker: Vivek from Kolkata, India.My favorite submodule would be ai_chatbot.</li> <li>thejimbirch: thejimbirch, AI CKeditor, as it is so easy to set up and configure and works great out of the box.</li> <li>yautja_cetanu (Jamie): Jamie and favourite is Automators 🙂</li> <li>Marcus Johansson: Marcus (marcus_johansson) from Berlin. The abstraction layer, aka core 🙂</li> <li>Jasper Lammens: Jasper (lammensj) from Nijlen. Favorite is the one for ECA .</li> <li>bircher: bircher :wave: my favourite is the image alt text generation, maybe with automators :smile:</li> <li>yautja_cetanu (Jamie): oh if that is the case @Scott Euser and especially @mandclu are working on making a widget front-end for AI stuff in the Drupal AI ecosystem like Augmentors and like the image alt text generation but probably with Automators (and other backends like simple prompts and agents)</li> <li>Scott Euser: scott_euser here, London UK, still AI search for me but lately getting more interested in the augmentor/automator updates like mentioned above. Currently working on <a href="https://www.drupal.org/node/3509235">Prompt Management</a> - should have enough progress in some days to start to gather features/feedback, but slightly too early still 🙂</li> <li>bircher: I am very interested also in experimenting with automators and the symfony messenger (sm on d.o)</li> </ul></li> <li>1️⃣ Followup to previous meeting[#3503762] <ul> </ul> </li> <li>1️⃣ 1️⃣ Bringing search_api out of experimental[#3485449] <ul> <li>valthebald: this one hasn't seen any change for about a month. How can we help push it?</li> <li>yautja_cetanu (Jamie): @Seogow has been working on it. Do you have any updates of when? We found issues with how we handle indexing very large nodes into chunks that we are trying to fix.</li> </ul> </li> <li>1️⃣ 2️⃣ Programmatically add operation type[#3500120] <ul> <li>valthebald: Marked RTBC but not merged yet. Also, the issue is open against 1.0.x but not 1.1.x (let's discuss this separately later)</li> </ul> </li> <li>1️⃣ 3️⃣ Normalize function call <a href="https://www.drupal.org/node/3479388">https://www.drupal.org/node/3479388</a> <ul> <li>valthebald: That's a huge topic, and there's great discussion about using contexts (and what context is)</li> <li>valthebald: any feedback appreciated</li> </ul> </li> <li>1️⃣ 4️⃣ Normalize structured responses[#3492869] <ul> <li>valthebald: If understand correctly, both this and previous topic can benefit from using MCP, am I right @Giorgi Jibladze @yautja_cetanu (Jamie)?</li> <li>yautja_cetanu (Jamie): kinda... its more that MCP will benefit from this. This is more generally interacting with LLMs.MCP adds a lot more stuff around it.</li> <li>yautja_cetanu (Jamie): Its hard to truly "use" MCP until MCP allows servers and clients to talk over the internet. Even then it adds loads of specific things like a transfer protocol layer that we'd never use inside drupal</li> <li>yautja_cetanu (Jamie): I think I misunderstood a lot beccause I thought the MCP tools schema was unique to MCP but a lot of the JSON schema is in the OpenAI SDK and API itself and so its something different models implement differently in theory</li> <li>kevinquillen: DTO or value objects?</li> <li>yautja_cetanu (Jamie): Confused by this?</li> <li>kevinquillen: Oh those structured responses. I thought by the title it meant a common response interface regardless of the provider</li> </ul></li> <li>1️⃣ 5️⃣ Deepchat history between sessions (chatgpt-like memory) <ul> <li>valthebald: Issue to be created</li> </ul> </li> <li>:one: :six: Drupal AI website <ul> <li> valthebald: Issue to be created. If no objections, I'll create a public repo on github (or do we prefer gitlab? I'd prefer github because of better packagist support)</li> <li>yautja_cetanu (Jamie): go for it! :+1:</li> <li>I think I have ideas of what I want the Drupal AI Module site to do: I want it to be a place where you can learn about the AI module in a way that is easier than d.org I want it to be a neutral place that isn't pure Freely Give to talk about the AI module. I want to be able to post updates on the whole AI Ecosystem there including patch notes and stuff and then have that in drupal planet. I want there to be a place where we can highlight key issues we are working on or need working on. I want someone who says "?Hey I have 2-3 hours free lets help AI" could come her and know what to do and where to go. I want to highlight organisations working on bits of the AI ecosystem, show what they are currently working on and what issues they are working on. I want to find a way that the Drupal AI ecosystem and maybe opencollective can get money and organisations that contribute can get rewarded. I'd like a workflows-of-ai.com list of use cases, that have recipes and ideally we'd strart providing basic hosting to trial loads of different uses of it. Have marketting video stype documentation alongside developer docs. Really show off why its cool :heavy_plus_sign:</li> <li>valthebald: @yautja_cetanu (Jamie) that's a solid list to start from!</li> <li>Scott Euser: Maybe a wishlist thing for one day; but a contributor login could allow revisions/wiki style like drupal.org does - if its intention is to become a living source of knowleged <img src="https://www.drupal.org/files/issues/2025-03-04/image.png" alt="Wish list"></li> <li>yautja_cetanu (Jamie): yes definitely!</li> </ul> </li> <li>2️⃣ Organisational questions <ul> </ul> </li> <li>2️⃣ 1️⃣ Issue versions <ul> <li>valthebald: Current default version is 1.0.x despite existence of 1.1.x. That leads to issues created against 1.0 (even adding new features)</li> <li>valthebald: I suggest changing default to 1.1.x and adding "needs backport to 1.x" tag when needed</li> <li>valthebald: and keep 1.0.x only for issues that do not affect 1.1.x (are there such issues?)</li> <li>Scott Euser: I believe the intention here was that 1.0.x is like current stream and things that need more review when upgrading (more major features) go into 1.1.x, but maybe @Marcus Johansson you can comment on your plan with it since its you leading that.FWIW the drupal core way of doing it would be:Everything targets highest branch which gets set as default (e.g. core 11.x)On merge maintainer decides whether to backport (e.g. core 10.4.x)</li> <li>valthebald: I was having drupal core flow in mind when suggesting 🙂in case of AI, next stable is 1.1.x and current stable is 1.0.xmy concern is that if issues are being committed by default to 1.0.x, 1.1.x will face regressions</li> <li>Marcus Johansson: So, we have still a 1.0.5 coming out and 1.1.x-dev is for major changes. Right now I'm trying to keep 1.1.x up to date manually, since we haven't merged anything there yet. We might have one breaking change also in terms of how streaming and the logging of that works.After 1.0.5, this could perhaps be changed like this, that 1.1.x is the default version.</li> </ul></li> <li>2️⃣ 2️⃣ Leave the Drupal island? <ul> <li>valthebald: AI module offers nice abstraction layer for different providers, how specific to Drupal is this abstraction?</li> <li>valthebald: Would it make sense to have "generic" PHP components in a way similar to commerce module and commerceguys/addressing component? (edited)</li> <li>valthebald: If yes, which components can be done Drupal-agnostic?</li> </ul> </li> <li>2️⃣ 3️⃣ Github/gitlab organisation <ul> <li>valthebald: Related to previous question, is there a need for a separate organisation (drupalai?) in github/gitlab?</li> </ul> </li> <li>2️⃣ 3️⃣ Decoupled DeepChat <ul> <li>valthebald: question from the previous meeting, issue to be created.</li> <li>valthebald: I still think this is something many AI users can benefit from</li> <li>Marcus Johansson: What does this refer to - using it without AI Assistants API?</li> <li>Marcus Johansson: There is this idea then,<a href="https://www.drupal.org/project/ai/issues/3492940" title="Status: Reviewed & tested by the community">#3492940: Add ChatProcessor</a>. But even without that we could easily make the endpoint being used a setting in the block.</li> <li>valthebald: @Marcus Johansson using it outside of Drupal admin interface. As an example, as Telegram mini-app</li> <li>Marcus Johansson: Like embeddable on other websites?</li> </ul></li> <li>3️⃣ Updates from module maintainers (what happens, where help is needed most etc.) <ul> <li>Scott Euser: I know "Boost with AI" processor within AI Search module is starting to get used a bit more; it works well with Search API Database as the primary index. However if anyone is using SOLR and that, this issue is still what's really holding back SOLR version:[#3491446]I don't think it should be too hard to follow the pattern I used for database and I think my suggestion in issue summary probably should do the trick.</li> </ul></li>  <li>:three: :one: @yautja_cetanu (Jamie) <ul> <li>yautja_cetanu (Jamie): We have a few big things on its way: Much better and easier to visualise logging. Can download the prompts from logs and export them to a playground for LLMs to help with prompt engineering. Hopefully soon tool support in the Logs. We have normalised responses and proper function calling. We will have a builder where you can create AI Agents yourself from the UI and all the Drupal Actions. We are next going to add ECA actions. This means making your own agents should become much easier We are creating a whole bunch of AI Agents now including: Agents that can create and edit content Agents that can scrape the web (We need config in the tools and permissions) Agents that can use RAG search (We need config in the tools and permissions before release) I want Agents to have their own roles before we unleash all these agents. <img src="https://www.drupal.org/files/issues/2025-03-04/image%20%282%29.png" alt="preview"> <img src="https://www.drupal.org/files/issues/2025-03-04/editagent.png" alt="preview"> <img src="https://www.drupal.org/files/issues/2025-03-04/subagent.png" alt="preview"> </li> <li>Jasper Lammens: this is so coooool :smile:</li> </ul> </li> <li>3️⃣ 2️⃣ @Marcus Johansson <ul> <li>Marcus Johansson: An issue to have thoughts about - its to make vector database recipe ready:<a href="https://git.drupalcode.org/project/ai/-/work_items/3509315" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/ai/-/work_items/3509315</a></li> <li>Scott Euser: Posted a reply there, interesting one!</li> </ul> </li> <li>4️⃣ Open mike! Suggest your topic here <ul> <li>jurgenhaas: Watching the space, it feels like a lot of entity related AI agents get created and maintained as individual modules for e.g. nodes, paragraphs, blocks, etc. just to name a few that I recently spotted.Wondering if really each entity type requires their own agent or if not a generic agent for entities would do the job?</li> <li>valthebald: good question! Given the size of[#3429343], it becomes more and more difficult even to navigate the list of AI-related modules</li> <li>Jasper Lammens: Initially, you might think that it’s not necessary to have an Agent per entity type. But the amount of operations that you can do with an Agent quickly grows and then it becomes unmaintainable.</li> <li>Jasper Lammens: And in order to keep the AI-module fairly small, I believe it’s a good approach to keep them separate.</li> <li>Jasper Lammens: Even not as a submodule, but as a general contrib one</li> <li>Jasper Lammens: Cause otherwise, AI would have a dependency on ECA, Paragraphs etc.</li> <li>Jasper Lammens: Basically every module that would have an Agent</li> <li>Jasper Lammens: But yeah, something could be approved in terms of providing an overview of available “agent-modules”</li> <li>jurgenhaas: OK, makes sense. Just maybe some common functions maintained by the AI module as a service for all the others? May just increase the overall quality</li> <li>Jasper Lammens: That’s certainly possible</li> <li>Jasper Lammens: The Agent is just a plugin</li> <li>Marcus Johansson: @jurgenhaas - we have an content creating agent using the new structure that is completely generic that works quite well.So its an agent that basically have the following tools it can loop over:Get field information for entity type (What field exists on an entity type/bundle combination and what are their names, types, descriptions, data names, required etc.)Get field data types, so it can figure out what values it needs to fill out and if there are options etc. So for instance a text_long field has the filter_format it needs to fill out.Get current values - if you ask to change a value with some instructions, it can get the current values.Content entity modifier which takes the values and modifies them, saves them.The difference, between the generic one and the specialized one, is that the first two tools are already in context for that specific content type.There are three issues I think with the generic one compared to creating one specifically for some content type/entity type.It needs to fetch a lot of context information on what fields are available, how to fill out those fields etc. - so even without the token generation needed to write actual content, it might take 1-2 minutes to fill out a simple node. Which with a chatbot UI at least becomes unusable.Outside of giving exact information to the agent already from the start with a specific agent, you can write the prompt so it understands the context behind the fields, if this is missing in the entity/field description - how you set a label on a taxonomy term, is different then a node title etc.It works as written quite well - and with newer models that might improve to working almost all the time, however if you see that its doing a common error when filling out a specific field type for a specific agent, you can do prompt engineering specifically there.So with better agents and other orchestration UI's it will work better, but for the Chatbot/Drupal CMS it might be necessary for now to have separate agents.The good thing is in 1.1.0 the creation of the agents will be a lot easier, so we might not have to ship one for each entity type - they could just be recipes.</li> <li>jurgenhaas: That makes sense. And the entity type specifics like labels/titles is something that the Drupal API got wrong in the first place. Same goes for entity ID and others. Although, even that could be generalised by extracting the entity keys, but it's annoying that such common metadata for entities was allowed to be different. It gets even worse with token syntax. Some maintainers have their own ideas how a token property should be named.And that's exactly where my concern comes from: building a new a large new ecosystem around AI, enforcing standards and avoiding duplicate code is something I'd try to apply whereever possible.</li> <li>Marcus Johansson: Right, that is the issue that it needs to check the label key, id key before hand, etc while with a specific agent it knows it from the get-go and can be pushed towards it as well using prompting.</li> <li>Marcus Johansson: And if you need to for instance create a taxonomy and one media image before you create your node, that is 3 lookups just for that, same for looking up other field information etc.</li> <li>Vivek Panicker: There is 1 issue that we can look into for code optimization[#3509181]</li> </ul></li>  Participants: Participants: valthebald, sudofly, Vivek Panicker, thejimbirch, yautja_cetanu (Jamie), Marcus Johansson, Jasper Lammens, bircher, Scott Euser, kevinquillen

issue