Distribution Modernization Initiative meeting on Dec 7, 2021
>>> [!note] Migrated issue
<!-- Drupal.org comment -->
<!-- Migrated from issue #3250918. -->
Reported by: [ronaldtebrake](https://www.drupal.org/user/2314038)
>>>
<p>This meeting:<br>
➤ Is for distribution developers, initiative contributors, the Drupal Association and anyone interested in the initiative.<br>
➤ Usually happens every other Tuesday at 1400 UTC.<br>
➤ Is done over chat.<br>
➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!<br>
➤ Has a public agenda anyone can add to.<br>
➤ *Transcript will be exported and posted* to the agenda issue. For anonymous comments, start with a :bust_in_silhouette: emoji. To take a comment or thread off the record, start with a :no_entry_sign: emoji.</p>
<p>ping: @kingdutch @bircher @briangilbert @navneet0693 @Tiago Barreiro de Siqueira @Mohammed Razem @finn @dww @Rajab Natshah (go to the issue of the next meeting to add or remove yourself from the list)</p>
<p>:thankful: I'd also like to take a moment to highlight our initiative sponsor! Discover Global Network has generously stepped forward to provide resources that are allowing the Drupal Association to make the necessary updates to modernize distribution support on Drupal.org. Feel free to check them out at: <a href="https://www.discoverglobalnetwork.com/">https://www.discoverglobalnetwork.com/</a></p>
<h2>0️⃣ Who is here today? Comment in the thread below to introduce yourself and tell us why you are joining us. Do you have a pet?</h2>
<table>
<tr>
<td>mixologic</td>
<td>It is, I, Ryan, from the Drupal Association. I maintain packages.drupal.org and will be implementing a lot of these changes. I have two big ol siberian cats, Conrad and Huxley.</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>Hi, This is Rajab, Technical Product Lead at Vardot. A maintainer of the Varbase distribution.I love to help in speeding up the work of web developers. by setting the way of building custom Drupal Distributions. + the extra Webapps</td>
</tr>
<tr>
<td>finn</td>
<td>Hello! Finn here from Agile Collective, Oxford, UK working with the LocalGov Drupal distribution. 'We' have a dog since lockdown, very cute, but it is not mine, terrible idea if you ask me! We also have a cat who is very cute.</td>
</tr>
<tr>
<td>ronaldtebrake</td>
<td>Hi! Ronald maintainer from Open Social here, interested to see where this all takes us :slightly_smiling_face:.And no pets that I know of.. :see_no_evil:</td>
</tr>
<tr>
<td>rszrama</td>
<td>Ryan here, from Centarro, checking in on updates. :slightly_smiling_face:</td>
</tr>
<tr>
<td>navneet0693</td>
<td>Hello, I am Navneet :wave: from Open Social, I have always loved to contribute in Drupal distributions, for example, OOTB.No pets yet, but love dogs :dog_puppy:</td>
</tr>
<tr>
<td>rszrama</td>
<td>hah, I missed the pet question … two dogs, two cats, one pet for each child :rolling_on_the_floor_laughing:</td>
</tr>
<tr>
<td>bircher</td>
<td>Fabian, no pets but lots of books with cute animals in them</td>
</tr>
<tr>
<td>Warped</td>
<td>Dave from Chicago. Had a german shepherd/lab mix and a black lab. None currently. Cats and little dogs get under my big feet, so not a good choice for me.</td>
</tr>
<tr>
<td>briangilbert (realityloop) (he/him)</td>
<td>Brian from Melbourne, Australia. have a Tiboodle names Chocci (Tibetan Spaniel crossed with Toy Poodle)</td>
</tr>
</table>
<h2>1️⃣ THE BIG AGENDA THREAD: Do you have any topics to propose for the meeting today? Feel free to propose them in this thread, and then I will give them their own unique threads for discussion. Conversation moving slow? Go ahead and open your own thread in the next numeric order</h2>
<table>
</table>
<h2>2️⃣ A big part of this initiative is making a much better home for distributions on drupal.org, and in conjunction, enabling proper composer support. We have a plan, please let us know what we missed: <span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/project_composer/-/issues/3252534" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/project_composer/-/issues/3252534</a></span></h2>
<table>
<tr>
<td>mixologic</td>
<td>First thing that has come up is a desire for a mechanism to preserve the current namespace for project templates. We will likely add an additional field to the general project type that toggles between using the external namespace, or using the drupal/ namespace. (edited)</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>I'm with the default of internal project template + internal profile</td>
</tr>
<tr>
<td>mixologic</td>
<td>@Rajab Natshah so would drupal/varbase work for the template, and drupal/varbase-profile be acceptable for the namespace of the install profile?</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>drupal/varbase and drupal/varbase-projector drupal/varbase-distro</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>One will do like drupal/varbase for the root composer.jsonno need for 2 composer files</td>
</tr>
<tr>
<td>ronaldtebrake</td>
<td>I think from Open Social side we are happy with the flexibility of a mechanism but in reality are more than happy with having the drupal/ namespace :slightly_smiling_face:</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>The distro git package can have all needed files in it + the installation profile in /web/profile/distro-nameThe Drupal.org project packing could not do anything rather than update the version for releases in the .info.yml ( which we do manually now )</td>
</tr>
<tr>
<td>mixologic</td>
<td>oh, I think that updating the .info.yml file is something we're thinking about removing.</td>
</tr>
<tr>
<td>mixologic</td>
<td>because we plan on not packaging distros. we should discuss this on thread 4️⃣</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>Noted</td>
</tr>
<tr>
<td>rszrama</td>
<td>@mixologic re: your latest comment in the d.o issue, are there update implications for projects like ours already using a different schema? or is it really just a simple name change and composer update? I suppose in the end, it doesn’t really matter what the composer package is called, since Drupal tracks the actual profile machine name, right? so we’d just have a package name that differed from the profile machine name?</td>
</tr>
<tr>
<td>mixologic</td>
<td>Right, we wouldnt change the profile machine name, just the namespace at the facade, which, while there has been something advertised for profiles, it's always been broken, so we can replace them without repercussions.</td>
</tr>
<tr>
<td>mixologic</td>
<td>The only thing that might need to change is what is required in the template.</td>
</tr>
<tr>
<td>bircher</td>
<td>I think drupal/distro_name-project is better for the template because you are going to composer create a thing for a project. so I wouldn't invert anything, but otherwise the plan looks great</td>
</tr>
<tr>
<td>Warped</td>
<td>@Rajab Natshah A composer.lock file included might be nice because they could install a working tested version. Then do updates after. A better chance of installing a working site before adding upgrade steps.</td>
</tr>
<tr>
<td>briangilbert (realityloop) (he/him)</td>
<td>@mixologic I'd like to see composerr projects being something that we can host on d.o ideally (I didn't infer that was part of the proposal from the issue?) (edited)</td>
</tr>
<tr>
<td>mixologic</td>
<td>@briangilbert (realityloop) (he/him) I think it is part of the proposal. If, by composer project you mean a 'composer template'.</td>
</tr>
<tr>
<td>briangilbert (realityloop) (he/him)</td>
<td>@mixologic if you install it usingcomposer create-projectthen that is what I mean (edited)</td>
</tr>
<tr>
<td>mixologic</td>
<td>yes. those are composer templates, and will have the composer type of project</td>
</tr>
</table>
<h2>3️⃣ Another question we ran into is whether or not distributions have a tendency to ship with optional modules that, in turn, have additional composer dependencies inside of them.</h2>
<table>
<tr>
<td>mixologic</td>
<td>Our current plan is to only support a composer.json at the root of a distribution profile project, and all dependencies, optional or otherwise, must be declared there. Drupal has always had the module install/uninstall idea, but composer has no such notion of optional dependencies.</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>A need for a Composer Drupal Plugin Requesterto work as an agent from the install/uninstall UIA bit like the Composer UIHaving a ComposerDrupalPluginRequester class as a service. could let any module get any other module. based on an optional list of modules (edited)</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>We do this a lot in our vardot/varbase-updater</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>The composer plugin will read a custom file like update-config.json and follow some instructions on what to do in case of adding a new module, skipping a module.The same mechanism could work with optional-composer.json</td>
</tr>
<tr>
<td>mixologic</td>
<td>I had a look at that updater. Very interesting, I think we should take a closer look at what you're doing there.</td>
</tr>
<tr>
<td>rszrama</td>
<td>presently Commerce Kickstart has a root composer.json that gathers all dependencies any of our optional modules need … we’ve been on the fence about our optional modules, to be honest - like, whether or not to just spin ’em out into separate projects … keeping them all together right now to avoid the maintenance overhead of a dozen separate issue trackers and such</td>
</tr>
<tr>
<td>rszrama</td>
<td>but if we truly needed additional dependencies or didn’t want to force the installation of code at the root level (i.e. Commerce Recurring) that a site may not need, we can easily just pull that out to an external project</td>
</tr>
<tr>
<td>kingdutch</td>
<td>Open Social declares all possible dependencies in the root. This makes the most sense for version management and disk space for unused packages is cheap.We can’t download anything during the site install phase. Multiple of the hosting providers we use have locked filesystems at that point (platform.sh being the most prominent one).We have non-public extensions with their own requirements and making sure all permutations always install is quite an amount of work. I expect pushing dependencies into more optional moments will cause similar pains.</td>
</tr>
<tr>
<td>kingdutch</td>
<td>(For those same reasons also very happy with just having this managed by the root distribution composer.json file :))</td>
</tr>
<tr>
<td>mixologic</td>
<td>Great. This all aligns. Drupal's history of optional modules was good back in pre-autoloader days, but disk is just not expensive enough to warrant concern about unused extra code IMO.</td>
</tr>
<tr>
<td>Warped</td>
<td>And less chance of overlooking any security updates in unused code if all the dependencies are stated up front.</td>
</tr>
</table>
<h2>4️⃣ part of the above proposal also included that fact that drupal.org would no longer provide packaged distributions. Thoughts?</h2>
<table>
<tr>
<td>mixologic</td>
<td>Licensing has always been a muddy bear, and its questionable if there is still value in providing a tar.gz of a profile. We plan on removing this, and directing all future users to starting a distribution with composer.</td>
</tr>
<tr>
<td>mixologic</td>
<td>Almost every distro has run into the issue where they want to include something from asset-packagist, or other third party libs, that are not clearly compatible with GPLV2_or_later</td>
</tr>
<tr>
<td>ronaldtebrake</td>
<td>Can only confirm that we’ve encountered these issues and we also noticed more evil than good coming out of this for our end users. Mainly with regards to updating/maintaining these. (edited)</td>
</tr>
<tr>
<td>mixologic</td>
<td>Ah, so one thing that happens is drupal.org packaging manipulates the .info.yml file in the tarball to add the version number etc. This gets used by updates.drupal.org to tell end users their profile may/may not have a security issue.</td>
</tr>
<tr>
<td>mixologic</td>
<td>so we'll need to discover a way to do that some other way. Perhaps something in <a href="https://drupal.org/project/composer_deploy">https://drupal.org/project/composer_deploy</a> could be migrated into core? unsure.</td>
</tr>
<tr>
<td>Gábor Hojtsy (he/him)</td>
<td>composer deploy is good and needed either way for people using composer?</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>It's ok to have no packaging.Releasing tags are important, I suggest keeping that.Maybe a pre-release checklist should be documented then.Like:Change the .info.ymlChange all *.0.x-dev versions with tagged versions.Update the composer.lock with the latest tag.... more could be listed .. as we do for every releaseIf you worked with npm or yarn ..They do have a command for releasing .. it will change things and tag then push that tag too</td>
</tr>
<tr>
<td>kingdutch</td>
<td>I suspect @ronaldtebrake meant “more evil than good” if I look at support requests he’s handled :yum:</td>
</tr>
<tr>
<td>ronaldtebrake</td>
<td>Haha thanks @kingdutch you’re absolutely right :sleeping: [editted the message above] (edited)</td>
</tr>
<tr>
<td>Warped</td>
<td>Any chance of requiring a function that could retrieve the current version for the dependent library or other object? That should only need a few examples to cover the majority of ways to check.</td>
</tr>
<tr>
<td>hestenet (he/him)</td>
<td>so we'll need to discover a way to do that some other way. Perhaps something in <a href="https://drupal.org/project/composer_deploy could">https://drupal.org/project/composer_deploy could</a> be migrated into core? unsure. Do we need to go ahead and open an issue about this in the core queue?</td>
</tr>
</table>
<h2>5️⃣ Another aspect of this is redesigning the distributions landing page and discovery for end users.</h2>
<table>
<tr>
<td>mixologic</td>
<td>Right now, when somebody navigates to the distributions link off of the main menu, they are presented with a views like query interface, with some values pre-selected.</td>
</tr>
<tr>
<td>mixologic</td>
<td>image.png</td>
</tr>
<tr>
<td>mixologic</td>
<td>What we'd like to do is figure out a better mechanism for discovery. Most of these data fields arent particularly relevant, or they are ambiguous, or they dont work at all ('works with')</td>
</tr>
<tr>
<td>mixologic</td>
<td>We are also now going to have both template projects and install profile projects.. do we only list templates here? both?</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>I hope that page gets better.Drupal now has built-in flavoursIf you look at Ubuntu Distros of flavours<a href="https://ubuntu.com/download/flavours">https://ubuntu.com/download/flavours</a></td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>I did not know that they are listing xubuntu for example. But it was not maintained by the same canonical company (edited)</td>
</tr>
<tr>
<td>mixologic</td>
<td>yeah, thats a good example of something we'd like to do. make it much less 'dashboardy' an much more use-case driven (edited)</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td>Both .. profile projects + flavour ^templates.For example the Drupal Commerce profileIf you look them up. They do have many project templatesLike:Drupal CommerceDrupal Commerce Digital assetsDrupal Commerce Physical assets ( Store )Drupal Commerce Demo(edited)</td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td><a href="https://github.com/drupalcommerce">https://github.com/drupalcommerce</a></td>
</tr>
<tr>
<td>Rajab Natshah</td>
<td><a href="https://github.com/drupalcommerce/demo-projecthttps://github.com/drupalcommerce/project-basehttps://github.com/drupalcommerce/demo_commerceI">https://github.com/drupalcommerce/demo-projecthttps://github.com/drupalcommerce/project-basehttps://github.com/drupalcommerce/demo_commerceI</a> was researching one day. And they do have one project demo which is exceptional. But it was hard to capture</td>
</tr>
</table>
<p>Participants:</p>
<p>Mixologic, Rajab Natshah, finn, ronaldtebrake, rszrama, navneet0693, bircher, Warped, briangilbert (realityloop) (he/him), kingdutch, Gábor Hojtsy, hestenet</p>
issue