>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3377210. --> Reported by: [hestenet](https://www.drupal.org/user/54034) >>> <p>This meeting:<br> &#10148; Is for core developers, initiative contributors, the Drupal Association and anyone interested in the initiative.<br> &#10148; Usually happens every other Wednesday at 11:00 PT / Thursday 18:00 UTC<br> &#10148; Is done over chat.<br> &#10148; Happens in threads, which you can follow to be notified of new replies even if you don&rsquo;t comment in the thread. You may also join the meeting later and participate asynchronously!<br> &#10148; Lots of threads get posted all at once! Don't worry - start at the top and then jump into the threads that most interest you.<br> &#10148; Has a public agenda anyone can add to in the issue<br> &#10148; *Transcript will be exported and posted* to the agenda issue. For anonymous comments, start with a :bust_in_silhouette: emoji. To take a comment or thread off the record, start with a :no_entry_sign: emoji.</p> <p>ping: @mixologic @FeyP @jurgenhaas (go to the issue of the next meeting to add or remove yourself from the list)</p> <h2>Transcript</h2> <h2>0&#65039;&#8419; Who is here today? Comment in the thread below to introduce yourself!</h2> <table> <tr> <td>hestenet (he/him)</td> <td>Tim from the DA kicking off threads</td> </tr> <tr> <td>markdorison</td> <td>Mark from Chromatic :wave:</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>Bj&ouml;rn, always interested.</td> </tr> <tr> <td>smustgrave</td> <td>Hello</td> </tr> <tr> <td>irinaz</td> <td>Irina Zaks, working from South Bay area&nbsp;(edited)</td> </tr> <tr> <td>kimb0</td> <td>Kim, Sydney. Enjoying the World Cup :soccer:.</td> </tr> <tr> <td>wimleers (he/him)</td> <td>:wave: catching up</td> </tr> </table> <h2>1&#65039;&#8419; Do you have any topics to propose for the meeting today? Feel free to propose them in this thread, and then I will give them their own unique threads for discussion. Conversation moving slow? Go ahead and open your own thread in the next numeric order.</h2> <table> <tr> <td>moshe</td> <td>I'd love for "issues on Gitlab" to get a thread today and in each meeting until its completed.&nbsp;(edited)</td> </tr> <tr> <td>hestenet (he/him)</td> <td>Sure - that's been more or less what I've been doing.</td> </tr> </table> <h2>2&#65039;&#8419; Personal Access Tokens!@drumm was able to get them enabled after we squared away the infra issues needed to get us onto GitLab 16.xHowever, we are aware that there are some api paths that need adjusting b/c they are not fully working as expected. Still - progress!</h2> <table> <tr> <td>cmlara</td> <td>I would suggest <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-1"><a href="https://www.drupal.org/project/infrastructure/issues/3379836" title="Status: Active">#3379836: Allow full access to Gitlab API (subject to user permission limits)</a></span> is worth discussing in relation to PAT&rsquo;s and access rather than targeting individual endpoint targets.</td> </tr> <tr> <td>drumm</td> <td>It&rsquo;ll take some time to collect the upstream GitLab issues which would need to be resolved. For example <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/377750">https://gitlab.com/gitlab-org/gitlab/-/issues/377750</a>. And <a href="https://gitlab.com/groups/gitlab-org/-/epics/4035">https://gitlab.com/groups/gitlab-org/-/epics/4035</a> will be huge when we can set access for things like deleting projects and changing their name. Until GitLab has more fine-grained access control, we can&rsquo;t really open up a whole slew of data integrity issues</td> </tr> <tr> <td>cmlara</td> <td>Fair on needing more time to get the issues together, it would be useful to have that so we could look at it and see if we can make some of these concerns non blockers.Delete a project role is an owner role permision which we don&rsquo;t grant to anyone so that seems like a non issue currently.Regarding emails addresses. Didn&rsquo;t we move that to GitLab recently to let it take control in <span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/drupalorg/-/work_items/3300281" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/drupalorg/-/work_items/3300281</a></span>? The rename one I&rsquo;ll give you looks like it could be more of a problem, I see that is a maintainer level permission. Though I&rsquo;m not sure we actually are preventing that right now either (I&rsquo;m not going to submit the API calls to try and validate that thought, though it would be interesting for somone with access to the D.O. Lab to test)</td> </tr> </table> <h2>3&#65039;&#8419; Gitlab CI3&#65039;&#8419; 1&#65039;&#8419; Optimizing testing - trying to make core runs perform comparable to on DCI@fjgarlin has been doing more work there, we're going to evaluate some config changes to our instance that may help.3&#65039;&#8419; 2&#65039;&#8419; D7 testing support - still could use attention from people who want to try and get it working. Mostly just needs the composer steps skipped with a flag in the template, I think.&nbsp;(edited)&nbsp;</h2> <table> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>Is there an issue somewhere that the core performance stuff is tracked?</td> </tr> <tr> <td>hestenet (he/him)</td> <td>Good question.. I know some back and forth has happened in Slack and a bit in our internal tracker. Let me see if there's an issue..</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>cool, yeah we had a small discussion on some options &#128578;</td> </tr> <tr> <td>hestenet (he/him)</td> <td><span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/gitlab_ci_testbed_for_drupal_core/issues/3375826" title="Status: Closed (fixed)">#3375826: Investigate performance optimizations</a></span></td> </tr> <tr> <td>hestenet (he/him)</td> <td>(really have to check out the MR for any detail)</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>ty</td> </tr> <tr> <td>hestenet (he/him)</td> <td>Also an older issue for some test inconsistencies that hasn't had new attention: <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/gitlab_ci_testbed_for_drupal_core/issues/3375827" title="Status: Closed (fixed)">#3375827: Investigate tests failures</a></span></td> </tr> <tr> <td>paul121</td> <td>curious if anyone has had a chance to look at this re: docker in gitlab CI? <a href="https://drupal.slack.com/archives/CGKLP028K/p1689954070118739">https://drupal.slack.com/archives/CGKLP028K/p1689954070118739</a></td> </tr> <tr> <td>hestenet (he/him)</td> <td>Oh... That's a very good reminder. We do need to look at that config change</td> </tr> <tr> <td>fjgarlin</td> <td>The above issues are the two main ones currently regarding gitlabci and D10 core. I need to update the performance issue as I&rsquo;ve done most tests locally. I haven&rsquo;t looked at the failing tests (other than accidentally messing the MR and reverting the change :sweat_smile: ). I will update it tomorrow.</td> </tr> <tr> <td>fjgarlin</td> <td>I updated <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/gitlab_ci_testbed_for_drupal_core/issues/3375827" title="Status: Closed (fixed)">#3375827: Investigate tests failures</a></span></td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>I think the whole 'put everything in memory' that drupalci seems to do it a big part really.We should perhaps try if we can use something like this:[[runners.kubernetes.volumes.empty_dir]]<br> name = "empty-dir"<br> mount_path = "/path/to/empty_dir"<br> medium = "Memory"This will probably not really work for the database, since that needs special attention (and scripting) to make that work. But this could work for the build directory. If so, we should be able to see improvements to the unit tests since that is IO+CPU mostly.</td> </tr> <tr> <td>fjgarlin</td> <td>We found this <a href="https://docs.gitlab.com/runner/executors/docker.html#mount-a-directory-in-ram">https://docs.gitlab.com/runner/executors/docker.html#mount-a-directory-in-ram</a> on gitlab documentation for the DB. maybe we can do the same for $CI_PROJECT_DIR . as in DB + current dir using tmpfsedit: not sure if the latter is possible: <a href="https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3090">https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3090</a>. need to investigate further on thisDB: <a href="https://docs.gitlab.com/runner/executors/kubernetes.html#mount-volumes-on-service-containers ">https://docs.gitlab.com/runner/executors/kubernetes.html#mount-volumes-on-service-containers </a>(edited)</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>Still is probably first validate with unit tests</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>I'd *</td> </tr> <tr> <td>drumm</td> <td>We went ahead and deployed putting the MySQL data directory in memory, please report any issues</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>Excitet to see what this does &#128578; (cc @fjgarlin;) )</td> </tr> <tr> <td>fjgarlin</td> <td>testing to see differences right now :grin:</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>:smile:</td> </tr> <tr> <td>fjgarlin</td> <td><a href="https://git.drupalcode.org/project/gitlab_ci_testbed_for_drupal_core/-/merge_requests/5/pipelinesBefore:">https://git.drupalcode.org/project/gitlab_ci_testbed_for_drupal_core/-/merge_requests/5/pipelinesBefore:</a> 1h 34minAfter: 28 min&#10084;&#65039; :tada: :muscle:</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>Yay</td> </tr> <tr> <td>wimleers (he/him)</td> <td>YAYYYYYYY :smile:</td> </tr> <tr> <td>wimleers (he/him)</td> <td>28 mins &hellip;</td> </tr> <tr> <td>wimleers (he/him)</td> <td>&hellip; is faster than DrupalCI?!</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>yeah</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>:stuck_out_tongue:</td> </tr> <tr> <td>wimleers (he/him)</td> <td>Is this a subset of the full test suite?</td> </tr> <tr> <td>wimleers (he/him)</td> <td>Or am I dreaming? :smile:</td> </tr> <tr> <td>wimleers (he/him)</td> <td>PINCH ME</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>Check the issue, it is actually faster by a little</td> </tr> <tr> <td>wimleers (he/him)</td> <td>If so holy shiiiiiiiiiiiiiiiiiiit</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>"We are on numbers slightly better than DrupalCI after that change, so I think this issue can be closed."</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>But there might be a little more performance to be had :wink:</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>although minimal</td> </tr> <tr> <td>fjgarlin</td> <td>yup, full suite :grin:&nbsp;(edited)</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>@fjgarlin little question, have you tried the parralel option that was added a while ago?</td> </tr> <tr> <td>fjgarlin</td> <td>the parallel is only for contrib. I did try with core but it wouldn&rsquo;t work. let me find the issue</td> </tr> <tr> <td>fjgarlin</td> <td><span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/gitlab_ci_testbed_for_drupal_core/issues/3374070" title="Status: Closed (fixed)">#3374070: Experiment with concurrency package</a></span></td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>i mean, if we can parralize this one, we could perhaps devide it by like 8 :x</td> </tr> <tr> <td>fjgarlin</td> <td>it seems that the run-tests.sh script is really leveraging the concurrency + apache already. <a href="https://git.drupalcode.org/project/drupal/-/blob/11.x/core/scripts/run-tests.sh#L553">https://git.drupalcode.org/project/drupal/-/blob/11.x/core/scripts/run-tests.sh#L553</a></td> </tr> <tr> <td>wimleers (he/him)</td> <td>@fjgarlin Dude, you&rsquo;re underselling this majestic improvement :smile: :smile:You made me enable CAPS LOCK for the first time in 2023 :joy:<span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/gitlab_ci_testbed_for_drupal_core/issues/3375826" title="Status: Closed (fixed)">#3375826: Investigate performance optimizations</a></span>#comment-15201276</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>ah fair enough</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>@wimleers (he/him) we need to make jsonapi tests faster, those eat up so much time lol :stuck_out_tongue_winking_eye:</td> </tr> <tr> <td>wimleers (he/him)</td> <td>@Bj&ouml;rn Brala (bbrala) I have been wondering if we can&rsquo;t just delete many of those tests &mdash; many of them were necessary to get things to be stable. Or maybe not the JSON:API ones &mdash; maybe just the REST ones (JSON+anon, JSON+cookie, JSON+basic_auth, XML+anon, XML+cookie, XML+basic_auth &larr; huge amount of repetition!)</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>There probably is yeah. At least a large part of the code path leading up to the different formats.</td> </tr> <tr> <td>Bj&ouml;rn Brala (bbrala)</td> <td>Sometimes I wish I had like 4 days a week for these kind of things lol</td> </tr> </table> <h2>4&#65039;&#8419; Issues on GitLabCurrent state:Awaiting SSO migrationOn our larger roadmap the things ahead of this are:Finalization of TUF/Rugged for AutoUpdates/Project browser - we are engaging a security auditor next - see #autoupdates (core target for 10.2)Project browser endpoints deployed - (there's a fairly major db maintenance project involved here as well (core target for 10.2)Then SSO (blocker for almost all of the next generation of work we want to do, just need to get it knocked out)Then deployment of fork management bot written by @fjgarlin as well as credit migration. Then issues enabled as opt-in per project. (edited)</h2> <table> <tr> <td>hestenet (he/him)</td> <td>SSO migration status: waiting to hear back from Tag1 project management team when we can schedule</td> </tr> <tr> <td>hestenet (he/him)</td> <td>DB maintenance status prior to PB endpoints: Also has Tag1 team support (mostly a different set of folks, some overlap)</td> </tr> <tr> <td>hestenet (he/him)</td> <td>I would personally love to have an issue opt-in available for Lille. It may be tight with the non-GitLab core priorities.</td> </tr> <tr> <td>markdorison</td> <td>That would be amazing, but yes there is a lot to accomplish by then</td> </tr> <tr> <td>wimleers (he/him)</td> <td>How&rsquo;s the security audit for AU/PB coming along? It&rsquo;s now been &gt;2 months since we discussed it at DrupalCon. What&rsquo;s the relevant issue on d.o?</td> </tr> <tr> <td>wimleers (he/him)</td> <td>Perhaps better here: <a href="https://drupal.slack.com/archives/C7QJNEY3E/p1691514114313849">https://drupal.slack.com/archives/C7QJNEY3E/p1691514114313849</a></td> </tr> <tr> <td>wimleers (he/him)</td> <td>Copying my comment there :+1:</td> </tr> </table> <p>Participants:</p> <p>moshe, hestenet, cmlara, drumm, Bj&ouml;rn Brala (bbrala), paul121, fjgarlin, Wim Leers, markdorison</p>