Random HTTP timeouts for GitLab CI jobs
>>> [!note] Migrated issue
<!-- Drupal.org comment -->
<!-- Migrated from issue #3445532. -->
Reported by: [cmlara](https://www.drupal.org/user/1790054)
>>>
<h3 id="summary-problem-motivation">Problem/Motivation</h3>
<p>I recently have had 3 consecutive weeks of failure to download Composer dependencies in my weekly jobs.</p>
<p><a href="https://git.drupalcode.org/project/tfa/-/jobs/1522954">https://git.drupalcode.org/project/tfa/-/jobs/1522954</a><br>
<a href="https://git.drupalcode.org/project/tfa/-/jobs/1460015">https://git.drupalcode.org/project/tfa/-/jobs/1460015</a><br>
<a href="https://git.drupalcode.org/project/tfa/-/jobs/1391716">https://git.drupalcode.org/project/tfa/-/jobs/1391716</a></p>
<pre> Failed to download composer/installers from dist: curl error 28 while downloading https://api.github.com/repos/composer/installers/zipball/c29dc4b93137acb82734f672c37e029dfbd95b35: Failed to connect to api.github.com port 443 after 10001 ms: Timeout was reached<br> Now trying to download from source</pre><p>It also appears these failures may be (one of) the cause(s) of random <code>413 Request Entity Too Large</code> errors in G.D.O. artifact archiving.:<br>
<a href="https://git.drupalcode.org/project/smart_date/-/jobs/1639062#L670">https://git.drupalcode.org/project/smart_date/-/jobs/1639062#L670</a><br>
<a href="https://git.drupalcode.org/issue/s3fs-3447227/-/jobs/1695614#L384">https://git.drupalcode.org/issue/s3fs-3447227/-/jobs/1695614#L384</a></p>
<p>A caching proxy would reduce the risks of “random” download failures and make D.O. A more respectful user of 3rd party services.</p>
<p>Alternative could be to deploy <span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/infrastructure/-/work_items/3387117" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/infrastructure/-/work_items/3387117</a></span> though that may require larger storage costs the benefit is that the cache is per project and can be more precisely controlled by project owners. Issue forks would have limited benefit as each fork would have its own cache. </p>
<h4 id="summary-steps-reproduce">Steps to reproduce</h4>
<h3 id="summary-proposed-resolution">Proposed resolution</h3>
<ul>
<li>Provide a caching proxy server for http/https requests</li>
<li>Use Satis to mirror the projects that Drupal Core requires into our own package server</li>
<li>Use <a href="https://github.com/gmta/velocita-proxy">https://github.com/gmta/velocita-proxy</a> as a composer plugin (I'm not sure what advantages this has over a standard caching proxy server)</li>
<li><del>Use a local mount shared between all projects</del> (security concerns)</li>
<li><del>Use GitLab per project local caching </del> (unlikely the same module will run on the same runner in the future)</li>
<li><del>Require every maintainer to obtain an OAUTH login for Github </del>(will not fix merge request pipelines)</li>
</ul>
<h3 id="summary-remaining-tasks">Remaining tasks</h3>
<h3 id="summary-ui-changes">User interface changes</h3>
<h3 id="summary-api-changes">API changes</h3>
<h3 id="summary-data-model-changes">Data model changes</h3>
> Related issue: [Issue #3387117](https://www.drupal.org/node/3387117)
> Related issue: [Issue #3477381](https://www.drupal.org/node/3477381)
issue