Resolve #3463286 "Browsertestbase race condition"
Closes #3463286
Merge request reports
Activity
added 1 commit
- 3ef3e203 - Increase the maximum sessions available and reduce timeouts and retry times.
added 1 commit
- 245ca6e1 - Stop mink sessions as early as possible after a test run.
92 92 variables: 93 93 JAVA_OPTS: "-Dwebdriver.chrome.logfile=/builds/chromedriver.log" 94 94 SE_NODE_OVERRIDE_MAX_SESSIONS: "true" 95 SE_NODE_MAX_SESSIONS: "5" 95 SE_NODE_MAX_SESSIONS: "24" - Comment on lines -95 to +95
changed this line in version 4 of the diff
136 138 137 139 '🖱️️️ PHPUnit Functional Javascript': 138 140 <<: [ *with-composer, *run-tests, *default-job-settings ] 139 parallel: 4 141 parallel: 2 140 142 variables: 141 143 TESTSUITE: PHPUnit-FunctionalJavascript 142 CONCURRENCY: 15 144 CONCURRENCY: 8 - Comment on lines -142 to +144
Probably not, but I tried 8 concurrency and 10 sessions and similar numbers like that, and got at least one instance of the race condition/stampede, so it needs to be a comfortably high number. Selenium recommends a max of 1 per CPU for stability - it's a maximum, they're only created on demand, so if the demand never goes above say 12 it doesn't matter if the maximum is higher, we just won't get there.
My feeling is that 16 is probably plenty since that's enough for 8 tests/methods to finish and 8 tests to start all at exactly the same time.
Oh also the main reason to reduce the concurrency so much here is because there's no benefit in job times putting it higher, when we run all the tests in two jobs - it's basically the same with 8 or 16 concurrency. With 16, all the tests run at the beginning of the job leaving the long running tests on their own at the end. With 8, the shorter tests are just finishing off as the longer running tests finish (at least when this is combined with the other issue).
However lowering to 8 means we maximise the headroom of chrome sessions.
If we can get functional tests to run under three minutes (which seems possible after the other issue), we could add one extra functional js job to make three, and I think that'll be all the test types finishing under 3 minutes then. But no point doing that until it's an overall improvement in pipeline times.