Execute CSPELL in project root folder and make fixing words easier (#3426136) · Issues · project / gitlab_templates

Execute CSPELL in project root folder and make fixing words easier

>>> [!note] Migrated issue   Reported by: [jonathan1055](https://www.drupal.org/user/92645) Related to !155 >>> <h3 id="summary-problem-motivation">Problem/Motivation</h3> Due to the CSPELL work-around for copying the projects files into $CI_PROJECT_DIR/gitlabci_cspell/$CI_PROJECT_NAME the paths in the downloadable _cspell_json.txt are not valid any more. <h4 id="summary-steps-reproduce">Steps to reproduce</h4> Download _cspell_json.txt and save it as .cspell.json to form the basis for the project's own file. The cspell job then fails with <pre>Dictionary Error with (drupal) Error: ENOENT: no such file or directory, open '/builds/project/core/misc/cspell/drupal-dictionary.txt' Dictionary Error with (dictionary) Error: ENOENT: no such file or directory, open '/builds/project/core/misc/cspell/dictionary.txt' CSpell: Files checked: 0, Issues found: 0 in 0 files</pre><h3 id="summary-proposed-resolution">Proposed resolution</h3> This MR also contains several other improvements to make the cspell job easier for maintainers to fix and run green. Here is a summary of the changes: <ol> <li>cspell is executed directly in the project root $CI_PROJECT_DIR instead of the copied folder $CI_PROJECT_DIR/gitlabci_cspell/$CI_PROJECT_NAME</li> <li>This eliminates the need to copy everything, so that folder is no longer created</li> <li>/webroot and /vendor are automatically ignored</li> <li>Manipulation of .cspell.json (either the projects own, or the default from /assets) is done in a new script prepare-cspell.php not the inline bash in .gitlab-ci.yml job step. This gives more flexibility and imporves readability</li> <li>The two drupal core dictionaries are always added, and that the paths to them are always corrected, regardless of what the maintainer might need for other running of cspell, locally or pre-existing on DrupalCI</li> <li>The input variables _cspell_words and _cspell_ignore_paths no longer require the tedious quoting of each item. This is all done in the php script. This reduces maintauiner effort in support</li> <li>A new variable _cspell_ignore_standard_files allows flexibility for ignoring the common list (composer.json, maintainers.text, etc). The default value is 1, so that these are ignored unless the maintainer specifically sets the value to 0</li> <li>Additional internal dictionaries (already part of cspell) are used to save maintainers adding some extra words</li> </ol> Here is the <a href="https://git.drupalcode.org/issue/gitlab_templates-3426136/-/blob/3426136-cspell-in-root-folder/docs/jobs/cspell.md" title="3426136-cspell-in-root-folder/docs/jobs/cspell.md">latest updated doc page for these changes</a> To test this MR use the following in your .gitlab-ci.yml <pre>include:   # MR 155 Run cspell in root folder   - project: 'issue/gitlab_templates-3426136'     ref: 3426136-cspell-in-root-folder variables:   _CURL_TEMPLATES_REPO: issue/gitlab_templates-3426136   _CURL_TEMPLATES_REF: 3426136-cspell-in-root-folder</pre><h3 id="summary-testing">Testing</h3> Tetsing needs to cover the following scenarios: <ol> <li>Project without any cspell related files or variables. Project does have composer.json Search API Totally un-customised. No .cspell-project-words.txt, no variables, no .cspell.json. The project does have a composer.json file Issue <a href="https://www.drupal.org/project/search_api/issues/3427068" title="Status: Closed (fixed)">#3427068: Fix cspell errors in Search API</a> MR <a href="https://git.drupalcode.org/project/search_api/-/merge_requests/116">https://git.drupalcode.org/project/search_api/-/merge_requests/116</a> Test <a href="https://git.drupalcode.org/issue/search_api-3427068/-/pipelines/120292">https://git.drupalcode.org/issue/search_api-3427068/-/pipelines/120292</a> Decoupled Pages Does not have .cspell.json, no .cpell-project-words.txt, no _CSPELL variables. There is currently one unrecognised word in composer.json, but with this MR the cspell job is green. MR <a href="https://git.drupalcode.org/project/decoupled_pages/-/merge_requests/6">https://git.drupalcode.org/project/decoupled_pages/-/merge_requests/6</a> Existing <a href="https://git.drupalcode.org/project/decoupled_pages/-/pipelines/121257">https://git.drupalcode.org/project/decoupled_pages/-/pipelines/121257</a> Test <a href="https://git.drupalcode.org/issue/decoupled_pages-3422594/-/pipelines/121458">https://git.drupalcode.org/issue/decoupled_pages-3422594/-/pipelines/121458</a> </li> <li>Project without any cspell variables, no .cspell.json. Project does not have composer.json CDN Issue <a href="https://www.drupal.org/project/cdn/issues/3347168" title="Status: Closed (fixed)">#3347168: Match core's code quality checks: cspell</a> <a href="https://git.drupalcode.org/project/cdn/-/merge_requests/18">https://git.drupalcode.org/project/cdn/-/merge_requests/18</a> <a href="https://git.drupalcode.org/issue/cdn-3347168/-/jobs/1076281">https://git.drupalcode.org/issue/cdn-3347168/-/jobs/1076281</a> </li> <li>Project with cspell variables Module Filter <a href="https://www.drupal.org/project/module_filter/issues/3431234" title="Status: Closed (fixed)">#3431234: Fix spelling and unrecognized words</a> <a href="https://git.drupalcode.org/project/module_filter/-/merge_requests/45">https://git.drupalcode.org/project/module_filter/-/merge_requests/45</a> The project does not have any .cspell.json or .cspell-project-words.txt Fixed entirely with additions to <code>_CSPELL_WORDS</code>. The sequence of commits and pipelines demonstrates that the new MR caters for unquoted words in the list, which makes it easier for maintainers. Able Player <a href="https://www.drupal.org/project/ableplayer/issues/3427424" title="Status: Closed (fixed)">#3427424: Resolve cspell errors found by gitlab pipeline</a> <a href="https://git.drupalcode.org/project/ableplayer/-/merge_requests/37">https://git.drupalcode.org/project/ableplayer/-/merge_requests/37</a> Testing shows that existing <code>_CSPELL_WORDS</code> and <code>_CSPELL_IGNORE_PATHS</code> work with the new changes. It also shows simplification and reduction in work. See the linked issue for details. </li> <li>Project with existing .cspell-project-words.txt file Open Telemetry Project has existing .cspell-project-words.txt and _CSPELL_IGNORE_PATHS and the cspell job currently passes green. Issue <a href="https://www.drupal.org/project/opentelemetry/issues/3425641">https://www.drupal.org/project/opentelemetry/issues/3425641</a> switch to MR155 - <a href="https://git.drupalcode.org/issue/opentelemetry-3425641/-/jobs/1109939">cspell job still green</a> remove variable _CSPELL_IGNORE_PATHS: "\"composer.json\"" - <a href="https://git.drupalcode.org/issue/opentelemetry-3425641/-/jobs/1110158">job remains passing green</a></li> <li>Project with existing .cspell.json file Automatic Updates This project contains a dictionary.txt file and was running CSPELL on DrupalCI before Gitlab Templates had it. Issue <a href="https://www.drupal.org/project/automatic_updates/issues/3426716" title="Status: Closed (outdated)">#3426716: gitlab CI phpunit tests in 3.0.x are now failing</a> MR <a href="https://git.drupalcode.org/project/automatic_updates/-/merge_requests/1030">1030</a> The current <a href="https://git.drupalcode.org/issue/automatic_updates-3426716/-/pipelines/124266">CSPELL job fails</a> With MR155 <a href="https://git.drupalcode.org/issue/automatic_updates-3426716/-/pipelines/124282">it passes</a> with no other changes </li> <li>Project with .cspell.json file as provided in artifacts when no file is present Scheduler First run on main branch (not new MR), with no project .cspell.json, to get a cspell_json.txt artifact that projects may have now <a href="https://git.drupalcode.org/project/scheduler/-/jobs/1120118">https://git.drupalcode.org/project/scheduler/-/jobs/1120118</a> The unrecognised words are in <code>copyright.txt</code> and <code>License.md</code>. Downloaded the cspelll_json.txt artifact, saved as .cspell.json with no changes, uploaded and run using MR155 <a href="https://git.drupalcode.org/project/scheduler/-/pipelines/125105">https://git.drupalcode.org/project/scheduler/-/pipelines/125105</a> The <code>copyright.txt</code> and <code>License.md</code> files are now ignored as expected. Only two unrecognised words - intentionally added to test _CSPELL_WORDS. Add two words to _CSPELL_WORDS - job ends green <a href="https://git.drupalcode.org/project/scheduler/-/pipelines/125164">https://git.drupalcode.org/project/scheduler/-/pipelines/125164</a></li> <li>Project using the new _CSPELL_IGNORE_STANDARD_FILES variable set to 0 to not those files. Scheduler As above, but with <code>_CSPELL_IGNORE_STANDARD_FILES=0</code> <a href="https://git.drupalcode.org/project/scheduler/-/pipelines/125204">https://git.drupalcode.org/project/scheduler/-/pipelines/125204</a> the jobs ends amber. You can see <a href="https://git.drupalcode.org/project/scheduler/-/jobs/1121185#L259">in the log here</a> that cspell_json['ignorePaths'] only contains 'vendor' 'web' 'node_modules' and '.git'. The unrecognised words are now reported in 4 files: composer.json, COPYRIGHT.txt, License.md and src/copyright.txt</li> </ol> > Related issue: [Issue #3425403](https://www.drupal.org/node/3425403)

issue