Loading
Issue #3629151: Add a denylist for crawling external URLs
Added the host_denylist configuration option to the External freelinking plugin:
- Added
host_denylisttextarea to the External plugin configuration form, plugin attribute, and default configuration. - Added host pattern matching in
shouldCrawlUrl()/isHostDenied():- Supports exact hostnames and wildcard domain patterns (e.g.
*.example.com). - Supports host:port patterns and wildcard ports (e.g.
*:8080,example.com:8080). - Automatically handles user-entered protocol prefixes, trailing slashes, comments (
#), and blank lines. - Retains existing default protections against crawling localhost and private/reserved IP ranges.
- Supports exact hostnames and wildcard domain patterns (e.g.
- Added configuration schema in
freelinking.schema.yml. - Added post-update hook
freelinking_post_update_external_host_denylistto initialize existing filter format configurations. - Added unit tests in
ExternalHostTestand updatedExternalTest.
Closes #3629151
Edited by Shamir vs