[Discuss] Add Website URL input interface for document_loader scraping
>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3573323. --> Reported by: [marcus_johansson](https://www.drupal.org/user/385947) Related to !1 >>> <p>[Tracker]<br> <strong>Update Summary: </strong>[One-line status update for stakeholders]<br> <strong>Short Description: </strong>Add Website URL input interface for document_loader scraping<br> <strong>Check-in Date: </strong>MM/DD/YYYY<br> [/Tracker]</p> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>The <code>document_loader</code> module currently supports various input types for loading and normalizing documents. However, there is no dedicated input interface for Website URL sources intended for scraping.</p> <p>In many AI and automation use cases, content must be loaded directly from a publicly accessible website URL and normalized into text, Markdown, HTML, or other supported output formats. Without a dedicated Website URL input type, implementers must create custom loaders or misuse existing input types, which leads to unclear responsibilities and inconsistent behavior.</p> <p>A proper Website URL input interface would allow document_loader to clearly define scraping behavior, validation rules, and transformation guarantees in a structured and extensible way.</p> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <ul> <li>Add a new input type under <code>src/DocumentLoaderType/Input</code> for Website URL.</li> <li>Define a UrlInputInterface extending <code>DocumentLoaderTypeInterface</code>.</li> </ul> <h3 id="summary-ai-usage">AI usage (if applicable)</h3> <p>[x] AI Assisted Issue<br> This issue was generated with AI assistance, but was reviewed and refined by the creator.</p> <p>[ ] AI Assisted Code<br> This code was mainly generated by a human, with AI autocompleting or parts AI generated, but under full human supervision.</p> <p>[x] AI Generated Code<br> This code was mainly generated by an AI with human guidance, and reviewed, tested, and refined by a human.</p> <p>[ ] Vibe Coded<br> This code was generated by an AI and has only been functionally tested.</p>
issue