>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3592070. --> Reported by: [mxr576](https://www.drupal.org/user/315522) >>> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p> <span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/infrastructure/-/work_items/3557316" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/infrastructure/-/work_items/3557316</a></span> was closed as fixed, but Perplexity AI assistants are still blocked from accessing Drupal.org issue pages. The PerimeterX (now HUMAN Security) bot protection continues to deny access to requests made via Perplexity, including both its autonomous crawlers and user-triggered fetches. </p> <p> In drumm's closing comment on <span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/infrastructure/-/work_items/3557316" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/infrastructure/-/work_items/3557316</a></span>, the issue was considered "won't fix" on the grounds that Perplexity uses "stealth crawling" and that useful information for troubleshooting and potential allowlisting was not provided. This follow-up issue aims to address both concerns directly. </p> <p> The distinction between crawlers is important here. Perplexity publishes two separate user agents with documented purposes: </p> <ul> <li><strong>PerplexityBot</strong> - autonomous background crawling to index content for search results. Full user-agent string: <code>Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)</code></li> <li><strong>Perplexity-User</strong> - triggered only when a real user explicitly asks Perplexity a question that requires fetching a page. Full user-agent string: <code>Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)</code>. Per Perplexity's own documentation, this agent generally ignores robots.txt because it is acting on behalf of a real user request.</li> </ul> <p> Even if blocking <code>PerplexityBot</code> is considered acceptable, blocking <code>Perplexity-User</code> means that real Drupal developers and community members who use Perplexity as their AI assistant cannot retrieve or discuss Drupal.org issue content through it. Given how widely Perplexity is used by developers - including via its Computer Use / agentic features and frontier model integrations - this is a practical barrier to contribution and community participation. </p> <p> Other major AI platforms (ChatGPT, Gemini) can access Drupal.org issue pages successfully, as demonstrated in the original issue. The inconsistency suggests this is a configuration gap rather than an intentional policy decision. </p> <h4 id="summary-steps-reproduce">Steps to reproduce</h4> <p>See the attached PDF and also previous debugging attempts in <span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/infrastructure/-/work_items/3557316" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/infrastructure/-/work_items/3557316</a></span>.</p> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <p> Allowlist all requests originating form Perplexity or at least those that are user initiated and identified as <code>Perplexity-User</code> user agent (and optionally <code>PerplexityBot</code>) in the PerimeterX / HUMAN Security WAF configuration. Perplexity publishes verified IP ranges that can be used to combine user-agent matching with IP-based verification for a more secure allowlist rule: </p> <h3 id="summary-remaining-tasks">Remaining tasks</h3> <p>As a follow up on <span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/infrastructure/-/work_items/3557316" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/infrastructure/-/work_items/3557316</a></span>: </p> <p>Drupal Infra team should provide explicit debugging steps to help with providing the necessary information for solving this task. </p> <p>OR</p> <p>Drupal Infrastructure team could reach out to Perplexity support directly to coordinate an allowlisting arrangement, which may be the most reliable path forward.</p> <h3 id="summary-ui-changes">User interface changes</h3> <h3 id="summary-api-changes">API changes</h3> <h3 id="summary-data-model-changes">Data model changes</h3>