>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #1376398. --> Reported by: [cpliakas](https://www.drupal.org/user/266779) >>> <p>The default robots.txt file provided by Drupal disallows crawling of search/* paths. However, the current 7.x versions of advanced search modules such as <a href="http://drupal.org/project/apachesolr" title="Apache Solr Search Integration project page">Apache Solr Search Integration</a> and <a href="http://drupal.org/project/search_api" title="Search API project page">Search API</a> enable site builders to create search pages via the admin interface which may (and usually do) exist outside of the standard search/* path. This causes SEO headaches when faceted navigation is used because the crawlers have a tendency to get stuck in loops and index loads of duplicate data. The conversation that brought this information to light is posted at <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/facetapi/issues/1370342" title="Status: Closed (fixed)">#1370342: Implement a setting to add "rel=nofollow" to facet links</a></span>. The following articles provide some supporting evidence:</p> <ul> <li><a href="http://www.localseoguide.com/faceted-search-seo-death/" target="_blank">Faceted Search = SEO Death</a></li> <li><a href="http://searchnewscentral.com/20110601167/General-SEO/solving-duplicate-content-issues-arising-from-faceted-navigation.html" target="_blank">Solving Duplicate Content Issues Arising From Faceted Navigation</a></li> <li><a href="http://www.netconcepts.com/faceted-navigation-article/" target="_blank">Is Your Retail Site Navigation Spamming Google?</a></li> <li><a href="http://www.netconcepts.com/learn/Faceted-Navigation-Natural-Search-Spam.pdf" target="_blank">The New Natural Search Spam: Faceted Navigation</a></li> </ul> <p>The approach taken by <a href="http://drupal.org/project/facetapi" title="Facet API project page">Facet API</a> is to provide site builders with a checkbox option to add the <code>rel="nofollow"</code> attribute to facet links. The downside is that faceted navigation could be used for a site's IA in some instances, so this would prevent valuable content from being crawled. I wanted to check with the SEO experts as to whether or not this is the correct approach, and if so whether it would make sense to add an item to the checklist instructing people to make sure this setting is enabled.</p> <p>Other issues on this topic posted against other projects are listed below for reference:<br> <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-1"><a href="https://www.drupal.org/project/faceted_search/issues/197783" title="Status: Active">#197783: Module makes database balloon in size - avoid logging the guided searches</a></span> (Faceted Search)<br> <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-5"><a href="https://www.drupal.org/project/apachesolr/issues/371542" title="Status: Closed (won't fix)">#371542: Ponder facets acting as link spam, add rel=nofollow?</a></span> (Apache Solr Search Integration)<br> <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/facetapi/issues/1370342" title="Status: Closed (fixed)">#1370342: Implement a setting to add "rel=nofollow" to facet links</a></span> (Facet API)</p>