Skip to content
Snippets Groups Projects
Commit 2cf618e1 authored by Grazyna Jaworska's avatar Grazyna Jaworska
Browse files

Add Nutch to the list of blocked crawlers in Nginx.

parent db25410a
No related branches found
No related tags found
No related merge requests found
Show whitespace changes
Inline Side-by-side
http/nginx/server.tpl.php
+ 2
− 2
View file @ 2cf618e1
...@@ -152,7 +152,7 @@ map $request_uri $key_uri { ...@@ -152,7 +152,7 @@ map $request_uri $key_uri {
### ###
map $http_user_agent $is_crawler { map $http_user_agent $is_crawler {
default ''; default '';
~*HTTrack|MJ12bot|HTMLParser|libwww|PECL|AutomaticSiteMap|ClickSense|ValueClick|SiteBot|BuzzTracker|sistrix|Offline|Screaming is_crawler; ~*HTTrack|MJ12bot|HTMLParser|libwww|PECL|AutomaticSiteMap|ClickSense|ValueClick|SiteBot|BuzzTracker|sistrix|Offline|Screaming|Nutch is_crawler;
} }
### ###
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment