blokuje a i tak se biegaja...

[SilverT]

nie wiem może coś spierdzieliłem, ale poblokowałem boty w .htaccess a i tak biegają sobie po mojej stronie

tak wygląda htaccess mojego seoka:

RewriteEngine On
RewriteBase /

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_URI} !(.*)/$
RewriteRule ^(.*)$ https://%{HTTP_HOST}/$1/ [L,R=301]
RewriteRule ^([-_!~,;*'()$a-zA-Z0-9]+),k,([0-9]+)/ index.php?action=cat&id=$2 [L]
RewriteRule ^([-_!~,;*'()$a-zA-Z0-9]+)/([-_!~,;*'()$a-zA-Z0-9]+),p,([0-9]+)/strona,([0-9]+)/ index.php?action=sub&id=$3&page=$4 [L]
RewriteRule ^([-_!~,;*'()$a-zA-Z0-9]+)/([-_!~,;*'()$a-zA-Z0-9]+),p,([0-9]+)/ index.php?action=sub&id=$3&page=1 [L]
RewriteRule ^([-_!~,;*'()$a-zA-Z0-9]+)/([-_!~,;*'()$a-zA-Z0-9]+),s,([0-9]+)/ index.php?action=site&id=$3 [L]
RewriteRule ^szukaj,(.+)/strona,([0-9]+)/ index.php?action=search&search=$1&page=$2 [L]
RewriteRule ^szukaj,(.+)/ index.php?action=search&search=$1&page=1 [L]
RewriteRule ^tag,(.+)/strona,([0-9]+)/ index.php?action=tag&tag=$1&page=$2 [L]
RewriteRule ^tag,(.+)/ index.php?action=tag&tag=$1&page=1 [L]
RewriteRule ^pr,([0-9]+)/strona,([0-9]+)/ index.php?action=pr&pr=$1&page=$2 [L]
RewriteRule ^pr,([0-9]+)/ index.php?action=pr&pr=$1&page=1 [L]
RewriteRule ^rules/ index.php?action=rules [L]
RewriteRule ^help/ index.php?action=info [L]
RewriteRule ^new,listings/ index.php?action=new [L]
RewriteRule ^hot,listings/ index.php?action=pop [L]
RewriteRule ^top,rated/ index.php?action=top [L]
RewriteRule ^friends/ index.php?action=friends [L]
RewriteRule ^map,of,directory/ index.php?action=map [L]
RewriteRule ^contact/ index.php?action=contact [L]
RewriteRule ^add,site/ index.php?action=global_add [L]
RewriteRule ^regulamin/ index.php?action=rules [L]
RewriteRule ^jak,dodac,wpis/ index.php?action=info [L]
RewriteRule ^najnowsze/ index.php?action=new [L]
RewriteRule ^popularne/ index.php?action=pop [L]
RewriteRule ^najlepsze/ index.php?action=top [L]
RewriteRule ^przyjaciele/ index.php?action=friends [L]
RewriteRule ^mapa,katalogu/ index.php?action=map [L]
RewriteRule ^kontakt/ index.php?action=contact [L]
RewriteRule ^dodaj,strone/ index.php?action=global_add [L]



SetEnvIfNoCase User-Agent "^ExtractorPro" bad_bot
SetEnvIfNoCase User-Agent "^CherryPicker" bad_bot
SetEnvIfNoCase User-Agent "^NICErsPRO" bad_bot
SetEnvIfNoCase User-Agent "^Teleport" bad_bot
SetEnvIfNoCase User-Agent "^EmailCollector" bad_bot
SetEnvIfNoCase User-Agent "^LinkWalker" bad_bot
SetEnvIfNoCase User-Agent "^Zeus" bad_bot
SetEnvIfNoCase User-Agent "^deltaSCAN" bad_bot
SetEnvIfNoCase User-Agent "^IRLbot" bad_bot
SetEnvIfNoCase User-Agent "^e-SocietyRobot" bad_bot
SetEnvIfNoCase User-Agent "^Accoona-AI-Agent" bad_bot
SetEnvIfNoCase User-Agent "^MJ12bot" bad_bot
SetEnvIfNoCase User-Agent "^TMCrawler" bad_bot
SetEnvIfNoCase User-Agent "^Twiceler" bad_bot
SetEnvIfNoCase User-Agent .*VoilaBot BETA 1.2.* bad_bot
SetEnvIfNoCase User-Agent .*libwww-perl/5.805.* bad_bot
SetEnvIfNoCase User-Agent .*Java/1.5.0_11.* bad_bot
SetEnvIfNoCase User-Agent .*Sogou web spider/3.0.* bad_bot
SetEnvIfNoCase User-Agent .*psbot.* bad_bot
SetEnvIfNoCase User-Agent .*Exabot.* bad_bot
SetEnvIfNoCase User-Agent .*Charlotte/1.0b.* bad_bot
SetEnvIfNoCase User-Agent .*boitho.com-dc.* bad_bot
SetEnvIfNoCase User-Agent .*ajSitemap.* bad_bot
SetEnvIfNoCase User-Agent .*bot/1.0.* bad_bot
SetEnvIfNoCase User-Agent .*panscient.com.* bad_bot
SetEnvIfNoCase User-Agent .*Java/1.6.0_11.* bad_bot
SetEnvIfNoCase User-Agent .*WebDataCentreBot/1.0.* bad_bot
SetEnvIfNoCase User-Agent .*Java.* bad_bot
SetEnvIfNoCase User-Agent .*SapphireWebCrawler.* bad_bot
SetEnvIfNoCase User-Agent .*Yandex.* bad_bot
SetEnvIfNoCase User-Agent .*Baiduspider.* bad_bot
SetEnvIfNoCase User-Agent .*Rankivabot.* bad_bot
SetEnvIfNoCase User-Agent .*DBLBot/1.0.* bad_bot
SetEnvIfNoCase User-Agent "^EmailSiphon" bad_bot
SetEnvIfNoCase User-Agent "^EmailWolf" bad_bot

order allow,deny
deny from env=bad_bot
allow from all

order allow,deny
deny from	109.196.1.115
deny from	109.239.48.34
deny from	173.224.125.91
deny from	178.235.246.173
deny from	184.72.214.30
deny from	188.137.100.94
deny from	188.143.232.36
deny from	188.143.232.54
deny from	196.207.208.8
deny from	208.90.239.210
deny from	212.113.37.106
deny from	213.229.72.62
deny from	41.214.112.207
deny from	41.214.120.182
deny from	50.17.21.202
deny from	50.30.32.248
deny from	66.219.58.45
deny from	66.249.66.140
deny from	66.249.72.239
deny from	66.249.82.1
deny from	72.229.139.192
deny from	74.125.16.3
deny from	79.139.97.33
deny from	79.143.182.232
deny from	83.238.48.22
deny from	83.3.130.90
deny from	89.161.141.247
deny from	89.161.186.176
deny from	89.25.228.254
deny from	90.156.127.111
deny from	94.23.242.29
deny from	94.23.89.25
deny from	94.246.128.25
deny from	94.75.224.107
deny from	94.76.208.5
deny from	95.211.0.141
deny from	95.64.12.20
deny from 66.249.72.239
deny from	209.249.53.32
deny from	83.11.155.152
deny from	109.231.7.57
deny from	85.232.225.130
deny from	83.5.234.95
deny from	95.50.136.242
deny from	109.173.200.57
deny from	95.108.154.251
deny from	95.50.169.58
deny from	91.209.51.176
allow from all

a tutaj z logów:

119.63.196.61 - - [17/Sep/2011:03:40:14 +0200] "GET /ja/ajaxstarrater/db.php?j=4&q=1044&t=209.85.228.80&c=10 HTTP/1.1" 403 574 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +https://www.baidu.com/search/spider.html)"

94.23.42.135 - - [17/Sep/2011:10:04:55 +0200] "GET /turystyka,i,rekreacja/noclegi,duszniki,zdroj,s,4956 HTTP/1.0" 301 628 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.4.0; https://www.majestic12.co.uk/bot.php?+)"

95.108.154.251 - - [17/Sep/2011:03:42:10 +0200] "GET /mk/firmy/waterjet,ciecie,woda,s,2875/ HTTP/1.1" 403 625 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +https://yandex.com/bots)"

[niktorek]

Blokowanie za pomoca SetEnvIfNoCase nie na każdym serwerze działa, widocznie brak na nim odpowiedniego moda. Jeśli nie możesz go zainstalować to możesz blokować boty też przez RewriteCond, u mnie się tak sprawdza.

[SilverT]

ok, pogadam z osobą, która wszystko instalowała, ale problem w tym, że nawet ip jest blokowane a też nie działa

[zgred]

wchodzić wchodza ale zaimplementowane rulezy dają error404 takim botom.

[SilverT]

dają nie dają... yandex kładzie mi serwer nawet kilka razy dziennie :/

[zgred]

jeśli to dedyk proponuję dodać do httpd.conf te robociki do blokowania (na końcu pliku):

# Bots blocked

SetEnvIfNoCase User-Agent "DotBot" bad_bot
#SetEnvIfNoCase User-Agent "MJ12bot" bad_bot
SetEnvIfNoCase User-Agent "sitebot" bad_bot
SetEnvIfNoCase User-Agent "LexxeBot" bad_bot
SetEnvIfNoCase User-Agent "Yandex" bad_bot
SetEnvIfNoCase User-Agent "Ezooms" bad_bot
SetEnvIfNoCase User-Agent "ichiro" bad_bot
SetEnvIfNoCase User-Agent "AppEngine-Google" bad_bot

<Location />
Order Allow,Deny
Deny from env=bad_bot
Allow from all
</Location>

w ten sposób u mnie to działa dla całego serwerka. Po zapisaniu oczywiście restarta "apacza"