| 人氣點閱:8206 |
發表人 |
|
|
Type
› 繪圖畫廊 › 設計藝廊
› 攝影相簿 › 留言板
› 最愛收藏 › 分類標籤
暱稱: Type
註冊: 2002-11-30
發表: 11173
來自: vovo2000.com
Ｖ幣: 901761
|
|
|
| Re: PChome: rude PchomeBot spider 2014-07-14 15:45 |
/ / /
|
|
PChome Robot 完全不理會 robots.txt
... 亂抓 robots.txt disallow 的連結。
以為 YodaoBot(有道) 已經是很誇張的 spider bot
沒想到 PCHOME 更是誇張！
pchomebot.htm 頁面也沒有任何聯絡資訊！
一天抓 300,000+ pages/day ，連續好幾天，令人各種無言。
Add this into your .htaccess or conf files
代碼:
BrowserMatch "PChomebot" is_crazy_bot
#
# Ban is_crazy_bot: PCHOME BOT
#
<FilesMatch "^(.*)$">
Order Allow,Deny
Allow from all
Deny from env=is_crazy_bot
</FilesMatch>
________________
美術插畫設計案子報價系統 v0.1 Beta
爪哇禾雀
|
|
Type
› 繪圖畫廊 › 設計藝廊
› 攝影相簿 › 留言板
› 最愛收藏 › 分類標籤
暱稱: Type
註冊: 2002-11-30
發表: 11173
來自: vovo2000.com
Ｖ幣: 901761
|
|
|
| Re: PChome: rude PchomeBot spider 2014-07-14 16:03 |
/ / /
|
|
These pchomebot are from SPARQnet & SPARQ.com.tw
代碼:
[07:50] > grep 'pchomebot.htm)' /var/log/apache2/access.log | awk '{print $1}' > /tmp/pchomebot.txt
[07:52] > grep 'pchomebot.htm)' /var/log/apache2/access.log.1 | awk '{print $1}' >> /tmp/pchomebot.txt
[07:54] > sort /tmp/pchomebot.txt | uniq
113.196.127.66
113.196.127.67
113.196.127.68
113.196.127.69
113.196.127.70
113.196.127.71
113.196.127.72
113.196.127.73
113.196.127.74
113.196.127.75
113.196.127.76
113.196.127.77
113.196.127.78
113.196.127.79
113.196.127.80
113.196.127.81
113.196.127.82
122.147.50.17
122.147.50.18
122.147.50.19
122.147.50.20
122.147.50.21
122.147.50.22
122.147.50.23
122.147.50.24
122.147.50.25
122.147.50.27
122.147.50.28
122.147.50.29
122.147.50.30
123.51.198.66
123.51.198.67
123.51.198.68
123.51.198.69
123.51.198.70
123.51.198.71
123.51.198.72
123.51.198.73
123.51.198.74
123.51.198.75
123.51.198.76
123.51.198.77
123.51.198.78
123.51.198.79
123.51.198.80
123.51.198.82
220.228.172.130
220.228.172.131
220.228.172.132
220.228.172.133
220.228.172.134
220.228.172.135
220.228.172.138
220.228.172.139
220.228.172.140
220.228.172.141
220.228.172.143
220.228.172.144
220.228.172.145
220.228.172.146
________________
美術插畫設計案子報價系統 v0.1 Beta
爪哇禾雀
|
|
Type
› 繪圖畫廊 › 設計藝廊
› 攝影相簿 › 留言板
› 最愛收藏 › 分類標籤
暱稱: Type
註冊: 2002-11-30
發表: 11173
來自: vovo2000.com
Ｖ幣: 901761
|
|
|
|
|
|
|
|
|
|
