首頁 繪圖設計 工作閒聊 比賽活動 美術討論 標籤 圖片
PChome: rude PchomeBot spider




Type(Type) 2013/11/8 14:34

PChome: Very rude PchomeBot spider/robot

Q: Does PCHomebot reads and follows robots.txtA: No, it doesn't care your robots.txt. That is, a rude one.Q: Does PCHomebot provide contact mailbox or contact form in user agent URL? A: No, there is NO contact info at that page. Again, a rude one.=> http://www.pchome.com.tw/pchomebot.htm代碼:
"Mozilla/5.0 (compatible; PChomebot/1.0; http://www.pchome.com.tw/pchomebot.htm) PChomebot 說明PChomebot是PChome為了提供更好的網路服務與購物經驗所開發出的程式。這個程式會根據網頁內文與連結,自動做資料收集與分析,原則上不會影響網站的運作。如果有對您的網站造成影響,請與我們聯絡。
Q: What do you think?A: If the PcHomeBot brings abnormal load to your sites, just ban it.Q: IP address range?A: From Seednet/SPARQ代碼:
220.228.172.*122.147.50.*113.196.127.*123.51.198.*122.147.50.*


Type(Type) 2014/7/14 15:45

PChome Robot 完全不理會 robots.txt

... 亂抓 robots.txt disallow 的連結。以為 YodaoBot(有道) 已經是很誇張的 spider bot沒想到 PCHOME 更是誇張!pchomebot.htm 頁面也沒有任何聯絡資訊!一天抓 300,000+ pages/day ,連續好幾天,令人各種無言。Add this into your .htaccess or conf files
代碼:
       BrowserMatch       "PChomebot"      is_crazy_bot    #    # Ban is_crazy_bot:  PCHOME BOT    #    <FilesMatch "^(.*)$">          Order Allow,Deny          Allow from all          Deny from env=is_crazy_bot    </FilesMatch>


Type(Type) 2014/7/14 16:03




These pchomebot are from SPARQnet & SPARQ.com.tw

代碼:
[07:50] > grep 'pchomebot.htm)' /var/log/apache2/access.log | awk '{print $1}' > /tmp/pchomebot.txt[07:52] > grep 'pchomebot.htm)' /var/log/apache2/access.log.1 | awk '{print $1}' >> /tmp/pchomebot.txt[07:54] > sort /tmp/pchomebot.txt | uniq
113.196.127.66113.196.127.67113.196.127.68113.196.127.69113.196.127.70113.196.127.71113.196.127.72113.196.127.73113.196.127.74113.196.127.75113.196.127.76113.196.127.77113.196.127.78113.196.127.79113.196.127.80113.196.127.81113.196.127.82122.147.50.17122.147.50.18122.147.50.19122.147.50.20122.147.50.21122.147.50.22122.147.50.23122.147.50.24122.147.50.25122.147.50.27122.147.50.28122.147.50.29122.147.50.30123.51.198.66123.51.198.67123.51.198.68123.51.198.69123.51.198.70123.51.198.71123.51.198.72123.51.198.73123.51.198.74123.51.198.75123.51.198.76123.51.198.77123.51.198.78123.51.198.79123.51.198.80123.51.198.82220.228.172.130220.228.172.131220.228.172.132220.228.172.133220.228.172.134220.228.172.135220.228.172.138220.228.172.139220.228.172.140220.228.172.141220.228.172.143220.228.172.144220.228.172.145220.228.172.146


(3,177 views)
[更多討論] 討論區 Windows, Linux, Perl, PHP, C/C++, Driver, Web 理論、應用、硬體、軟體


"PChome: rude PchomeBot spider" 傳統頁面(電腦版)

首頁 繪圖設計 工作閒聊 比賽活動 美術討論 標籤 圖片
傳統桌面版
© Vovo2000.com Mobile Version 小哈手機版 2019