IP被封已经三天了,刚刚用代理上后台查看Latest Visitors ,发现GG和YAHOO还在抓,呵呵,百度就不见踪影了,555555.
看来百度太本土化了,国外的IP一封,百度也没办法抓取了.
cPanel可以看到蜘蛛的痕迹,不错的功能:
Http Code: 200 Date: Jun 26 01:33:11 Http Version: HTTP/1.1 Size in Bytes: 2866
Referer: -
Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +
http://www.google.com/bot.html)
Http Code: 200 Date: Jun 26 00:46:04 Http Version: HTTP/1.0 Size in Bytes: 218
Referer: -
Agent: Mozilla/5.0 (compatible; Yahoo! Slurp;
http://help.yahoo.com/help/us/ysearch/slurp)