首页 > 程序员开发工具 > 搜索引擎蜘蛛IP地址大全

搜索引擎蜘蛛IP地址大全

建议idc公司加入机房防火墙白名单
更新日期: 2019-12-07
交流QQ群:699611280

搜索引擎 ASN数据 iP段 Spider & Robot UserAgent 相关资料
百度 baidu.com AS4808
AS4808
AS4808
AS55967
AS23724
AS23724
AS23724
61.135.128.0/18
123.125.64.0/18
111.206.192.0/18
180.76.0.0/20
180.149.128.0/19
220.181.32.0/19
220.181.96.0/19
Baiduspider 61.135.186.13
123.125.71.138
111.206.221.38
180.76.15.138
180.149.133.38
220.181.32.38
220.181.108.138
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko) Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
Mozilla/5.0 (iPhone;CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko)Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0;Smartapp; +http://www.baidu.com/search/spider.html)
Baiduspider-image+(+http://www.baidu.com/search/spider.htm)
Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;)
搜狗 sogou.com AS4808
AS4808
AS4808
AS23724
AS23724
AS23724
AS9808
AS17623
AS23724
AS4816
61.135.0.0/16
123.126.64.0/18
111.202.64.0/18
106.38.224.0/19
220.181.96.0/19
106.120.160.0/19
111.13.0.0/16
58.250.0.0/16
218.30.96.0/19
183.36.96.0/19
Sogou web spider 61.135.189.138
123.126.113.138
111.202.100.138
106.38.241.37
220.181.125.138
106.120.173.138
111.13.94.45
58.250.125.37
218.30.103.50
183.36.114.102
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25(compatible; Sogou web spider/4.0; +http://www.sogou.com/docs/help/webmasters.htm#07)
360 so.com AS4812
AS4837
180.153.0.0/16
42.224.0.0/12
360Spider https://www.so.com/help/spider_ip.html
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider
神马 sm.cn AS37963
AS37963
AS37963
42.156.192.0/18
42.120.128.0/17
106.11.144.0/20
YisouSpider 42.156.254.138
42.120.236.138
106.11.159.138
Mozilla/5.0 (iPhone; CPU iPhone OS 10_3 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.75 Mobile/14E5239e YisouSpider/5.0 Safari/602.1
必应 bing.com AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
13.64.0.0/11
157.55.0.0/16
207.46.0.0/19
40.64.0.0/10
52.224.0.0/11
52.160.0.0/11
bingbot 13.66.139.138
157.55.39.138
207.46.13.138
40.77.167.138
52.247.2.138
52.160.?.?
https://www.bing.com/toolbox/verify-bingbot-verdict
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 BingPreview/1.0b
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
谷歌 google.com AS24424
AS15169
203.208.32.0/19
66.249.64.0/20
Googlebot 203.208.60.138
66.249.64.138
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36
Mozilla/5.0 (Linux; Android 4.2.1; en-us; Nexus 5 Build/JOP40D) AppleWebKit/535.19 (KHTML, like Gecko; googleweblight) Chrome/38.0.1025.166 Mobile Safari/535.19
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.google.com/bot.html)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36 (compatible; Google-Read-Aloud; +https://support.google.com/webmasters/answer/1061943)
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot-Image/1.0
Googlebot-Video/1.0
今日头条 toutiao.com AS4134
AS4808/AS9808/AS23724
AS4837
111.224.0.0/14
220.243.128.0/20
110.240.0.0/12
Bytespider 111.225.149.138
220.243.135.138
110.249.201.138
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.3072.1769 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2125.1549 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.7282.1240 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2466.1117 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.7004.1867 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.5219.1367 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.1666.1244 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2711.1431 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.7862.1276 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.8502.1519 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.1865.1898 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.1953.1175 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.3062.1821 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.9548.1423 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.5095.1942 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.9321.1429 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.8793.1741 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.1431.1066 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.5735.1944 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.6835.1927 Mobile Safari/537.36; Bytespider;https://zhanzhang.toutiao.com/
其它 AS4812 180.160.0.0/13 360Spider 没做好记录,不知道是上面哪家了
f