Robots, Bots, Crawler und deren IP-Adressen

Folgende Webcrawler beachten (mehr oder weniger) die Einträge in der robots.txt:

UA-String IPs
AppEngine-Google; (+http://code.google.com/appengine) 64.233.172.6
Baiduspider+(+http://www.baidu.com/search/spider.htm) 119.63.198.10
119.63.198.13 bis 119.63.198.14
119.63.198.19
119.63.198.26 bis 119.63.198.27
119.63.198.33 bis 119.63.198.34
119.63.198.37
119.63.198.39
119.63.198.41 bis 119.63.198.42
119.63.198.47
119.63.198.49
119.63.198.5
119.63.198.51 bis 119.63.198.52
119.63.198.54 bis 119.63.198.55
119.63.198.57 bis 119.63.198.58
119.63.198.8 bis 119.63.198.9
121.14.89.166
123.125.64.12 bis 123.125.64.13
123.125.66.100 bis 123.125.66.104
123.125.66.106 bis 123.125.66.116
123.125.66.128
123.125.66.15 bis 123.125.66.31
123.125.66.33 bis 123.125.66.38
123.125.66.40 bis 123.125.66.43
123.125.66.45 bis 123.125.66.50
123.125.66.52 bis 123.125.66.55
123.125.66.99
220.181.7.100 bis 220.181.7.102
220.181.7.104
220.181.7.106 bis 220.181.7.114
220.181.7.117
220.181.7.13 bis 220.181.7.29
220.181.7.32 bis 220.181.7.38
220.181.7.40
220.181.7.42 bis 220.181.7.54
220.181.7.95
220.181.7.97 bis 220.181.7.99
61.135.168.170
61.135.168.173
Baiduspider+(+http://www.baidu.jp/spider/) 119.63.193.129
CatchBot/1.0; +http://www.catchbot.com 139.130.192.34
203.6.203.75
CazoodleBot/0.0.2 (http://www.cazoodle.com/contact.php; cbot@cazoodle.com) 72.36.115.86
Cityreview Robot (+http://www.cityreview.org/crawler/) 78.46.68.102
CiZilla/Nutch-1.0-dev (CiZilla.com - Complete Internet search engine; http://www.cizilla.com; crawl@cizilla.com) 75.149.51.89
ConveraCrawler/0.9e (+http://ews.converasearch.com/crawl.htm) 38.96.10.10
copyright sheriff (+http://www.copyrightsheriff.com/) 58.147.52.113
85.181.234.164
DataparkSearch/4.48 (+http://dataparksearch.org/bot) 201.231.63.57
DataparkSearch/4.50 (+http://dataparksearch.org/bot) 79.143.115.178
DataparkSearch/4.51-10082008 (+http://dataparksearch.org/bot) 69.50.220.4
DealGates Bot/1.1 by Luc Michalski (http://spider.dealgates.com/bot.html) 81.220.43.28
DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html) 72.14.199.45
DoCoMo/2.0 P900i(c100;TB;W24H11) 203.131.250.103
DotBot/1.0.1 (http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org) 208.115.111.242
208.115.111.248
DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html) 72.94.249.36
Eurobot/1.0 (http://eurobot.ayell.eu) 78.46.71.165
78.46.78.51
78.46.80.180
88.198.16.203
Eurobot/1.0 (http://www.ayell.eu) 78.46.78.51
78.46.80.180
88.198.16.203
findlinks/1.1.3-beta9 (+http://wortschatz.uni-leipzig.de/findlinks/) 132.180.136.10
Gaisbot/3.0+(robot06@gais.cs.ccu.edu.tw;+http://gais.cs.ccu.edu.tw/robot.php) 122.147.76.66 bis 122.147.76.67
122.147.76.77
140.123.100.12
203.70.235.36
203.70.235.39
203.73.108.68
203.73.176.40
203.73.176.43
210.66.69.161
210.66.69.184
220.228.152.82
Gigabot/3.0 (http://www.gigablast.com/spider.html) 66.231.188.103
66.231.188.140
66.231.188.164
66.231.188.72
66.231.189.19
66.231.189.74
67.16.94.2
gonzo/1[D] (+http://www.suchen.de/faq.html) 86.109.255.22
gonzo/1[P] (+http://www.suchen.de/faq.html) 212.34.185.67
gonzo/2[D] (+http://www.suchen.de/faq.html) 86.109.255.22
gonzo/2[P] (+http://www.suchen.de/faq.html) 212.34.185.68
gonzo1[D] +http://www.suchen.de/faq.html 86.109.255.22
87.139.37.64 bis 87.139.37.65
gonzo1[P] +http://www.suchen.de/faq.html 212.34.175.71 bis 212.34.175.72
gonzo2[D] +http://www.suchen.de/faq.html 87.139.37.64 bis 87.139.37.65
gonzo2[P] +http://www.suchen.de/faq.html 212.34.175.83
212.34.185.68
Googlebot-Image/1.0 66.249.65.186
66.249.65.194
66.249.65.207
66.249.65.51
66.249.65.60
66.249.65.66
66.249.66.133
66.249.67.109
66.249.67.121
66.249.67.4
66.249.68.226
66.249.70.186
66.249.71.110
66.249.71.134
66.249.71.140
66.249.71.17
66.249.71.173 bis 66.249.71.175
66.249.71.177 bis 66.249.71.178
66.249.71.213
66.249.71.215 bis 66.249.71.216
66.249.71.229
66.249.71.251
66.249.71.43
66.249.71.52
66.249.71.65
66.249.71.9
66.249.71.90
66.249.72.137
66.249.73.111
66.249.73.198
Googlebot/1.0 (Googlebot; http://www.google.com/; googlebot@google.com) 67.246.158.111
Googlebot/2.1 (+http://www.google.com/bot.html) 66.249.65.163
66.249.66.133
66.249.71.176
66.249.71.216
77.22.142.122
Grub/2.0 (Grub.org crawler; http://www.grub.org/; bot@grub.org) 149.20.54.197
GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html) 72.20.109.34
72.20.109.36 bis 72.20.109.37
holmes/3.12 (OnetSzukaj/5.0; +http://szukaj.onet.pl) 213.180.137.71 bis 213.180.137.73
holmes/3.12.3 (http://morfeo.centrum.cz/bot) 90.183.39.12
holmes/3.12.4 (http://morfeo.centrum.cz/bot) 90.183.39.17
90.183.39.29
ia_archiver 209.234.171.194
209.234.171.45
75.101.239.6
ia_archiver (+http://www.alexa.com/site/help/webmasters; crawler@alexa.com) 67.202.35.13
67.202.53.216
67.202.55.193
75.101.222.169
75.101.246.104
ia_archiver-web.archive.org 207.241.227.100
207.241.227.58
207.241.227.68 bis 207.241.227.70
207.241.227.72 bis 207.241.227.79
207.241.227.81 bis 207.241.227.95
207.241.227.97 bis 207.241.227.99
207.241.228.197
207.241.228.202
207.241.228.206
207.241.228.208
207.241.228.210
207.241.228.213
207.241.228.216
207.241.228.218
207.241.228.220
207.241.228.222
207.241.228.224 bis 207.241.228.225
207.241.228.232
207.241.228.235
207.241.229.140
207.241.229.148
207.241.229.157
207.241.229.58
207.241.229.60
207.241.229.62
207.241.229.64 bis 207.241.229.65
207.241.229.68
207.241.229.71
207.241.229.76
207.241.229.80 bis 207.241.229.82
207.241.229.87 bis 207.241.229.88
207.241.229.90 bis 207.241.229.91
207.241.229.96
ICCrawler - ICjobs (http://www.icjobs.de/bot.htm) 212.227.76.83
88.84.153.210
88.84.153.214
ichiro/3.0 (http://help.goo.ne.jp/door/crawler.html) 203.131.250.103
infometrics-bot, http://www.infometrics.de 89.146.194.169
89.146.194.172
inutch/Nutch-0.9 (inutch description; inutch.info; inuthc@inutch.info) 91.19.164.30
91.19.164.49
jasmin, neofonie search:heritrix 1.14.1_p2 (+http://spider.neofonie.de) 81.17.208.4
kalooga/KaloogaBot (Kalooga; http://www.kalooga.com/info.html?page=crawler; crawler@kalooga.com) 195.210.57.83
KnowItAll/Nutch-0.9 (Nutch-UW-Crawler; http://cs.washington.edu/homes/mjc/crawler.html; uwcrawler08@gmail.com) 128.208.3.93
larbin_2.6.3 (larbin2.6.3@unspecified.mail) 207.67.117.172
LCSG-Crawler/Nutch-0.9 (LCSG-Crawler; http://www.aschoff.nameu; lionel@aftermath.de) 84.167.204.210
librabot/1.0 (+http://search.msn.com/msnbot.htm) 202.96.51.154
202.96.51.159
219.142.53.16
219.142.53.18 bis 219.142.53.31
219.142.53.50
LinguaBot/v0.001-dev (MultiLinual Sarch Engine v0.001; LinguSeek; adin at linguaseek dot com) 98.210.243.184
Metaspinner/0.01 (Metaspinner; http://www.meta-spinner.de/; support@meta-spinner.de/) 213.203.245.153
MLBot (www.metadatalabs.com/mlbot) 71.41.201.38
Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml) 65.214.45.49
Mozilla/4.0 (compatible; Vagabondo/4.0Beta; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/) 82.94.179.42
Mozilla/4.0 (compatible; NaverBot/1.0; http://help.naver.com/customer_webtxt_02.jsp) 114.111.36.26
114.111.36.31
Mozilla/4.0 (compatible; NaverBot/1.0; http://help.naver.com/delete_main.asp) 114.111.36.26
114.111.36.31 bis 114.111.36.32
202.179.180.45
202.179.180.51 bis 202.179.180.54
Mozilla/5.0 (compatible; AdShadow +http://adshadow.de) 79.222.88.201
Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml) 65.214.45.101
65.214.45.124
65.214.45.175
65.214.45.213 bis 65.214.45.218
66.235.112.5
66.235.124.105
66.235.124.4
66.235.124.54 bis 66.235.124.59
66.235.124.76
66.235.124.78 bis 66.235.124.79
66.235.124.8
66.235.124.81
66.235.124.99
Mozilla/5.0 (compatible; attributor/1.13.2 +http://www.attributor.com) 64.41.145.176
64.41.145.211
64.41.145.240
Mozilla/5.0 (compatible; Charlotte/1.1; http://www.searchme.com/support/) 208.111.154.15 bis 208.111.154.16
208.111.154.23
Mozilla/5.0 (compatible; discobot/1.0; +http://discoveryengine.com/discobot.html) 208.96.54.70 bis 208.96.54.72
208.96.54.74
Mozilla/5.0 (compatible; DotBot/1.1; http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org) 208.115.111.242
Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) 193.47.80.48
193.47.80.77
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) 212.95.32.166
64.120.147.149
66.249.65.108
66.249.65.11
66.249.65.119
66.249.65.12
66.249.65.135
66.249.65.163 bis 66.249.65.164
66.249.65.167
66.249.65.169
66.249.65.18
66.249.65.184 bis 66.249.65.188
66.249.65.194
66.249.65.20
66.249.65.207
66.249.65.216
66.249.65.233
66.249.65.236
66.249.65.243 bis 66.249.65.244
66.249.65.25
66.249.65.34
66.249.65.41
66.249.65.45 bis 66.249.65.47
66.249.65.55
66.249.65.60
66.249.65.66 bis 66.249.65.67
66.249.65.71
66.249.65.80
66.249.65.83
66.249.65.86
66.249.65.97 bis 66.249.65.98
66.249.66.130
66.249.66.133
66.249.66.141
66.249.66.161
66.249.66.175
66.249.66.75
66.249.66.81
66.249.67.109
66.249.67.115
66.249.67.121
66.249.67.4
66.249.67.45
66.249.67.54
66.249.67.71
66.249.67.74
66.249.68.1
66.249.68.130
66.249.68.134
66.249.68.145
66.249.68.168
66.249.68.206
66.249.68.209
66.249.68.215
66.249.68.226
66.249.68.232
66.249.68.241
66.249.68.244
66.249.68.247
66.249.68.250
66.249.70.186
66.249.71.110
66.249.71.112
66.249.71.116
66.249.71.134 bis 66.249.71.135
66.249.71.137
66.249.71.140 bis 66.249.71.143
66.249.71.146
66.249.71.149
66.249.71.152
66.249.71.168
66.249.71.17
66.249.71.172 bis 66.249.71.178
66.249.71.195
66.249.71.198 bis 66.249.71.199
66.249.71.2
66.249.71.204
66.249.71.21
66.249.71.213 bis 66.249.71.216
66.249.71.228 bis 66.249.71.230
66.249.71.233
66.249.71.236
66.249.71.243 bis 66.249.71.245
66.249.71.248
66.249.71.251
66.249.71.34
66.249.71.37
66.249.71.43
66.249.71.46
66.249.71.5
66.249.71.52
66.249.71.65 bis 66.249.71.67
66.249.71.7
66.249.71.77
66.249.71.8 bis 66.249.71.9
66.249.71.90
66.249.72.108
66.249.72.137
66.249.72.161
66.249.72.239
66.249.72.244
66.249.72.75
66.249.73.111
66.249.73.198
66.249.73.26
89.149.197.204
Mozilla/5.0 (compatible; heritrix/@VERSION@ +http://www.kingsoft.com) 125.89.65.194
Mozilla/5.0 (compatible; heritrix/1.12.1 +http://www.neofonie.de) 81.17.210.28
Mozilla/5.0 (compatible; heritrix/2.0.0 +http://www.mariamosolova.com) 64.236.128.12
Mozilla/5.0 (compatible; jobs.de-Robot +http://www.jobs.de) 81.17.208.4
Mozilla/5.0 (compatible; MJ12bot/v1.2.3; http://www.majestic12.co.uk/bot.php?+) 77.133.148.166
82.95.238.2
85.178.81.21
85.23.67.140
Mozilla/5.0 (compatible; MJ12bot/v1.2.4; http://www.majestic12.co.uk/bot.php?+) 138.47.102.92
213.93.147.235
217.14.6.172
62.249.173.117
68.112.173.126
69.115.241.123
72.90.57.210
77.133.205.29
77.186.61.34
84.202.171.50
84.29.102.71
91.17.60.242
91.66.201.123
92.77.137.27
94.23.23.121
Mozilla/5.0 (compatible; MJ12bot/v1.2.5; http://www.majestic12.co.uk/bot.php?+) 173.25.58.214
205.209.170.18 bis 205.209.170.19
217.14.6.210
217.14.6.72
62.249.174.223
69.115.241.123
78.53.11.251
82.231.61.186
83.101.43.251
85.23.34.55
87.118.90.32
88.191.108.76
91.195.55.72
91.205.174.49
92.249.115.128
Mozilla/5.0 (compatible; MJ12bot/v1.3.0; http://www.majestic12.co.uk/bot.php?+) 195.159.6.21
91.67.216.229
Mozilla/5.0 (compatible; MJ12bot/v1.3.1; http://www.majestic12.co.uk/bot.php?+) 131.180.146.99
138.47.102.198
91.67.216.203
Mozilla/5.0 (compatible; MJ12bot/v1.3.2; http://www.majestic12.co.uk/bot.php?+) 205.209.170.19
69.115.241.123
91.205.173.243
91.205.174.49
93.217.110.251
94.23.238.27
Mozilla/5.0 (compatible; OpenX Spider; http://www.openx.org) 174.129.133.121
75.101.150.3
75.101.227.34
Mozilla/5.0 (compatible; Plukkie/1.0; http://www.botje.com/plukkie.htm) 87.118.86.133
Mozilla/5.0 (compatible; Plukkie/1.1; http://www.botje.com/plukkie.htm) 87.118.86.133
Mozilla/5.0 (compatible; Plukkie/1.2; http://www.botje.com/plukkie.htm) 87.118.86.133
Mozilla/5.0 (compatible; ScoutJet; +http://www.scoutjet.com/) 38.108.180.11 bis 38.108.180.12
38.108.180.3 bis 38.108.180.4
38.108.180.8 bis 38.108.180.9
64.13.159.70
64.13.160.3
64.13.160.5
Mozilla/5.0 (compatible; Semager/1.4; http://www.semager.de/blog/semager-bots/) 212.114.209.251
Mozilla/5.0 (compatible; Steeler/3.4; http://www.tkl.iis.u-tokyo.ac.jp/~crawler/) 157.82.156.160
Mozilla/5.0 (compatible; SuchbaerBot/0.4; +http://bot.suchbaer.de/info.html) 78.46.96.83
Mozilla/5.0 (compatible; SuchbaerBot/0.83; +http://bot.suchbaer.de/info.html) 188.40.62.140
Mozilla/5.0 (compatible; WebmasterCoffee/0.7; +http://webmastercoffee.com/about) 88.198.2.105
Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html) 202.160.178.154
202.160.178.79
202.160.188.223 bis 202.160.188.224
202.160.188.229
202.160.188.232 bis 202.160.188.233
203.209.252.15 bis 203.209.252.16
203.209.252.60 bis 203.209.252.61
203.209.252.63
203.209.252.70
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) 67.195.37.105
67.195.37.107 bis 67.195.37.109
67.195.37.111 bis 67.195.37.112
67.195.37.114 bis 67.195.37.116
67.195.37.120
67.195.37.122 bis 67.195.37.124
67.195.37.126
67.195.37.153 bis 67.195.37.157
67.195.37.159
67.195.37.163 bis 67.195.37.164
67.195.37.166 bis 67.195.37.167
67.195.37.169 bis 67.195.37.170
67.195.37.173
67.195.37.175 bis 67.195.37.177
67.195.37.179
67.195.37.184 bis 67.195.37.188
67.195.37.90
67.195.37.92 bis 67.195.37.93
67.195.37.95
67.195.37.97
72.30.142.105 bis 72.30.142.109
72.30.142.111
72.30.142.113 bis 72.30.142.114
72.30.142.116 bis 72.30.142.119
72.30.142.122 bis 72.30.142.123
72.30.142.125 bis 72.30.142.126
72.30.142.151 bis 72.30.142.156
72.30.142.158 bis 72.30.142.160
72.30.142.162 bis 72.30.142.168
72.30.142.170 bis 72.30.142.175
72.30.142.177
72.30.142.179 bis 72.30.142.180
72.30.142.182 bis 72.30.142.183
72.30.142.185 bis 72.30.142.187
72.30.142.189
72.30.142.215 bis 72.30.142.224
72.30.142.226
72.30.142.228 bis 72.30.142.229
72.30.142.231
72.30.142.233 bis 72.30.142.235
72.30.142.237 bis 72.30.142.238
72.30.142.240 bis 72.30.142.241
72.30.142.243 bis 72.30.142.245
72.30.142.247 bis 72.30.142.254
72.30.142.82 bis 72.30.142.83
72.30.142.85
72.30.142.87 bis 72.30.142.96
72.30.142.99
72.30.161.215
72.30.161.217 bis 72.30.161.236
72.30.161.240 bis 72.30.161.243
72.30.161.246 bis 72.30.161.250
72.30.161.253
72.30.65.23 bis 72.30.65.24
72.30.65.26 bis 72.30.65.27
72.30.65.30 bis 72.30.65.32
72.30.65.34
72.30.65.36 bis 72.30.65.38
72.30.65.40 bis 72.30.65.42
72.30.65.46 bis 72.30.65.48
72.30.65.50 bis 72.30.65.51
72.30.65.53 bis 72.30.65.56
72.30.65.58 bis 72.30.65.62
72.30.78.215
72.30.78.217 bis 72.30.78.223
72.30.78.225 bis 72.30.78.228
72.30.78.230 bis 72.30.78.234
72.30.78.236
72.30.78.238
72.30.78.241 bis 72.30.78.245
72.30.78.247 bis 72.30.78.248
72.30.78.250 bis 72.30.78.253
72.30.79.106 bis 72.30.79.112
72.30.79.115 bis 72.30.79.117
72.30.79.119
72.30.79.122 bis 72.30.79.126
72.30.79.23 bis 72.30.79.40
72.30.79.42
72.30.79.44 bis 72.30.79.45
72.30.79.49
72.30.79.51 bis 72.30.79.52
72.30.79.54 bis 72.30.79.55
72.30.79.57 bis 72.30.79.61
72.30.79.83 bis 72.30.79.85
72.30.79.87 bis 72.30.79.89
72.30.79.91 bis 72.30.79.96
72.30.79.98 bis 72.30.79.99
72.30.81.152 bis 72.30.81.163
72.30.81.165 bis 72.30.81.171
72.30.81.173 bis 72.30.81.187
72.30.81.190
72.30.87.105 bis 72.30.87.109
72.30.87.112 bis 72.30.87.116
72.30.87.119 bis 72.30.87.122
72.30.87.124 bis 72.30.87.125
72.30.87.82
72.30.87.84
72.30.87.86
72.30.87.88
72.30.87.91 bis 72.30.87.92
72.30.87.94 bis 72.30.87.99
74.6.17.151 bis 74.6.17.190
74.6.18.215 bis 74.6.18.233
74.6.18.235 bis 74.6.18.254
74.6.22.101 bis 74.6.22.108
74.6.22.112
74.6.22.119 bis 74.6.22.122
74.6.22.124 bis 74.6.22.126
74.6.22.151 bis 74.6.22.170
74.6.22.172 bis 74.6.22.190
74.6.22.87 bis 74.6.22.99
74.6.27.245
74.6.79.180
74.6.8.100 bis 74.6.8.108
74.6.8.110 bis 74.6.8.126
74.6.8.87 bis 74.6.8.99
Mozilla/5.0 (compatible; YoudaoBot/1.0; http://www.youdao.com/help/webmaster/spider/; ) 202.108.7.217
61.135.249.217
Mozilla/5.0 (http://textdigger.com) Gecko/20061208 Firefox/2.0.0.1 64.124.138.164
Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html) 208.36.144.9
64.1.215.164
Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html) 208.36.144.9
64.1.215.164
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0/1.0 (bot; http://; bot@bot.com) 216.158.1.202 bis 216.158.1.203
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (support.voilabot@orange-ftgroup.com) 193.252.149.15
81.52.143.15
Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7.8) Gecko/20050512 Firefox/1.0.4 84.63.101.207
Mozilla/5.0 (Yahoo-MMCrawler/4.0; mailto:vertical-crawl-support@yahoo-inc.com) 68.142.212.222 bis 68.142.212.223
68.142.212.232
68.142.212.234
68.142.215.153
MSNBOT_Mobile MSMOBOT Mozilla/2.0 (compatible; MSIE 4.02; Windows CE; Default)/1.1 (+http://search.msn.com/msnbot.htm) 202.96.51.153
219.142.53.26
msnbot-media/1.0 (+http://search.msn.com/msnbot.htm) 65.55.212.218 bis 65.55.212.220
65.55.235.202
msnbot-media/1.1 (+http://search.msn.com/msnbot.htm) 207.46.199.163
207.46.199.171 bis 207.46.199.172
65.55.108.130 bis 65.55.108.149
65.55.108.154 bis 65.55.108.173
65.55.217.45
65.55.217.53 bis 65.55.217.69
65.55.220.205 bis 65.55.220.224
65.55.220.88
65.55.230.213
65.55.230.232
65.55.230.234
65.55.230.236
65.55.230.242 bis 65.55.230.243
65.55.230.248 bis 65.55.230.255
65.55.231.100 bis 65.55.231.107
65.55.231.112 bis 65.55.231.131
65.55.231.16 bis 65.55.231.35
65.55.231.40 bis 65.55.231.59
65.55.231.64 bis 65.55.231.83
65.55.231.88 bis 65.55.231.99
65.55.234.161
65.55.234.164 bis 65.55.234.169
65.55.37.181
65.55.37.188 bis 65.55.37.198
msnbot/1.1 (+http://search.msn.com/msnbot.htm) 65.55.104.165 bis 65.55.104.168
65.55.104.21 bis 65.55.104.25
65.55.210.100 bis 65.55.210.104
65.55.211.161
65.55.211.164
65.55.211.167
65.55.211.169 bis 65.55.211.175
65.55.211.180 bis 65.55.211.195
65.55.211.197 bis 65.55.211.199
65.55.232.22 bis 65.55.232.24
65.55.25.145
65.55.25.147
91.209.196.76
msnbot/2.0b (+http://search.msn.com/msnbot.htm) 65.55.207.48 bis 65.55.207.54
65.55.207.69 bis 65.55.207.78
65.55.207.93 bis 65.55.207.99
65.55.215.74 bis 65.55.215.76
65.55.216.10 bis 65.55.216.11
65.55.216.31 bis 65.55.216.33
65.55.216.53 bis 65.55.216.55
65.55.216.9
65.55.218.158 bis 65.55.218.160
65.55.37.202 bis 65.55.37.203
65.55.51.110 bis 65.55.51.112
65.55.51.114 bis 65.55.51.115
65.55.51.69 bis 65.55.51.70
MSRBOT (http://research.microsoft.com/research/sv/msrbot/) 209.249.11.4
Nebullabot/2.2 (+http://www.nebullabot.de) 80.150.6.184
Netluchs/Nutch-1.0-dev ( ; http://www.netluchs.de/; _do_not_spam_me___humans_please_use_info_at_netluchs.de_without_the_ 193.164.8.62
NokiaN70/. FASTMobileCrawl/6.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 62.113.137.5
nrsbot/5.0(loopip.com/robot.html) 74.95.0.186
Nutch/0.9 (OpenX Spider) 67.202.3.142
75.101.227.34
nutch/Nutch-1.0-dev (nutch) 62.54.4.138
OOZBOT/0.17 (--; http://www.setooz.com/oozbot.html; agentname at setooz dot_com) 67.215.230.10
OOZBOT/0.17 (--; http://www.setooz.com/oozbot.html#; agentname at setooz dot_com) 67.215.230.11
OOZBOT/0.20 ( -- ; http://www.setooz.com/oozbot.html ; agentname at setooz dot_com ) 67.215.230.22 bis 67.215.230.23
psbot/0.1 (+http://www.picsearch.com/bot.html) 217.212.224.183
QEAVis agent/Nutch-0.9 (http://nlp.uned.es/qeavis/) 62.204.207.46
Rankivabot/3.2 (www.rankiva.com; 3.2; vzmxikn) 64.228.129.91
70.48.115.55
RAYSPIDER/Nutch-0.9 199.46.198.231 bis 199.46.198.232
199.46.199.230
199.46.199.232
199.46.245.237
robotgenius/Nutch-1.0-dev 208.96.10.200
Ruky-Bot (Version: 1.06, powered by www.ruky.de +http://www.ruky.de/pfadzurbotseite/bot.html) 81.169.145.25
SapphireWebCrawler/Nutch-1.0-dev (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; mhoy@cs.cmu.e 64.88.164.196
64.88.164.198
Scooter/3.3 69.147.79.37
Seekbot/1.0 (http://www.seekbot.net/bot.html) RobotsTxtFetcher/1.2 195.27.115.50
Semager/1.2 http://www.semager.de/blog/semager-bots/ 212.114.209.250 bis 212.114.209.251
Semager/1.3 http://www.semager.de/blog/semager-bots/ 212.114.209.251 bis 212.114.209.252
Shelob (shelob@gmx.net) 78.46.84.165
ShopWiki/1.0 ( +http://www.shopwiki.com/wiki/Help:Bot) 38.98.120.75
Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/) 88.131.106.15
suchbaer.de 88.198.32.110
suchbaer.de/0.3.24 88.198.32.110
SurveyBot/2.3 (Whois Source) 216.145.11.94
216.145.17.190
64.246.165.10
64.246.165.150
64.246.165.160
64.246.165.170
64.246.165.180
ThumbShots-Bot (+http://thumbshots.in/bot.html) 217.79.182.92
89.163.146.123
thumbshots-de-bot (+http://www.thumbshots.de) 212.112.229.155
TinEye/1.1 (http://tineye.com/crawler.html) 75.101.207.102
Touche (+http://www.touche.com.ve) 190.198.38.39
TurnitinBot/2.1 (http://www.turnitin.com/robot/crawlerinfo.html) 65.98.224.2
TwengaBot/1.1 (+http://www.twenga.com/bot.html) 85.17.165.75
85.17.165.89
85.17.165.97
85.17.216.161
85.17.226.77
85.17.226.81
85.17.89.75
VisBot/2.0 (Visvo.com Crawler; http://www.visvo.com/bot. 72.249.60.74
WebAlta Crawler/2.0 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU) 77.91.224.5 bis 77.91.224.6
wwwster/1.4 (Beta, mailto:gue@cis.uni-muenchen.de) 129.187.148.244
yacy/0.5910493 (www.yacy.net; x86 Windows XP 5.1; java 1.6.0_07; Europe/de) Jakarta Commons-HttpClient/3.1 84.167.77.66
yacybot (amd64 Linux 2.6.24-19-generic; java 1.6.0; Europe/de) http://yacy.net/bot.html 84.143.67.161
yacybot (i386 Linux 2.6.16.62; java 1.5.0_14; Europe/de) http://yacy.net/bot.html 82.83.43.139
yacybot (i386 Linux 2.6.24.3; java 1.6.0_10; GMT/en) http://yacy.net/bot.html 217.232.100.10
yacybot (x86 Windows XP 5.1; java 1.6.0_07; Europe/de) http://yacy.net/bot.html 91.21.109.234
Yandex/1.01.001 (compatible; Win16; H) 77.88.22.113
77.88.22.91
Yandex/1.01.001 (compatible; Win16; I) 77.88.27.27
93.158.149.32
Yanga WorldSearch Bot v1.1/beta (http://www.yanga.co.uk/) 77.91.224.5
91.205.124.5
91.205.125.15
Yeti/1.0 (+http://help.naver.com/robots/) 61.247.221.43
61.247.221.45
61.247.222.52 bis 61.247.222.56
Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/) 61.247.221.36 bis 61.247.221.45
61.247.222.47 bis 61.247.222.50
61.247.222.52 bis 61.247.222.56
61.247.222.82
Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/), Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/) 61.247.221.43

Haben Ihnen diese Informationen weitergeholfen? Ja Nein

2005 • Komerci • 2010