List of Crawlers & Bots, and their I.P. Addresses.
I use some code to identify crawlers and bots on TheInterPost so that they do not get included in the usage stats on the Research page. This way only human visits are counted in the stats.
Updated: 26 September 2009 (I will try to update this list at least once a month).
Over time I’ve collected a long list of crawlers, bots (or whatever you like to call automated access to sites) and apart from helping create a better Research page, the list os currently of no other use to me.
So here it is for you to do what you wish with it. Have fun!
(row) string 51
namestring 15
ip1 crawl-66-249-71-248.googlebot.com 66.249.71.248 2 crawl-66-249-66-38.googlebot.com 66.249.66.38 3 crawl-66-249-65-142.googlebot.com 66.249.65.142 4 crawl-66-249-65-210.googlebot.com 66.249.65.210 5 crawl-66-249-66-201.googlebot.com 66.249.66.201 6 crawl-66-249-65-4.googlebot.com 66.249.65.4 7 crawl-66-249-65-102.googlebot.com 66.249.65.102 8 livebot-65-55-208-24.search.live.com 65.55.208.24 9 livebot-65-55-208-21.search.live.com 65.55.208.21 10 livebot-65-55-213-39.search.live.com 65.55.213.39 11 livebot-65-55-208-19.search.live.com 65.55.208.19 12 livebot-65-55-208-20.search.live.com 65.55.208.20 13 livebot-65-55-208-23.search.live.com 65.55.208.23 14 livebot-65-55-208-22.search.live.com 65.55.208.22 15 livebot-65-55-208-25.search.live.com 65.55.208.25 16 livebot-65-55-212-19.search.live.com 65.55.212.19 17 crawl-66-249-66-196.googlebot.com 66.249.66.196 18 crawl-66-249-66-66.googlebot.com 66.249.66.66 19 crawl-66-249-65-241.googlebot.com 66.249.65.241 20 crawl-66-249-66-51.googlebot.com 66.249.66.51 21 crawl-66-249-65-48.googlebot.com 66.249.65.48 22 crawl-66-249-65-98.googlebot.com 66.249.65.98 23 livebot-65-54-188-14.search.live.com 65.54.188.14 24 livebot-65-55-208-18.search.live.com 65.55.208.18 25 livebot-65-55-208-17.search.live.com 65.55.208.17 26 livebot-65-54-188-15.search.live.com 65.54.188.15 27 livebot-65-54-188-13.search.live.com 65.54.188.13 28 livebot-65-54-188-16.search.live.com 65.54.188.16 29 crawl13.exabot.com 193.47.80.49 30 livebot-65-55-212-232.search.live.com 65.55.212.232 31 livebot-65-55-213-36.search.live.com 65.55.213.36 32 livebot-65-54-188-17.search.live.com 65.54.188.17 33 livebot-207-46-98-77.search.live.com 207.46.98.77 34 crawl-66-249-65-12.googlebot.com 66.249.65.12 35 livebot-65-55-208-16.search.live.com 65.55.208.16 36 crawl-66-249-65-11.googlebot.com 66.249.65.11 37 crawl-66-249-65-45.googlebot.com 66.249.65.45 38 crawl-66-249-66-48.googlebot.com 66.249.66.48 39 crawl-66-249-66-229.googlebot.com 66.249.66.229 40 crawl-66-249-65-227.googlebot.com 66.249.65.227 41 crawl-66-249-65-112.googlebot.com 66.249.65.112 42 livebot-207-46-98-76.search.live.com 207.46.98.76 43 livebot-207-46-98-78.search.live.com 207.46.98.78 44 crawl-66-249-65-106.googlebot.com 66.249.65.106 45 crawl11.springright.com 66.135.44.43 46 livebot-65-55-208-145.search.live.com 65.55.208.145 47 crawler4.irl.cs.tamu.edu 128.194.135.94 48 crawl14.springright.com 66.135.44.46 49 livebot-65-55-212-233.search.live.com 65.55.212.233 50 livebot-65-55-212-234.search.live.com 65.55.212.234 51 crawl-66-249-66-142.googlebot.com 66.249.66.142 52 crawl-66-249-65-34.googlebot.com 66.249.65.34 53 crawl-66-249-65-39.googlebot.com 66.249.65.39 54 livebot-65-55-208-124.search.live.com 65.55.208.124 55 livebot-65-55-208-132.search.live.com 65.55.208.132 56 livebot-65-55-208-125.search.live.com 65.55.208.125 57 crawl-66-249-73-170.googlebot.com 66.249.73.170 58 crawl-66-249-66-129.googlebot.com 66.249.66.129 59 crawl-66-249-66-133.googlebot.com 66.249.66.133 60 crawl-66-249-66-244.googlebot.com 66.249.66.244 61 crawl-66-249-66-115.googlebot.com 66.249.66.115 62 crawl-66-249-66-14.googlebot.com 66.249.66.14 63 crawl-66-249-66-71.googlebot.com 66.249.66.71 64 crawl-66-249-66-18.googlebot.com 66.249.66.18 65 crawl-66-249-72-99.googlebot.com 66.249.72.99 66 crawl-66-249-65-235.googlebot.com 66.249.65.235 67 38.98.120.69 38.98.120.69 68 crawl-66-249-66-10.googlebot.com 66.249.66.10 69 crawl-66-249-66-131.googlebot.com 66.249.66.131 70 crawl-66-249-72-83.googlebot.com 66.249.72.83 71 crawl-66-249-65-145.googlebot.com 66.249.65.145 72 crawl-66-249-66-102.googlebot.com 66.249.66.102 73 livebot-65-55-209-122.search.live.com 65.55.209.122 74 livebot-65-55-209-128.search.live.com 65.55.209.128 75 livebot-65-55-209-121.search.live.com 65.55.209.121 76 crawl-66-249-66-180.googlebot.com 66.249.66.180 77 crawl-66-249-72-129.googlebot.com 66.249.72.129 78 crawl-66-249-66-80.googlebot.com 66.249.66.80 79 crawl-66-249-72-3.googlebot.com 66.249.72.3 80 crawler100.ask.com 65.214.45.100 81 38.99.44.106 38.99.44.106 82 crawl-66-249-66-110.googlebot.com 66.249.66.110 83 crawl-66-249-72-193.googlebot.com 66.249.72.193 84 crawl-66-249-72-46.googlebot.com 66.249.72.46 85 crawl-66-249-66-143.googlebot.com 66.249.66.143 86 crawl-66-249-66-161.googlebot.com 66.249.66.161 87 srv07.id-search.org 67.159.44.232 88 srv03.id-search.org 66.90.101.75 89 spider10.picsearch.com 217.212.224.159 90 spider6.picsearch.com 217.212.224.146 91 spider7.picsearch.com 217.212.224.147 92 spider3.picsearch.com 217.212.224.143 93 spider4.picsearch.com 217.212.224.144 94 spider20.picsearch.com 217.212.224.170 95 spider18.picsearch.com 217.212.224.168 96 spider21.picsearch.com 217.212.224.177 97 domU-12-31-37-00-02-81.usma3.compute.amazonaws.com 72.44.62.118 98 domU-12-31-37-00-06-0B.usma3.compute.amazonaws.com 72.44.58.245 99 crawl-66-249-65-173.googlebot.com 66.249.65.173 100 crawl-16.cuill.com 208.36.144.7 101 crawl-13.cuill.com 38.99.44.104 102 208.36.144.6.ptr.us.xo.net 208.36.144.6 103 crawl-66-249-72-205.googlebot.com 66.249.72.205 104 xcrawl103.alexa.com 64.208.172.175 105 crawl-66-249-72-43.googlebot.com 66.249.72.43 106 egspd42135.ask.com 65.214.44.116 107 livebot-65-55-212-235.search.live.com 65.55.212.235 108 crawl-66-249-72-40.googlebot.com 66.249.72.40 109 egspd42100.ask.com 65.214.44.81 110 crawl-66-249-72-50.googlebot.com 66.249.72.50 111 speedyspider.entireweb.com 88.131.153.91 112 crawler2010.ask.com 63.123.238.50 113 balin.seekbot.net 195.27.115.51 114 crawler4026.ask.com 65.214.45.135 115 ss110.dal0.gigablast.com 38.114.104.120 116 livebot-65-55-212-46.search.live.com 65.55.212.46 117 livebot-65-55-212-237.search.live.com 65.55.212.237 118 crawl-66-249-72-163.googlebot.com 66.249.72.163 119 ec2-67-202-22-47.compute-1.amazonaws.com 67.202.22.47 120 ec2-67-202-26-120.compute-1.amazonaws.com 67.202.26.120 121 s15252624.onlinehome-server.info 87.106.129.54 122 crawl-66-249-72-135.googlebot.com 66.249.72.135 123 xcrawl106.alexa.com 64.208.172.178 124 crawl-66-249-73-111.googlebot.com 66.249.73.111 125 crawl-66-249-65-205.googlebot.com 66.249.65.205 126 crawl-66-249-66-208.googlebot.com 66.249.66.208 127 crawl-66-249-66-112.googlebot.com 66.249.66.112 128 crawl-66-249-65-38.googlebot.com 66.249.65.38 129 crawl-66-249-65-42.googlebot.com 66.249.65.42 130 crawl-66-249-72-18.googlebot.com 66.249.72.18 131 crawl-66-249-72-195.googlebot.com 66.249.72.195 132 spider25.picsearch.com 217.212.224.181 133 crawl-66-249-72-106.googlebot.com 66.249.72.106 134 crawl-66-249-72-198.googlebot.com 66.249.72.198 135 crawl-66-249-66-177.googlebot.com 66.249.66.177 136 crawl-66-249-66-145.googlebot.com 66.249.66.145 137 crawl-66-249-66-3.googlebot.com 66.249.66.3 138 spider23.picsearch.com 217.212.224.179 139 crawl-66-249-66-228.googlebot.com 66.249.66.228 140 crawl-66-249-66-168.googlebot.com 66.249.66.168 141 crawl-66-249-65-105.googlebot.com 66.249.65.105 142 crawl-66-249-66-19.googlebot.com 66.249.66.19 143 crawl-66-249-66-33.googlebot.com 66.249.66.33 144 crawl-66-249-66-11.googlebot.com 66.249.66.11 145 192.168.1.41 192.168.1.41 146 crawl-66-249-66-236.googlebot.com 66.249.66.236 147 crawl-66-249-65-164.googlebot.com 66.249.65.164 148 AAmiens-151-1-49-17.w83-192.abo.wanadoo.fr 83.192.36.17 149 lns-bzn-23-82-248-100-152.adsl.proxad.net 82.248.100.152 150 201-25-108-130.cbace301.ipd.brasiltelecom.net.br 201.25.108.130 151 crawl-66-249-65-82.googlebot.com 66.249.65.82 152 crawl-66-249-72-45.googlebot.com 66.249.72.45 153 crawl-66-249-66-194.googlebot.com 66.249.66.194 154 crawl-66-249-72-212.googlebot.com 66.249.72.212 155 crawl-66-249-66-70.googlebot.com 66.249.66.70 156 crawl-66-249-65-197.googlebot.com 66.249.65.197 157 per92-7-82-236-196-90.fbx.proxad.net 82.236.196.90 158 crawl-66-249-72-111.googlebot.com 66.249.72.111 159 crawl-66-249-65-141.googlebot.com 66.249.65.141 160 host220-59-dynamic.0-87-r.retail.telecomitalia.it 87.0.59.220 161 host123-196-dynamic.58-82-r.retail.telecomitalia.it 82.58.196.123 162 crawl-66-249-72-35.googlebot.com 66.249.72.35 163 crawl-66-249-65-5.googlebot.com 66.249.65.5 164 crawl-66-249-67-181.googlebot.com 66.249.67.181 165 pool-72-90-71-132.syrcny.fios.verizon.net 72.90.71.132 166 crawl-66-249-65-67.googlebot.com 66.249.65.67 167 pool-71-168-122-164.cncdnh.fios.verizon.net 71.168.122.164 168 77-101-62-103.cable.ubr03.craw.blueyonder.co.uk 77.101.62.103 169 89-149-227-78.internetserviceteam.com 89.149.227.78 170 crawl-66-249-65-208.googlebot.com 66.249.65.208 171 p15201384.pureserver.info 212.227.102.33 172 crawl-66-249-71-8.googlebot.com 66.249.71.8 173 crawl-66-249-71-9.googlebot.com 66.249.71.9 174 crawl-66-249-71-7.googlebot.com 66.249.71.7 175 rbi0342.giga-dns.com 193.37.152.242 176 77.91.224.12 77.91.224.12 177 crawl-2.cuill.com 38.99.13.123 178 crawl-66-249-72-48.googlebot.com 66.249.72.48 179 223.Red-79-153-27.staticIP.rima-tde.net 79.153.27.223 180 49.Red-83-44-60.dynamicIP.rima-tde.net 83.44.60.49 181 crawler5017.ask.com 66.235.124.17 182 crawler.kalooga.com 195.210.57.83 183 c-76-24-48-135.hsd1.nh.comcast.net 76.24.48.135 184 crawl-66-249-71-77.googlebot.com 66.249.71.77 185 crawl-66-249-71-78.googlebot.com 66.249.71.78 186 crawl-66-249-71-79.googlebot.com 66.249.71.79 187 crawl-66-249-71-179.googlebot.com 66.249.71.179 188 crawl-66-249-71-180.googlebot.com 66.249.71.180 189 crawl-66-249-71-181.googlebot.com 66.249.71.181 190 38.103.63.62 38.103.63.62 191 205.209.170.18 205.209.170.18 192 82-44-90-16.cable.ubr03.craw.blueyonder.co.uk 82.44.90.16 193 crawl-66-249-71-152.googlebot.com 66.249.71.152 194 crawl-66-249-71-151.googlebot.com 66.249.71.151 195 crawl-66-249-71-150.googlebot.com 66.249.71.150 196 ukspider3.wise-guys.nl 82.94.176.143 197 llf520195.crawl.yahoo.net 74.6.17.169 198 llf531267.crawl.yahoo.net 72.30.79.99 199 llf520055.crawl.yahoo.net 74.6.22.168 200 llf531151.crawl.yahoo.net 72.30.142.88 201 77-96-83-75.cable.ubr03.craw.blueyonder.co.uk 77.96.83.75 202 llf520051.crawl.yahoo.net 74.6.22.180 203 llf531331.crawl.yahoo.net 72.30.78.247 204 ANancy-257-1-131-51.w90-40.abo.wanadoo.fr 90.40.162.51 205 llf531251.crawl.yahoo.net 72.30.79.85 206 crawl-66-249-71-209.googlebot.com 66.249.71.209 207 ec2-75-101-218-215.compute-1.amazonaws.com 75.101.218.215 208 llf531049.crawl.yahoo.net 72.30.142.243 209 crawl-66-249-71-85.googlebot.com 66.249.71.85 210 crawl-66-249-71-138.googlebot.com 66.249.71.138 211 ec2-174-129-100-129.compute-1.amazonaws.com 174.129.100.129 212 crawl-66-249-71-3.googlebot.com 66.249.71.3 213 llf520032.crawl.yahoo.net 74.6.8.125 214 llf520009.crawl.yahoo.net 74.6.8.115 215 crawl-66-249-71-238.googlebot.com 66.249.71.238 216 llf520197.crawl.yahoo.net 74.6.17.171 217 llf520073.crawl.yahoo.net 74.6.22.183 218 llf520049.crawl.yahoo.net 74.6.22.190 219 crawl10.dotnetdotcom.org 208.115.111.251 220 llf520048.crawl.yahoo.net 74.6.22.185 221 llf520017.crawl.yahoo.net 74.6.8.121 222 llf531333.crawl.yahoo.net 72.30.78.236 223 crawl-66-249-71-231.googlebot.com 66.249.71.231 224 llf531378.crawl.yahoo.net 72.30.65.36 225 crawl-66-249-71-247.googlebot.com 66.249.71.247 226 llf531379.crawl.yahoo.net 72.30.65.25 227 crawl-2c.cuil.com 216.129.119.12 228 crawl-15c.cuil.com 216.129.119.45 229 llf520109.crawl.yahoo.net 74.6.22.107 230 crawl-15c.cuil.com 38.99.13.116 231 llf531117.crawl.yahoo.net 72.30.142.189 232 llf520101.crawl.yahoo.net 74.6.22.122 233 crawl2.nat.svl.searchme.com 208.111.154.16 234 crawl-66-249-66-45.googlebot.com 66.249.66.45 235 crawl-66-249-71-182.googlebot.com 66.249.71.182 236 crawl-66-249-71-146.googlebot.com 66.249.71.146 237 b3091252.crawl.yahoo.net 67.195.112.36 238 llf320020.crawl.yahoo.net 67.195.37.93 239 crawl-66-249-71-47.googlebot.com 66.249.71.47 240 crawl-13c.cuil.com 216.129.119.43 241 llf531250.crawl.yahoo.net 72.30.79.119 242 crawl-66-249-71-173.googlebot.com 66.249.71.173 243 b3090794.crawl.yahoo.net 67.195.115.88 244 crawl-66-249-72-132.googlebot.com 66.249.72.132 245 crawl-66-249-72-130.googlebot.com 66.249.72.130 246 crawl-15c.cuil.com 67.218.116.162 247 94-169-190-143.cable.ubr06.craw.blueyonder.co.uk 94.169.190.143 248 llf531069.crawl.yahoo.net 72.30.142.218 249 llf531355.crawl.yahoo.net 72.30.78.253 250 crawl-66-249-71-27.googlebot.com 66.249.71.27 251 llf531050.crawl.yahoo.net 72.30.142.24 252 llf531285.crawl.yahoo.net 72.30.79.30 253 llf520173.crawl.yahoo.net 74.6.17.177 254 llf531010.crawl.yahoo.net 72.30.161.218 255 94-169-191-32.cable.ubr06.craw.blueyonder.co.uk 94.169.191.32 256 llf531011.crawl.yahoo.net 72.30.161.235 257 llf531050.crawl.yahoo.net 72.30.142.248 258 crawl-66-249-65-73.googlebot.com 66.249.65.73 259 crawl-66-249-65-76.googlebot.com 66.249.65.76 260 b3091312.crawl.yahoo.net 67.195.111.237 261 crawl-66-249-65-71.googlebot.com 66.249.65.71 262 crawl-66-249-65-108.googlebot.com 66.249.65.108 263 crawl-66-249-65-153.googlebot.com 66.249.65.153 264 crawl-66-249-65-154.googlebot.com 66.249.65.154 265 b3091289.crawl.yahoo.net 67.195.111.252 266 llf531301.crawl.yahoo.net 72.30.79.32 267 crawl-66-249-71-38.googlebot.com 66.249.71.38 268 llf531207.crawl.yahoo.net 72.30.81.184
RATE THIS POST
Move Your Mouse Over Stars and Click
(No Ratings Yet)
Move Your Mouse Over Stars and Click
(No Ratings Yet)
Loading ...
Loading ...