store | blogs | forums | twitter | facebook | wiki | mailing lists | downloads | support portal
Atomic Secure Linux
It is currently Thu Apr 24, 2014 8:08 am

» Feed - Atomicorp

All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 2 posts ] 
Author Message
 Post subject: 80legs.com crawler: blocked by crawler-protection?
Unread postPosted: Tue Oct 23, 2012 7:39 am 
Offline
Forum User
Forum User

Joined: Mon May 07, 2012 9:37 am
Posts: 51
Location: Zurich
hello,

I'm not sure if this is the right forum for this. We just got hammered by crawler 80legs.com. They came from multiple IP-Addresses simultaneously and hammered 2 sites of ours quite hard.

The useragent-string seems to stay the same:
Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/webcrawler.html) Gecko/2008032620

I blocked them via .htaccess but was wondering. Would the recently added search-engine/crawler-protection help with this? I haven't turned it on yet, but plan to do so soon.

If it doesn't get blocked, can you add it or can I do this myself server-wide?

kind regards

-Stephan


Top
 Profile  
 
 Post subject: Re: 80legs.com crawler: blocked by crawler-protection?
Unread postPosted: Fri Nov 09, 2012 9:44 pm 
Offline
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
User avatar

Joined: Thu Feb 07, 2008 7:49 pm
Posts: 3548
Location: Chantilly, VA
Forgot to follow up, this was added to the real time rules and ASL on the 24th of October. So if you are using the latest rules or ASL this bot will be blocked.

_________________
Michael Shinn
Atomicorp - Security For Everyone

Co-Author of Troubleshooting Linux Firewalls.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 2 posts ] 

» Feed - Atomicorp

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group