Thousands of asl-shun.pl processes

breun · **Posted:** Wed Aug 10, 2011 2:58 pm

I believe this isn't normal:

Code:

# ps x | grep asl-shun\.pl | grep -c add
1331
# ps x | grep asl-shun\.pl | grep -c delete
927

That's over 2000 instances of asl-shun.pl in a process listing. I went and killed them before, but they came back...

aus-city · **Joined:** Thu Oct 26, 2006 11:56 pm **Posts:** 665

Hi Breun,

That is weird.. Does it change hour to hour?

Have you looked in the CP to see if its an attack or something?

Cheers

breun · **Posted:** Thu Aug 11, 2011 5:34 am

We see the number of asl-shun.pl processes build up continuously. It seems that all those processes are waiting for a lock. With thousands of them not much seems to go through and in the end the server runs out of memory. As a temporary workaround we configured the following to run hourly:

Code:

killall --quiet --user root --signal KILL asl-shun.pl

But we'd rather have a real solution of course...

abergman · **Posted:** Thu Aug 11, 2011 9:50 am

We had the same problem, at the same time we had a lot of IPs blocked in iptables, killing the asl-shun.pl processes and changed the $standard_time to 5 min and increasing number of open files seem to have solved it.

breun · **Posted:** Thu Aug 11, 2011 2:08 pm

What $standard_time are you talking about exactly?

ryanz · **Joined:** Mon Dec 20, 2004 2:43 am **Posts:** 67

Yip, we also have this problem since yesterday.

The logs are also filled with this on various ports and ID's:

clamd[29095]: stream(127.0.0.1@1135): Suspect.Bredozip-zippwd-10 FOUND

mikeshinn · **Posted:** Fri Aug 12, 2011 2:49 pm

Are you restarting ossec-hids? And is this on a VPS or a dedicated system?

ryanz · **Joined:** Mon Dec 20, 2004 2:43 am **Posts:** 67

We are running this as part of Plesk on Xen VPS's
We have restarted ossec-hids but it keeps happening.

The logs are also filled with this on various ports and ID's:

clamd[29095]: stream(127.0.0.1@1135): Suspect.Bredozip-zippwd-10 FOUND

mikeshinn · **Posted:** Fri Aug 12, 2011 3:20 pm

Quote:

We have restarted ossec-hids but it keeps happening.

No, don't do that, thats what causes it. When you restart the HIDS, it has no state anymore and will reload all the shuns. You should never restart the HIDS or need to do so, even ASL won't restart it. Theres no need.

ryanz · **Joined:** Mon Dec 20, 2004 2:43 am **Posts:** 67

Mike,

We've had this problem for 18 hours now and we only restarted ossec-hids in the past 2 hours but there was no change.

The only thing that helped was to kill the processes like Breun indicated almost every hour, and we were forced to reboot at a time today as the servers simply couldn't run with thousands of these asl-shun.pl processes.

mikeshinn · **Posted:** Fri Aug 12, 2011 3:32 pm

Can you check to make sure you have all your asl components up to date by running a yum upgrade, followed by an asl -s -f? And what event(s) are triggering the shuns? You can tell by looking at the ASL GUI under blocking, and that will tell you the event ID and signature ID. You can also check the ossec logs which will also tell you this.

Its possible you may be getting hammered with attacks too, so lets see if we can figure out what security events are triggering shuns.

ryanz · **Joined:** Mon Dec 20, 2004 2:43 am **Posts:** 67

That was one of the first things we did this morning.
We ran asl -u and asl -s -f this morning, in fact a few times today.

We've been running ASL 3.0.x for a while and a check shows 3.0.6
ClamAV is updated and we also ran freshclam manually.

mikeshinn · **Posted:** Fri Aug 12, 2011 3:42 pm

And yum upgrade, what does that show?

Also, do the shuns complete? (How long lived are these processes)

ryanz · **Joined:** Mon Dec 20, 2004 2:43 am **Posts:** 67

yum check-update has a few to do but all the asl updates are done regularly.

The only things not updated are the asl kernel updates as we find these give problems with booting and the atomic spamassassin update cause issues with 4psa Spamguardian.

Since the last delete almost a hour ago most asl-shun.pl processes show 0sec with a few 1 to 3 sec.
I must also add that it's now 22h00 here and the servers are running very low access and load so there are much less shuns running compared to during the day when the access is high.

mikeshinn · **Posted:** Fri Aug 12, 2011 5:34 pm

What kinds of events are triggering the shuns?

Thousands of asl-shun.pl processes

Who is online