store | blogs | forums | twitter | facebook | wiki | downloads | support portal
Atomic Secure Linux
It is currently Wed Oct 22, 2014 11:01 pm

» Feed - Atomicorp

All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 39 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Sat Jul 16, 2011 3:52 pm 
Offline
Forum User
Forum User

Joined: Sat Jan 20, 2007 6:57 pm
Posts: 83
After upgrading to 3.0 I have noticed that occasionally the CPU usage will ramp up from my usual average of under 10% to around 20% and stay at that level until I restart asl.
Any ideas why or how to determine why?


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Sat Jul 16, 2011 4:27 pm 
Offline
Long Time Forum Regular
Long Time Forum Regular

Joined: Sat Aug 20, 2005 9:30 am
Posts: 2812
Location: The Netherlands
Have you checked top to see what process is using the CPU time?

_________________
Lemonbit Internet Dedicated Server Management


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Tue Jul 19, 2011 2:02 pm 
Offline
Forum Regular
Forum Regular

Joined: Sat Dec 11, 2004 2:33 pm
Posts: 239
Location: South Africa
I am having the same issue

I have noticed that /var/ossec/bin/ossec-syscheckd has been running for hours and CPU usage is between 80 - 100%
The only way to get it down is to run asl -f -s or hit the fix button in the web interface.

I tried changing the values in

/var/ossec/etc/ossec.conf

<syscheck>
<frequency>72000</frequency>
</syscheck>

and

/var/ossec/etc/internal_options.conf

syscheck.sleep=5
syscheck.sleep_after=8

PS: I also find the web interface alot slower.


Attachments:
ossec.png
ossec.png [ 4.92 KiB | Viewed 8937 times ]

_________________
Mark Brindley
2Large Networks - Web solutions that work
Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Tue Jul 19, 2011 2:20 pm 
Offline
Forum User
Forum User

Joined: Sat Jan 20, 2007 6:57 pm
Posts: 83
kram wrote:
I am having the same issue

I have noticed that /var/ossec/bin/ossec-syscheckd has been running for hours and CPU usage is between 80 - 100%
The only way to get it down is to run asl -f -s or hit the fix button in the web interface.

I tried changing the values in

/var/ossec/etc/ossec.conf

<syscheck>
<frequency>72000</frequency>
</syscheck>

and

/var/ossec/etc/internal_options.conf

syscheck.sleep=5
syscheck.sleep_after=8

PS: I also find the web interface alot slower.


Yes I think for me the culprit was ossec-syscheckd
Also for me the web interface is much slower.


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Tue Jul 19, 2011 2:37 pm 
Offline
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
User avatar

Joined: Thu Feb 07, 2008 7:49 pm
Posts: 3656
Location: Chantilly, VA
For ossec-syscheckd the amount of CPU it uses is caused by the number of files its monitoring and the time it takes to complete is driven by the sleep values, and the speed of your I/O bus and drives.

So, first check the directories you are monitoring to make sure you actually want to monitor all the files and directories syscheckd is configured to monitor for changes. For example, some Splunk installations put their logs in /opt, and /opt is monitored in its entirety by default. In configurations like this (putting your logs in /opt instead of /var, which syscheckd is configured to not monitor), would cause the HIDS system to monitor a *lot* of files that change repeatedly (and likely do not need to monitored for changes). This will drive up utilization as syscheckd is doing more work. So, check your monitored directories to ensure that you are not monitoring files you should not. Otherwise, if you have a situation like this, then you will see syscheckd doing a lot more work (unnecessarily).

syscheckd is also a prisoner of your I/O bus and drive speeds. If you have a slow or busy bus, or slow drives, it will take a lot longer for file integrity checks to finish. So if you do not have a lot of files being monitored by syscheckd, and its taking a long time to complete, check your I/O speeds for your drives and partitions for any contention.

_________________
Michael Shinn
Atomicorp - Security For Everyone

Co-Author of Troubleshooting Linux Firewalls.


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Tue Jul 19, 2011 2:37 pm 
Offline
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
User avatar

Joined: Thu Feb 07, 2008 7:49 pm
Posts: 3656
Location: Chantilly, VA
Quote:
Also for me the web interface is much slower.


Could you elaborate? Do you mean when you first log in, or do you mean after all the windows have loaded? And are you using RC4 or an earlier beta?

_________________
Michael Shinn
Atomicorp - Security For Everyone

Co-Author of Troubleshooting Linux Firewalls.


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Wed Jul 20, 2011 9:41 am 
Offline
Forum Regular
Forum Regular

Joined: Tue May 10, 2005 1:24 pm
Posts: 193
I am having this problem too. Could you please explain the difference in what is monitored in ASL 2 and ASL 3 by default as I have always just used the default that is set by ASL? This started yesterday after I upgraded to 3 and I never had this problem with 2. It is fine for a while and then the CPU jumps and stays up until I run asl -s -f.


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Wed Jul 20, 2011 10:33 am 
Offline
Forum Regular
Forum Regular

Joined: Sat Mar 28, 2009 6:58 pm
Posts: 865
Location: Germany
I can see that similar behaviour as well.
In my case the process ossec-analysisd is causing the "new" load and it's constant.
The disk load is low and they are running fine.
How to tweak?
Thanks


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Wed Jul 20, 2011 11:15 am 
Offline
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
User avatar

Joined: Thu Feb 07, 2008 7:49 pm
Posts: 3656
Location: Chantilly, VA
Quote:
In my case the process ossec-analysisd is causing the "new" load and it's constant.


Can you be more specific, how much cpu time is it using?

_________________
Michael Shinn
Atomicorp - Security For Everyone

Co-Author of Troubleshooting Linux Firewalls.


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Wed Jul 20, 2011 11:23 am 
Offline
Forum Regular
Forum Regular

Joined: Sat Mar 28, 2009 6:58 pm
Posts: 865
Location: Germany
round about 10%. It jumps/varies constantly between 1-10%. but the frequency is so high that in average it consums about 10%.
What can I provide to give you more details?


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Wed Jul 20, 2011 11:30 am 
Offline
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
User avatar

Joined: Thu Feb 07, 2008 7:49 pm
Posts: 3656
Location: Chantilly, VA
10%? Thats probably normal then, its looking at a lot more events than 2.2 was.

_________________
Michael Shinn
Atomicorp - Security For Everyone

Co-Author of Troubleshooting Linux Firewalls.


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Wed Jul 20, 2011 12:18 pm 
Offline
Forum Regular
Forum Regular

Joined: Sat Mar 28, 2009 6:58 pm
Posts: 865
Location: Germany
Ok thanks. Do you have an approximate load increase deviation rate between ASL 2.2 and ASL 3.0?

In my case it's a "dramatic" difference because my server is not heavy loaded at any time.
That means ...my average CPU load with ASL 2.2 was ~5-6%
Now with ASL 3.0 it is ~10-12%
In total numbers that's an increase of ~100% more load.
But at the end just 10% more load for the system.
And if I have to take that medicine to be more secure I will take it for sure :)


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Wed Jul 20, 2011 12:26 pm 
Offline
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin

Joined: Wed Dec 31, 1969 8:00 pm
Posts: 7939
Location: earth
Ah so one big change from 2.2 to 3.0 is that it can create diff's of files it watches & email you the results (so instead of just telling you a file changes, it can tell you exactly what changed).

By default its looking at /etc, and /var/www/vhosts (for files ending in .php, .js, and .html). Thats some additional file overhead, so you could start by turning the "Report" flag off on those 2 directories in the file integrity checks. I'll bet you that its hitting a big /var/www/vhosts, and that could be the culprit here.


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Wed Jul 20, 2011 2:07 pm 
Offline
Forum Regular
Forum Regular

Joined: Sat Mar 28, 2009 6:58 pm
Posts: 865
Location: Germany
Thanks Scott,

I will give it a try and disable this for /var/www/vhosts.
But as you said that it's normal I will re-enable it for better security afterwards.
If I need some additional CPU power I could disable it then.


Top
 Profile  
 
 Post subject: Re: CPU useage increases occasionally after upgrade to 3.0
Unread postPosted: Wed Jul 20, 2011 3:01 pm 
Offline
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
User avatar

Joined: Thu Feb 07, 2008 7:49 pm
Posts: 3656
Location: Chantilly, VA
You may not even want to monitor /var/www/vhosts. Keep in mind that 3.0 does real time change detection, so thats a pretty big tree. The code can handle it, but your CPU might complain. Currently we only alert (in /var/www/vhosts) if files that end in .php, .js and .html change. On a big box with lots of virtual domains, you may not want that, you may only want to monitor your own stuff.

So, if thats the case for you, either disable it entirely (delete it), or just monitor the directories for specific sites for changes that you have higher security requirements (like you own sites for example). Don't be afraid to add in ignores either for things like temp and cache directories if you think you might have .php, .js, etc. stuff in there you dont care about. The file integrity monitor exists to tell you if something changes, so you probably dont want to waste cycles on stuff you dont care about. ;-)

_________________
Michael Shinn
Atomicorp - Security For Everyone

Co-Author of Troubleshooting Linux Firewalls.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 39 posts ]  Go to page 1, 2, 3  Next

» Feed - Atomicorp

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group