store | blogs | forums | twitter | facebook | wiki | downloads | support portal
Atomic Secure Linux
It is currently Thu Oct 30, 2014 9:41 am

» Feed - Atomicorp

All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 6 posts ] 
Author Message
 Post subject: swap is killing servers instead of saving them
Unread postPosted: Fri Oct 21, 2011 10:41 am 
Offline
Forum Regular
Forum Regular

Joined: Sun Mar 29, 2009 6:52 pm
Posts: 350
Ok. Over the last few months I am thinking of disabling swap on my servers.
It has turned out to be a real pain in the *ss.

I see servers without having memory pressure since 20-50% of their memory is shown as cached and not us active and I have mysql going to swap and writing even 500mbyte taking the disk into death.
I have also found many people which have the same issues.
Offcourse some ammount of swap is good for emergency but every crash I have suffered over the last 3 years is due to swap.

A nice article explaining the numa arch.
Code:
http://jcole.us/blog/archives/2010/09/28/mysql-swap-insanity-and-the-numa-architecture/

_________________
Hello IT.
Phone : Blah Blah ....
Have you tried turning it on and off again ?
Phone : Blah Blah ....
....
I'm sorry, are you from the Past ?!
http://www.youtube.com/watch?v=-E4fm4Wqego


Top
 Profile  
 
 Post subject: Re: swap is killing servers instead of saving them
Unread postPosted: Tue Oct 25, 2011 6:45 pm 
Offline
Forum Regular
Forum Regular

Joined: Sun Mar 29, 2009 6:52 pm
Posts: 350
Since today I have disabled swap. I will try to push as much as I can the server and I'll let you know !

_________________
Hello IT.
Phone : Blah Blah ....
Have you tried turning it on and off again ?
Phone : Blah Blah ....
....
I'm sorry, are you from the Past ?!
http://www.youtube.com/watch?v=-E4fm4Wqego


Top
 Profile  
 
 Post subject: Re: swap is killing servers instead of saving them
Unread postPosted: Tue Oct 25, 2011 7:13 pm 
Offline
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin

Joined: Wed Dec 31, 1969 8:00 pm
Posts: 7945
Location: earth
Is this on a x86_64 or i386 box? What does your /etc/my.cnf look like?

Oh and what kernel are you on?


Top
 Profile  
 
 Post subject: Re: swap is killing servers instead of saving them
Unread postPosted: Tue Oct 25, 2011 7:22 pm 
Offline
Forum Regular
Forum Regular

Joined: Sun Mar 29, 2009 6:52 pm
Posts: 350
scott wrote:
Is this on a x86_64 or i386 box? What does your /etc/my.cnf look like?

Oh and what kernel are you on?


Thanks for your reply Scott !

This is a x86_64.

Kernel version is
Code:
Linux server.hostname.com 2.6.32.43-6.art.x86_64 #1 SMP Thu Jul 14 14:14:48 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux


my.cnf

Code:
[mysqld]
local-infile=0
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
old_passwords=1

thread_concurrency=4
max_connections = 150
max_user_connections = 150
key_buffer = 150M
myisam_sort_buffer_size = 64M
join_buffer_size = 2M
read_buffer_size = 1M
sort_buffer_size = 1M
table_cache = 1024
thread_cache_size = 128
interactive_timeout = 25
wait_timeout = 1800
connect_timeout = 10
max_allowed_packet = 1M
max_connect_errors = 1000
query_cache_limit = 1M
query_cache_size = 32M
query_cache_type = 1
tmp_table_size = 16M
max_heap_table_size = 16M

# symbolic-links=0

_________________
Hello IT.
Phone : Blah Blah ....
Have you tried turning it on and off again ?
Phone : Blah Blah ....
....
I'm sorry, are you from the Past ?!
http://www.youtube.com/watch?v=-E4fm4Wqego


Top
 Profile  
 
 Post subject: Re: swap is killing servers instead of saving them
Unread postPosted: Wed Oct 26, 2011 10:20 am 
Offline
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin

Joined: Wed Dec 31, 1969 8:00 pm
Posts: 7945
Location: earth
Ok so, can you try this to see if it helps:

Install numactl:
yum install numactl

back up /usr/bin/mysqld_safe

Find this line (this is if you're using mysql 5.5) in /usr/bin/mysqld_safe

cmd="`mysqld_ld_preload_text`$NOHUP_NICENESS"

add after that line (dont replace it!):

cmd="/usr/bin/numactl --interleave all $cmd"

and restart mysqld


Top
 Profile  
 
 Post subject: Re: swap is killing servers instead of saving them
Unread postPosted: Thu Oct 27, 2011 8:55 am 
Offline
Forum Regular
Forum Regular

Joined: Sun Mar 29, 2009 6:52 pm
Posts: 350
Well I can try that.
But its not only mysql. I see 2gb cache and 380mb on swap. This 380mb are from all around services. Apache, mysql, qmail, clamav etc ...

So even if I solve the mysql problem I would still pretty much have the same issue.

_________________
Hello IT.
Phone : Blah Blah ....
Have you tried turning it on and off again ?
Phone : Blah Blah ....
....
I'm sorry, are you from the Past ?!
http://www.youtube.com/watch?v=-E4fm4Wqego


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 

» Feed - Atomicorp

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group