nginx serving large static files

premierhosting · **Joined:** Wed Aug 04, 2010 2:52 pm **Posts:** 256

Howdy,

Anyone using nginx? I'm serving some 10-30mb files at peak times up to 250-300 requests per second, if I'm reading these RDD'd /status logs correctly. These are on VPS servers, which may mean some network settings aren't available.

It mostly works, but sometimes the massive simultaneous connections overwhelms the system, and I'm wondering if anyone here is doing something like that and has suggestions for nginx.conf or sysctl.conf.

Most info out there is on using nginx as a proxy, and tuning the proxy settings. I'm not using it as a proxy, I'm using nginx as a web server.

Turning sendfile on seemed to help dramatically. Setting sendfile_max_chunk to 1m helped too. Right now I'm looking at the TCP settings like tcp_fin_timeout to see if optimizations can be had at that level. There's a lot of conflicting info out there so I figured if someone had it dialed in, they'd be lurking here

biggles · **Posted:** Wed Jan 18, 2012 5:36 am

Almost sounds lika a mission for a CDN...

faris · **Joined:** Thu Dec 09, 2004 11:19 am **Posts:** 1843

biggles wrote:

Almost sounds lika a mission for a CDN...

I second that.

To be honest I'm not (yet) sure what all the fuss is about regarding nginx. A CDN would seem like the best option in this case.

Ooh! I've just thought of a new project: An auto-CDN Plugin for Plesk (auto-uploads changed files in specified directories ever X minutes)

I'm aware that some CDNs can connect via FTP and do this themselves, but Amazon (AFAIK) doesn't, and that's what I use.

dayo · **Joined:** Sun Jul 12, 2009 1:33 pm **Posts:** 158

I tried to post a response but it seems something in the setup here doesn't like Nginx

. The ASL threat mitigation measures kicked in as soon as I submitted the post and I got frozen out. I am now using a proxy server to post this as I am no longer able to browse the site otherwise

As I don't know what exactly it was in my post that triggered the alarm, I'll just say the best place to get information on Nginx is the mailing list.

Good luck!

premierhosting · **Joined:** Wed Aug 04, 2010 2:52 pm **Posts:** 256

Cost to benefit, CDNs still too expensive when lighter resources will do. Assuming the lighter resources work

faris · **Joined:** Thu Dec 09, 2004 11:19 am **Posts:** 1843

I can't argue with that.

dayo · **Joined:** Sun Jul 12, 2009 1:33 pm **Posts:** 158

After thinking about it I reckon the reason I got frozen out was because I had five links in my original post which must be above a certain magic number on the threat/spam mitigation front so I have stripped them all out in the reproduction of the post below

Quote:

A CDN would obviously help but if you are trying to get more out of your setup, you might want to make sure you are eliminating disc I/O bottlenecks.

In this case, what you want to set up is a system where the files are saved into RAM and served from there. This can be done using a set of modules such as the Memc, srcache and perhaps, the Lua modules (documentation on each on the Nginx wiki site). When a request hits, Nginx can check to see if the file is in the memcache and serve it from there or get it from the filesystem if not and update the cache. This can be local or on (a) separate VPS installation(s) serving as (a) memcache server(s)

The company behind what is apparently one of the largest e-commerce sites in China have been behind some interesting developments with Nginx with these modules along with others that make it more of a web application framework instead of just a webserver and they have been releasing these modules as open source. Basically, you can look at Nginx as a scriptable application that can interact directly with data management systems such as MySQL, PgSQL, Memcache, Redis, MongoDB etc. A lot of stuff don't need to hit a backend application and can be run swftly and with low memory impact directly within the webserver space itself. Your imagination is your limit and people are doing all sorts of wonderful things. That is what the fuss is all about.

There are some basic things you need to look into as well such as the Worker Processes and Worker Connections settings etc.

The best place to find information and advice is the Nginx Mailing List as the developers themselves are very active there.

premierhosting · **Joined:** Wed Aug 04, 2010 2:52 pm **Posts:** 256

Good info dayo. I'm going to check into the file caching options. There's a high peak and a long tail for which content is accessed, so if the caching system could prioritize into RAM the most often accessed files for any given time frame, it could really be slick.

Do you think doing bandwidth limiting per client is helpful? The one area that it looks like I can't optimize is the tcp_fin_wait, as the virtual server environment is relying on the underlying tcp stack, I think.

dayo · **Joined:** Sun Jul 12, 2009 1:33 pm **Posts:** 158

I don't think this is a suitable forum to look for detailed responses on this sorts of issues regarding Nginx at this point and as said, your best bet is to post your queries to the Nginx mailing list.

premierhosting · **Joined:** Wed Aug 04, 2010 2:52 pm **Posts:** 256

OK, no problem.

mandi22 · **Joined:** Sat Oct 06, 2007 8:01 am **Posts:** 11

premierhosting wrote:

OK, no problem.

Seriously,
Nginx is too slow! I am gonna try installing a second Apache instance for serving static content.
At the moment my actual apache is loaded with PHP and other modules and is faster serving files from the same server than nginx which is configured with no mods at all.
Yes, nginx is very lite. I can barely see it on 'top' but I would like to see how apache plain with no mods behave in comparison to nginx plain.

scott · **Posted:** Fri Jan 27, 2012 10:31 am

Thats on my list with the apache benchmark thread here: viewtopic.php?f=1&t=5576. I just finished up the cgi/suphp/fcgi performance thread earlier this week, so I've got a baseline to compare it against.

nginx serving large static files

Who is online