NGINX + Mongrel Clusters + Multiple Domains = Good Times for All
June 27th, 2007
Problem:
Hosting multiple ruby on rails applications on your server - and of course we want our applications to run at least as fast as an apple fanboy on the day of the iphone release.
Solution:
Mongrel clusters with NGINX as a front-end using virtual hosts all playing nicely on your server. I recommend going with a Virtual Private Server (VPS) which is essentially one step above shared hosting but it gives you full root access. If you aren’t ballin’ on a budget, you could always splurge for a dedicated server - but you should probably wait until your user base demands such luxury. If you do go for a VPS I recommend Rimuhosting.
A word of advice: You pretty much get what you pay for with web servers, you’ll know this if you have ever ran a rails site on a shared server (eg. Dreamhost or Textdrive). Just dont expect to pay $7.00/month and host a site like Digg….or even survive a “digg” for that matter.
I setup my VPS with Fedora Core 6 and had rimu install the “ruby on rails hosting stack” upon order.
Mongrel and Mongrel Clusters
So now you have a server with sudo or root access. The first step is to install mongrel and friends. Mongrel is a webserver by itself and does very well at handling dynamic data. Sidenote: I actually just use mongrel by itself as my webserver on localhost (in replace of webrick).
sudo gem install mongrel sudo gem install mongrel_cluster
The next step is to add a “mongrel” user so we don’t run mongrel as “root”.
sudo /usr/sbin/adduser -r mongrel
Next we set our mongrel user as the owner of both of our applications. This is important since our Mongrel servers need to have access to create PID log files in each of our applications (eg. /var/www/testapp1/log/)
sudo chown -R mongrel:mongrel /var/www/testapp1 sudo chown -R mongrel:mongrel /var/www/testapp2
Seeing that we have installed the mongrel cluster gem above, we can now configure the a cluster of mongrels for each of our applications. A mongrel cluster will configure and control several mongrel servers, or groups of mongrel servers. By running the configuration command below for each of our apps, it will create a mongrel_cluster.yml file in our config folder within each app directory.
In /var/www/testapp1/
sudo mongrel_rails cluster::configure -e production \
-p 7000 -N 2 -c /var/www/testapp1 -a 127.0.0.1 \
--user mongrel --group mongrel
In /var/www/testapp2/
sudo mongrel_rails cluster::configure -e production \
-p 8000 -N 2 -c /var/www/testapp2 -a 127.0.0.1 \
--user mongrel --group mongrel
Dissecting what we just did here:
- “-p 8000” and “-p 7000” - setting the starting port number for each application
- “-N 2” - specifies the amount of mongrels in each cluster. So this means testapp1 will run on 7000 and 7001 while testapp2 will run on 8000 and 8001
- “-c /var/www/testapp1” and “-c /var/www/testapp2” specifies the location of each application.
- “-a 127.0.0.1” - means that it will be running on localhost (your server)
- “–user mongrel –group mongrel” - specifies the user and group that we created above.
NGINX, our front-end friend
Although mongrel is great at serving up the dynamic stuff, it is suggested that a load balancing reverse proxy server should be used as the front-end to handle the static and cached files. Enter NGINX. NGINX is lightweight front-end server (like Apache or Lighttpd). Additionally, NGINX will handle our virtual hosts which allow us to direct traffic to the correct port with multiple domains. At first I was using Apache (which was a pain to setup), but decided to go with NGINX after Zed Shaw (creator of mongrel) recommended it when I was asking an apache question on the Seattle.rb mailing list.
To setup NGINX we first install it on our server.
sudo yum install nginx
Once NGINX is installed, all we need to do is edit our NGINX configuration file at /etc/nginx/nginx.conf. Since we are configuring NGINX to handle two domains, we need to setup two different “servers” as virtual hosts in the configuration file - these have been given the labels “mongrel1” and “mongrel2” in the configuration file below. Also be sure to specify the correct ports for each application. Remember that earlier we set our mongrel config file for testapp1 to use ports 7000 and 7001, and set testapp2 to use ports 8000 and 8001. To map our labels to our applications, we set mongrel1 to use ports 7000 and 7001, and set mongrel2 to 8000 and 8001. With this mapping we can then put all of testapp1 config info in the server block that pertains to mongrel1 and the same goes for testapp2 with mongrel2.
Below is a configuration file from Ezra at Brainsplat with the variables modified by me for this writeup.
user jordan;
# number of nginx workers
worker_processes 4;
# pid of nginx master process
pid /var/run/nginx.pid;
# Number of worker connections. 1024 is a good default
events {
worker_connections 1024;
}
# start the http module where we config http access.
http {
# pull in mime-types. You can break out your config
# into as many include's as you want to make it cleaner
include /etc/nginx/mime.types;
# set a default type for the rare situation that
# nothing matches from the mimie-type include
default_type application/octet-stream;
# configure log format
log_format main '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
# main access log
access_log /var/log/nginx_access.log main;
# main error log
error_log /var/log/nginx_error.log debug;
# no sendfile on OSX
sendfile on;
# These are good default values.
tcp_nopush on;
tcp_nodelay off;
# output compression saves bandwidth
gzip on;
gzip_http_version 1.0;
gzip_comp_level 2;
gzip_proxied any;
gzip_types text/plain text/html text/css application/x-javascript text/xml application/xml
application/xml+rss text/javascript;
# this is where you define your mongrel clusters.
# you need one of these blocks for each cluster
# and each one needs its own name to refer to it later.
upstream mongrel1 {
server 127.0.0.1:7000;
server 127.0.0.1:7001;
}
upstream mongrel2 {
server 127.0.0.1:8000;
server 127.0.0.1:8001;
}
# the server directive is nginx's virtual host directive.
server {
# port to listen on. Can also be set to an IP:PORT
listen 80;
# Set the max size for file uploads to 50Mb
client_max_body_size 50M;
# sets the domain[s] that this vhost server requests for
server_name www.testapp1.com testapp1.com;
# doc root
root /var/www/testapp1/public;
# vhost specific access log
access_log /var/log/nginx.vhost.access.log main;
# this rewrites all the requests to the maintenance.html
# page if it exists in the doc root. This is for capistrano's
# disable web task
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html last;
break;
}
location / {
# needed to forward user's IP address to rails
proxy_set_header X-Real-IP $remote_addr;
# needed for HTTPS
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect false;
proxy_max_temp_file_size 0;
# If the file exists as a static file serve it directly without
# running all the other rewite tests on it
if (-f $request_filename) {
break;
}
# check for index.html for directory index
# if its there on the filesystem then rewite
# the url to add /index.html to the end of it
# and then break to send it to the next config rules.
if (-f $request_filename/index.html) {
rewrite (.*) $1/index.html break;
}
# this is the meat of the rails page caching config
# it adds .html to the end of the url and then checks
# the filesystem for that file. If it exists, then we
# rewite the url to have explicit .html on the end
# and then send it on its way to the next config rule.
# if there is no file on the fs then it sets all the
# necessary headers and proxies to our upstream mongrels
if (-f $request_filename.html) {
rewrite (.*) $1.html break;
}
if (!-f $request_filename) {
proxy_pass http://mongrel1;
break;
}
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /var/www/testapp1/public;
}
}
# This server is setup for ssl. Uncomment if
# you are using ssl as well as port 80.
server {
# port to listen on. Can also be set to an IP:PORT
listen 80;
# Set the max size for file uploads to 50Mb
client_max_body_size 50M;
# sets the domain[s] that this vhost server requests for
server_name www.testapp2.com testapp2.com;
# doc root
root /var/www/testapp2/public;
# vhost specific access log
access_log /var/log/nginx.vhost.access.log main;
# this rewrites all the requests to the maintenance.html
# page if it exists in the doc root. This is for capistrano's
# disable web task
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html last;
break;
}
location / {
# needed to forward user's IP address to rails
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect false;
proxy_max_temp_file_size 0;
# If the file exists as a static file serve it directly without
# running all the other rewite tests on it
if (-f $request_filename) {
break;
}
# check for index.html for directory index
# if its there on the filesystem then rewite
# the url to add /index.html to the end of it
# and then break to send it to the next config rules.
if (-f $request_filename/index.html) {
rewrite (.*) $1/index.html break;
}
# this is the meat of the rails page caching config
# it adds .html to the end of the url and then checks
# the filesystem for that file. If it exists, then we
# rewite the url to have explicit .html on the end
# and then send it on its way to the next config rule.
# if there is no file on the fs then it sets all the
# necessary headers and proxies to our upstream mongrels
if (-f $request_filename.html) {
rewrite (.*) $1.html break;
}
if (!-f $request_filename) {
proxy_pass http://mongrel2;
break;
}
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /var/www/testapp2/public;
}
}
}sudo ./nginx start
Now our mongrel servers:
In /var/www/testapp1/
sudo mongrel_rails cluster::start
In /var/www/testapp2/
sudo mongrel_rails cluster::start
If all went well, you should be able to see all of your processes using:
ps aux | grep mongrel ps aux | grep nginx
You should have 4 mongrel processes running on ports 7000, 7001, 8000 and 8001. For NGINX, you should have a master process and 4 worker processes (if you put “4” for “worker_processes” in nginx.conf).
Assuming you have all of your DNS stuff configured correctly, if you go to each of your domains now, you should connect to the correct application. For example: http://www.testapp1.com should be forwarded by NGINX to go to the mongrel cluster serving ports 7000 and 7001.
Showcase
So you can see the actual speed of multiple domains on a VPS, these are the sites that are currently hosted on my rimuhosting server ($30/mo MicroVPS2 option):
Note: See Caution section below.
Extra Credit:
Go back to the mongrel home page (toward the bottom of the page) to check out how to initialize the server on boot and to setup a command to launch all of your mongrel clusters for both of your applications at once. (ie. /etc/init.d/mongrel_cluster start)
Further Reading on NGINX + Mongrel:
- Ezra’s setup
- Ryan Norbauer’s setup
- A rimuhost forum post
- Deploying Rails Applications Book by Ezra Zygmuntowicz and Bruce Tate
Caution: this is amateur hour
I am far from a sysAdmin guru - so I am positive that there are better ways to do this. If you know of a better one, please let me know in the comments section. Furthermore, as of now, I don’t have a large user base using each of my applications so I’ll have to update this article if I ever reach that stage to report on the speed.