Disclaimer
By no means is this suppose to be a definitive guide to setting up nGinX on AWS as a Global Server Load Balancer (GSLB). Its goal is to serve as a reminder of what I have done to satisfy my requirements. That said, please feel free to comment and/or correct any part, as I am an nGinX novice.
Requirements
Provide a means to route global traffic accurately based on the source IP of the client. This means nGinX must be able to determine anything from the source IP's country code to the organizations code the IP is originating from.
Since nGinX does not work at the dns level (aka over port 53) - the magic that happens behind the scenes where traffic is routed and the users is unaware can not take place. (I will go into this in my next blog entry when I discuss setting up Stingray Traffic Manager [STM] as a Global Load Balancer [GLB] on AWS/EC2)
nGinX must route all requests individually.
Step by Step
1) Get the source code
Create a source directory
> sudo mkdir /etc/nginx/src
Go to the newly created source directory
> cd /etc/nginx/src
Get the source code
> wget http://nginx.org/download/nginx-1.3.10.tar.gz
Expand the compressed file
> gunzip nginx-1.3.10.tar.gz
> tar xvf nginx-1.3.10.tar
2) Get the global IP data sets
Maxmind provides a number of GeoIP databases at different granularities. Several are supplied at no cost. You can use yum to install a no cost lite GeoIP database.
> yum install GeoIP-devel
The GeoIP database is placed: /usr/share/GeoIP/GeoIP.dat
If the yum version of the database is outdated, it is possible to get the latest from the maxmind site with the following steps:
> mv /usr/share/GeoIP/GeoIP.dat /usr/share/GeoIP/GeoIP.dat_bak
> cd /usr/share/GeoIP/
> wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
> gunzip GeoIP.dat.gz
3) Compile
If there is an initial installation of nGinX, the newly compiled version can be based from it. Its also a good opportunity to change the directory structure around so that nGinX is centralized more. I personally find it annoying to hunt up and down the file structure in order to look at logs then changes configs, etc. However, I'm sure this is done for a reason so for a production deployment, I'd suggest leaving it how it is.
Lets start with pulling the current configs from the installed version - manually formatted:
> nginx -V
nginx version: nginx/1.3.9
built by gcc 4.6.2 20111027 (Red Hat 4.6.2-2) (GCC)
TLS SNI support enabled
configure arguments:
--prefix=/etc/nginx/
--sbin-path=/usr/sbin/nginx
--conf-path=/etc/nginx/nginx.conf
--error-log-path=/var/log/nginx/error.log
--http-log-path=/var/log/nginx/access.log
--pid-path=/var/run/nginx.pid
--lock-path=/var/run/nginx.lock
--http-client-body-temp-path=/var/cache/nginx/client_temp
--http-proxy-temp-path=/var/cache/nginx/proxy_temp
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp
--http-scgi-temp-path=/var/cache/nginx/scgi_temp
--user=nginx
--group=nginx
--with-http_ssl_module
--with-http_realip_module
--with-http_addition_module
--with-http_sub_module
--with-http_dav_module
--with-http_flv_module
--with-http_mp4_module
--with-http_gzip_static_module
--with-http_random_index_module
--with-http_secure_link_module
--with-http_stub_status_module
--with-mail
--with-mail_ssl_module
--with-file-aio
--with-ipv6
--with-cc-opt='-O2 -g'
From the source directory the was created earlier, run the following command to get the make ready. Notice the alterations to path variables and new modules. These next few commands can take some time to execute. Also, if you encounter errors - please see the Troubleshooting section below to help resolve the issue.
> ./configure --prefix=/etc/nginx/ --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/etc/nginx/logs/error.log --http-log-path=/etc/nginx/logs/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/etc/nginx/cache/client_temp --http-proxy-temp-path=/etc/nginx/cache/proxy_temp --http-fastcgi-temp-path=/etc/nginx/cache/fastcgi_temp --http-uwsgi-temp-path=/etc/nginx/cache/uwsgi_temp --http-scgi-temp-path=/etc/nginx/cache/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-mail --with-mail_ssl_module --with-file-aio --with-ipv6 --with-cc-opt='-O2 -g' --with-http_geoip_module --with-debug
> make
> make install
Done! ~~> This part!
4) Start & Confirm
Since nGinX is hosts everything on port 80 and has a default homepage - it can be tested by simply starting and pointing to the webserver hosting nGinX.
> service nginx start
Open a browser and navigate to the nginx server. If you see the homepage, we are good to go.
5) Add GSLB/Reverse Proxy
First and foremost, you will need to define at least 2 endpoints to distribute requests to and a-record (literally) for the primary domain. Consider the following:
www.example.com --> A-Record --> nginx.ip.address
www1.example.com --> CNAME --> 70.70.70.1
www2.example.com --> CNAME --> 70.70.70.2
Add the following to the nGinX config. It will setup the geoIp database, geo targeting as per country codes [NA-North America & AS-Asia] and a reverse proxy.
geoip_country /usr/share/GeoIP/GeoIP.dat;
if($geoip_city_continent_code=NA){proxy_pass http://ad-1;}
if($geoip_city_continent_code=AS){proxy_pass http://ad-2;}
upstream ad-1 { server www1.example.com; }
upstream ad-2 { server www2.example.com; }
Yes - that's it!
6) Test
There are other configurations you can setup to make it easier to test. However, for this config, you will need access to a browser in North America and in Asia.
Be sure to clear your browser cache before testing.
Troubleshooting
This is an assortment of issues and thier solutions that I ran into while trying to build nGinX.
Find nginx without all the noise
A lot of null data is printed making it hard to read the page
FIX: > find / -name nginx 2>/dev/null
WinSCP - make it work with sudo
The files you are trying to touch are owned by root and WinSCP won't let you do anything.
FIX: todo
Missing gcc compiler
checking for OS + Linux 3.2.34-55.46.amzn1.x86_64 x86_64checking for C compiler ... not found./configure: error: C compiler gcc is not found
FIX: > yum install gcc
Missing PCRE
./configure: error: the HTTP rewrite module requires the PCRE library.You can either disable the module by using --without-http_rewrite_moduleoption, or install the PCRE library into the system, or build the PCRE librarystatically from the source with nginx by using --with-pcre= option.
FIX: http://www.asep.us/2011/05/30/nginx-instalation-pcre-library-not-found/
Missing make
make is missing---
FIX: > yum install make
Missing g++
compile error
FIX: > yum install gcc-c++
PCRE not making after g++ is installed
files have been corrupted after the make and must be cleaned up
FIX: from the /usr/local/src/pcre-8.10 dir
> make distclean
> ./configure
> make
> make install
nGinX ./configure: error: the HTTP gzip module requires the zlib library
need to install zlib
FIX: find out what is avail
> yum search zlib
> yum install zlib-devel.x86_64
nGinX ./configure: error: SSL modules require the OpenSSL library
FIX: > yum install openssl-devel.x86_64
PHP Is missing
Fix: > yum install php-fpm.x86_64
Install Aptitude
FIX: Download and Install
> wget http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm
> rpm -Uvh rpmforge-release*rpm
How to STOP nGinX (3 options)
> net service nginx stop
> /usr/sbin/nginx -s stop
> kill -QUIT $( cat /usr/local/nginx/logs/nginx.pid )
.jpg)
