This tutorial explains how to set up a two-node load balancer IN AN active/passive configuration with HAProxy and heartbeat.The load balancer sits between the user and two (or more) backend Apache web servers that hold the same content. Not only does the load balancer distribute the requests to the two backend Apache servers, it also checks the health of the backend servers. If one of them is down, all requests will automatically be redirected to the remaining backend server. In addition to that, the two load balancer nodes monitor each other using heartbeat, and if the master fails, the slave becomes the master, which means the users will not notice any disruption of the service. HAProxy is session-aware, which means you can use it with any web application that makes use of sessions (such as forums, shopping carts, etc.).
1. Preliminary Note
Load Balancer 1: lb1.example.com, IP address: 192.168.0.100
Load Balancer 2: lb2.example.com, IP address: 192.168.0.101
Web Server 1: http1.example.com, IP address: 192.168.0.102
Web Server 2: http2.example.com, IP address: 192.168.0.103
We also need a virtual IP address that floats between lb1 and lb2: 192.168.0.99
2. Preparing The Backend Web Servers
We will configure HAProxy as a transparent proxy, i.e., it will pass on the original user's IP address in a field called X-Forwarded-For to the backend web servers. Of course, the backend web servers should log the original user's IP address in their access logs instead of the IP addresses of our load balancers. Therefore we must modify the LogFormat line in /etc/apache2/apache2.conf and replace %h with %{X-Forwarded-For}i:
http1/http2
# vim /etc/apache2/apache2.conf
#LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
Also, we will configure HAProxy to check the backend servers' health by continuously requesting the file check.txt (translates to /var/www/check.txt if /var/www is your document root) from the backend servers. Of course, these requests would totally bloat the access logs and mess up your page view statistics (if you use a tool like Webalizer or AWstats that generates statistics based on the access logs).
Therefore we open our vhost configuration (in this example it's in /etc/apache2/sites-available/default) and put these two lines into it (comment out all other CustomLog directives in your vhost configuration):
# vim /etc/apache2/sites-available/default
SetEnvIf Request_URI "^/check\.txt$" dontlog
CustomLog /var/log/apache2/access.log combined env=!dontlog
Note:- This configuration above prevents that requests to check.txt get logged in Apache's access log
Afterwards we restart Apache:
# /etc/init.d/apache2 restart
... and create the file check.txt (this can be an empty file):
# touch /var/www/check.txt
We are finished already with the backend servers; the rest of the configuration happens on the two load balancer nodes.
3. Installing HAProxy
lb1/lb2
We can install HAProxy as follows:
# aptitude install haproxy
4. Configuring The Load Balancers
We back up the original /etc/haproxy/haproxy.cfg and create a new one like this:
lb1/lb2
# cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg_orig
# echo > /etc/haproxy/haproxy.cfg
# vim /etc/haproxy/haproxy.cfg
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
#log loghost local0 info
maxconn 4096
#debug
#quiet
user haproxy
group haproxy
defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
maxconn 2000
contimeout 5000
clitimeout 50000
srvtimeout 50000
listen webfarm 192.168.0.99:80
mode http
stats enable
stats auth "username":"password"
balance roundrobin
cookie JSESSIONID prefix
option httpclose
option forwardfor
option httpchk HEAD /check.txt HTTP/1.0
server webA 192.168.0.102:80 cookie A check
server webB 192.168.0.103:80 cookie B check
Afterwards, we set ENABLED to 1 in /etc/default/haproxy:
# vim /etc/default/haproxy
5. Setting Up Heartbeat
We've just configured HAProxy to listen on the virtual IP address 192.168.0.99, but someone has to tell lb1 and lb2 that they should listen on that IP address. This is done by heartbeat which we install like this:
lb1/lb2
# aptitude install heartbeat
To allow HAProxy to bind to the shared IP address, we add the following line to /etc/sysctl.conf:
# vim /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind=1
... and run:
# sysctl -p
Now we have to create three configuration files for heartbeat, /etc/ha.d/authkeys, /etc/ha.d/ha.cf, and /etc/ha.d/haresources. /etc/ha.d/authkeys and /etc/ha.d/haresources must be identical on lb1 and lb2, and /etc/ha.d/ha.cf differs by just one line!
lb1/lb2
# vim /etc/ha.d/authkeys
auth 3
3 md5 somerandomstring
Note:- somerandomstring is a password which the two heartbeat daemons on lb1 and lb2 use to authenticate against each other
/etc/ha.d/authkeys should be readable by root only, therefore we do this:
# chmod 600 /etc/ha.d/authkeys
lb1
# vim /etc/ha.d/ha.cf
#
# keepalive: how many seconds between heartbeats
#
keepalive 2
#
# deadtime: seconds-to-declare-host-dead
#
deadtime 10
#
# What UDP port to use for udp or ppp-udp communication?
#
udpport 694
bcast eth0
mcast eth0 225.0.0.1 694 1 0
ucast eth0 192.168.0.101
# What interfaces to heartbeat over?
#udp eth0
#
# Facility to use for syslog()/logger (alternative to log/debugfile)
#
logfacility local0
#
# Tell what machines are in the cluster
# node nodename ... -- must match uname -n --> Very Important
node lb2.example.com
node lb2.example.com
Note:- on lb1 and lb2.The udpport, bcast, mcast, and ucast options specify how the two heartbeat nodes communicate with each other to find out if the other node is still alive. You can leave the udpport, bcast, and mcast lines as shown above, but in the ucast line it's important that you specify the IP address of the other heartbeat node in this case it's 192.168.0.101 (lb2.example.com).
On lb2 the file looks pretty much the same, except that the ucast line holds the IP address of lb1
lb2
# vim /etc/ha.d/ha.cf
#
# keepalive: how many seconds between heartbeats
#
keepalive 2
#
# deadtime: seconds-to-declare-host-dead
#
deadtime 10
#
# What UDP port to use for udp or ppp-udp communication?
#
udpport 694
bcast eth0
mcast eth0 225.0.0.1 694 1 0
ucast eth0 192.168.0.100
# What interfaces to heartbeat over?
#udp eth0
#
# Facility to use for syslog()/logger (alternative to log/debugfile)
#
logfacility local0
#
# Tell what machines are in the cluster
# node nodename ... -- must match uname -n --> Very Important
node lb1.example.com
node lb2.example.com
lb1/lb2
# vim /etc/ha.d/haresources
lb1.example.com 192.168.0.99 --> must match uname -n
lb1/lb2
# /etc/init.d/heartbeat start
Then run:
lb1
# ip addr sh eth0
... and you should find that lb1 is now listening on the shared IP address, too:
lb1:/etc/ha.d# ip addr sh eth0
2: eth0: mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
link/ether 00:0c:29:63:f7:5c brd ff:ff:ff:ff:ff:ff
inet 192.168.0.100/24 brd 192.168.0.255 scope global eth0
inet 192.168.0.99/24 brd 192.168.0.255 scope global secondary eth0:0
inet6 fe80::20c:29ff:fe63:f75c/64 scope link
valid_lft forever preferred_lft forever
lb1:/etc/ha.d#
You can check this again by running:
# ifconfig
eth0:0 Link encap:Ethernet HWaddr 00:0c:29:63:f7:5c
inet addr:192.168.0.99 Bcast:192.168.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:18 Base address:0x1400
As lb2 is the passive load balancer, it should not be listening on the virtual IP address as long as lb1 is up. We can check that with:
lb1
# ip addr sh eth0
The output should look like this:
lb2:~# ip addr sh eth0
2: eth0: mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
link/ether 00:0c:29:be:7b:3b brd ff:ff:ff:ff:ff:ff
inet 192.168.0.101/24 brd 192.168.0.255 scope global eth0
inet6 fe80::20c:29ff:febe:7b3b/64 scope link
valid_lft forever preferred_lft forever
lb2:~#
The output of ifconfig will not include eth0:0
6. Starting HAProxy
Note:- You have to stop apache since HAproxy listen on port 80 and the haproxy service will not start as apache service is on
Now we can start HAProxy:
lb1/lb2
# /etc/init.d/haproxy start
7. Testing
You can now make HTTP requests to the virtual IP address 192.168.0.99 (or to any domain/hostname that is pointing to the virtual IP address), and you should get content from the backend web servers.
You can test its high-availability/failover capabilities by switching off one backend web server - the load balancer should then redirect all requests to the remaining backend web server. Afterwards, switch off the active load balancer (lb1) - lb2 should take over immediately. You can check that by running:
8. HAProxy Statistics
You might have noticed that we have used the options stats enable and stats auth someuser:somepassword in the HAProxy configuration in chapter 4. This allow us to access (password-protected) HAProxy statistics under the URL http://192.168.0.99/haproxy?stats
