Exploring Basic HAProxy Monitoring Techniques

30 minutes
  • 3 Learning Objectives

About this Hands-on Lab

It’s not enough to just set up an HAProxy installation, we need to keep an eye on the status of the environment and the servers in our backend. Whether this means keeping an eye on statistics or the contents of logs, HAProxy has you covered. In this lab, we’re going to get hands-on with managing an HAProxy environment. We’re going to explore some monitoring options that are installed along with HAProxy. Upon completion of this lab, you will know how to leverage some included features to monitor your HAProxy environment.

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Interact Directly with the HAProxy stats Socket
  1. Look at and monitor the status of our HAProxy installation by directly interacting with the stats socket.

  2. Look at the socket, using file /var/lib/haproxy/stats.

  3. Let’s try some non-interactive commands. Pull raw data from the stats socket, using nc, by sending the show stat command to the stats socket.

    It returns a lot of information in a format that’s not really human-friendly, but it can be reformatted.

  4. Add some formatting using cut and column, and a looping mechanism using watch. Pick as many or as few fields as you like. Play around with the formatting and see what you can come up with.

    You should see that we’re getting statistics for the fields we specified in the cut command, then using column to make a nicely formatted table. The watch command refreshes every second until we interrupt it with CTRL-C.

  5. Try an interactive connection using nc and the /var/lib/haproxy/stats socket.

    This will connect us to the socket and allow us to enter one command.

  6. Use the prompt command to get an interactive interface. If you’d like help, use ?. Sending an empty line or using the quit command followed by ENTER will get you out of the prompt.

So, this is the most direct, hands-on way to do it, but it’s a lot of overhead for daily use, unless we put a framework around it. Is there a better way?

Using the HAProxy stats Web Interface

The /var/lib/haproxy/stats socket is extremely useful, but manual interaction is a bit cumbersome. What if we had a better way to leverage it?

  1. Add a code block to the end of the /etc/haproxy/haproxy.cfg file that starts an HAProxy stats web server on port 8050 for all addresses at the URI of /.
  2. Hide the version of HAProxy in the statistics report for better security.
  3. Restart the haproxy service.
  4. Connect to the stats web page on port 8050 on the public IP (or DNS) of the lab server and check things out.

    We’ve got a nice, clean, web-based interface that presents our statistics in an easy-to-read format. If we want to update our statistics, we’re just a refresh away.

Examining HAProxy Logs Using HALog

Another handy utility we can use to get a handle on what’s happening in our HAProxy environment is the inlcuded halog command-line utility.

Before we dig deeper into the logs, we need to put more data in.

  1. Launch 2 ApacheBench sessions:

    ab -n 100 -c 10 https://www.site1.com/ > ~/ab_site1.log > /dev/null 2>&1 &
    ab -n 100 -c 10 https://www.site2.com/ > ~/ab_site2.log > /dev/null 2>&1 &
  2. Pull HTTP traffic using curl:

    for conn in `seq 1 100` ; do curl -k https://www.site1.com/ ; done > /dev/null 2>&1 &
    for conn in `seq 1 100` ; do curl -k https://www.site2.com/ ; done > /dev/null 2>&1 &
  3. Pull HTTP traffic using wget:

    for conn in `seq 1 100` ; do wget --no-check-certificate -O - https://www.site1.com/test.txt ; done > /dev/null 2>&1 &
    for conn in `seq 1 100` ; do wget --no-check-certificate -O - https://www.site2.com/test.txt ; done > /dev/null 2>&1 &
  4. Pull scp traffic:

    for conn in `seq 1 100` ; do bash -c 'scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -P 2222 cloud_user@ssh.site3.com:/sshfiles/ssh-test.txt . &' ; done > /dev/null 2>&1 &

    Now that our logs are filled with data, we can proceed!

  5. Extract data from the /var/log/haproxy-combined-traffic.log file using halog:

    • Pull data on per-server HTTP statistics
    • List URLs by the number of HTTP requests
    • List URLs with 429 errors
    • List URLs by the number of errors generated
    • Try some of your own!

So, halog is a handy tool for parsing HAProxy logs to present the information contained within in an easy-to-read format that tells us exactly what we’re looking for.

Congratulations, Cloud Guru! You mastered 3 helpful monitoring methods that come with HAProxy!

Additional Resources

We need to prepare for the future!

Our HAProxy installation has been installed, configured, and secured. Now we're ready to move on to the next phase and go live with HAProxy, so we're going to need a few things. We need logging customized to our needs and tools and procedures to monitor and troubleshoot our environment and logs.

How do we do it?

You have been provided with an RHEL instance with HAProxy installed and configured.

When the lab starts, you will want to open an SSH connection to your 3 lab instances:

ssh cloud_user@PUBLIC_IP_ADDRESS

Replace PUBLIC_IP_ADDRESS with either the public IP or DNS of the instance(s). The cloud_user password has been provided with the instance information.

Entries for www.site1.com and www.site2.com have been created in /etc/hosts that point to Additionally, SSL certificates for HAProxy have been generated in /etc/haproxy/certs/. The HAProxy package has also been installed, but is not running.

On our system, we have two sites, site1 and site2, configured, with three web server containers in each, running rootlessly by the cloud_user account. They've been pre-populated with a test text file at /test.txt that identifies which site and server we're accessing.

The nginx containers are configured as follows:

  • site1_server1: web server accessible on port 8081
  • site1_server2: web server accessible on port 8082
  • site1_server3: web server accessible on port 8083
  • site2_server1: web server accessible on port 8084
  • site2_server2: web server accessible on port 8085
  • site2_server3: web server accessible on port 8086

The sshd container is configured as follows:

  • sshd1_server1: sshd server accessible on port 2223

Good luck and enjoy!

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Get Started
Who’s going to be learning?