Even well-designed and maintained HAProxy installations experience issues of all sorts; an HAProxy configuration issue, one or more servers down or misconfigured, content issues, application or site configuration issues — the possibilities are endless. HAProxy troubleshooting skills are key to maintaining a healthy and happy HAProxy environment. In this lab, we’re going to get hands-on with troubleshooting in an HAProxy environment. We have a number of issues to track down and fix in our HAProxy lab environment, and we will use a number of utilities and techniques that are available on a basic HAProxy installation to accomplish our goals. Upon completion of this lab, you will be able to track down and correct issues in an HAProxy environment.
Learning Objectives
Successfully complete this lab by achieving the following learning objectives:
- Troubleshoot HAProxy Service Issues
HAProxy Troubleshooting Using
systemctl.The first thing we might want to check is the status of our
haproxyservice itself. If it’s not running, it’s not working. Ourhaproxyservice is managed bysystemctl.Check the status of the
haproxyservice usingsystemctl. Ensure that the service isenabledandactive. If it is not, fix it.HAProxy Troubleshooting Using
journalctl.Both
haproxyandsystemdlog to thesystemdjournal, so the next place we might want to check is there.Check the
systemdjournal usingjournalctlfor HAProxy errors. Address any issues you might find that prevent HAProxy from starting.- Troubleshoot HAProxy Configuration Issues
Checking the HAProxy Configuration File.
Sometimes we have one or more errors in our configuration file that prevent the
haproxyservice from starting. We can use thehaproxycommand to validate our configuration file.Validate the
/etc/haproxy/haproxy.cfgfile using thehaproxycommand. Address any issues you find that would prevent HAProxy from working properly.- Troubleshooting HAProxy Issues Using the stats Interface
Using the
statsSocket for Troubleshooting.We might want to try using the
statssocket directly to glean information about our environment. Let’s give it a try! Add some formatting to thestatssocket output usingcutandcolumn, and a looping mechanism usingwatch. UseCTRL-Cto exit. Address any issues you find.Using the
statsWeb Interface for Troubleshooting.If we want an easy-to-read interface to HAProxy’s
statsfacility, we can use the HAProxystatsweb interface. Let’s open a web browser and connect to port8050on the public IP of our HAProxy server. Address any issues you find.- Troubleshoot HAProxy Using Log Information
HAProxy Troubleshooting Using HAProxy Logs and
halog.We can leverage the detailed and precise HAProxy logs for troubleshooting. When combined with the
halogcommand, we can generate meaningful statistics that can help pinpoint issues.Try extracting some data from the
/var/log/haproxy-combined-traffic.logfile usinghalog:- Pull data on per-server HTTP statistics.
- List URLs with
404errors. - List URLs by the number of errors generated.
- Try some of your own! See what evidence you can find!
Examining HAProxy Logs Using
grep.We can use our old friend
grepto hand-examine our HAProxy logs. We can usegrepto look for pattern matches in the log files.Try the following:
- Look for elements with a DOWN status.
- Find all the
404error log entries.
