Prometheus alerts allow you to issue automated notifications when certain events occur, triggered by your metric data. Alertmanager is a necessary component in this process, handling the process of sending alerts to the appropriate destination, as well as adding some additional control over alerting functionality. In this lab, you will have the opportunity to install and configure an Alertmanager instance, and connect it to an existing Prometheus server.
Learning Objectives
Successfully complete this lab by achieving the following learning objectives:
- Install Alertmanager
Log in to the Prometheus server.
Create a user and group for Alertmanager:
sudo useradd -M -r -s /bin/false alertmanager
Download and install the Alertmanager binaries:
wget https://github.com/prometheus/alertmanager/releases/download/v0.20.0/alertmanager-0.20.0.linux-amd64.tar.gz
tar xvfz alertmanager-0.20.0.linux-amd64.tar.gz
sudo cp alertmanager-0.20.0.linux-amd64/alertmanager /usr/local/bin/
sudo chown alertmanager:alertmanager /usr/local/bin/alertmanager
sudo mkdir -p /etc/alertmanager
sudo cp alertmanager-0.20.0.linux-amd64/alertmanager.yml /etc/alertmanager
sudo chown -R alertmanager:alertmanager /etc/alertmanager
sudo mkdir -p /var/lib/alertmanager
sudo chown alertmanager:alertmanager /var/lib/alertmanager
Create a
systemd
unit for Alertmanager:sudo vi /etc/systemd/system/alertmanager.service
[Unit] Description=Prometheus Alertmanager Wants=network-online.target After=network-online.target [Service] User=alertmanager Group=alertmanager Type=simple ExecStart=/usr/local/bin/alertmanager --config.file /etc/alertmanager/alertmanager.yml --storage.path /var/lib/alertmanager/ [Install] WantedBy=multi-user.target
Start and enable the
alertmanager
service:sudo systemctl enable alertmanager
sudo systemctl start alertmanager
Verify the service is running and you can reach it:
sudo systemctl status alertmanager
curl localhost:9093
You can also access Alertmanager in a web browser at
http://<PROMETHEUS_SERVER_PUBLIC_IP>:9093
.
- Install `amtool`
Install the
amtool
binary:sudo cp alertmanager-0.20.0.linux-amd64/amtool /usr/local/bin/
Create a config file for
amtool
:sudo mkdir -p /etc/amtool
sudo vi /etc/amtool/config.yml
Enter the following content in the config file:
alertmanager.url: http://localhost:9093
Verify
amtool
is working by pulling the current Alertmanager configuration:amtool config show
- Configure Prometheus to Use Alertmanager
Edit the Prometheus config:
sudo vi /etc/prometheus/prometheus.yml
Under
alerting
, add your Alertmanager as a target:alerting: alertmanagers: - static_configs: - targets: ["localhost:9093"]
Restart Prometheus to reload the configuration:
sudo systemctl restart prometheus
Verify Prometheus is able to reach the Alertmanager. Access the Prometheus Expression Browser in a web browser at
http://<PROMETHEUS_SERVER_PUBLIC_IP>:9090/graph
. Run the following query and ensure the current value is1
:prometheus_notifications_alertmanagers_discovered