Using Grafana to Visualize Prometheus Data

30 minutes
  • 6 Learning Objectives

About this Hands-on Lab

While Prometheus is a wonderful took for keep track of our metrics, its graphing and dashboard capabilities leave a lot to be desired. This is where Grafana comes in. Grafana allows us to create beautiful persistent dashboards using our Prometheus data. These dashboards let us gain insight in the parts of our infrastructure that are most important to us, without having to continually manually query with PromQL or viewing and click on our recording rules. From our Node Exporter data to tracking our current alerts, Grafana provides us with visual insight into what’s happening.

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Add Prometheus as a data source

Click Add data source from the home page. Add the Prometheus data source by setting the URL to http://localhost:9090. Save & Test.

Add the suggested dashboards for the Prometheus data source

Click the Dashboards tab. Import the Prometheus 2.0 Stats and Grafana metrics dashboards.

Import the “Node Exporter Full” dashboard
  1. Click the plus sign on the side menu, then select Import.

  2. In a new window or tab, open the JSON data for the dashboard and copy the provided data.

  3. Return to your Grafana instance. Paste in the JSON in the provided text box.

Create a new dashboard called “Forethought”

Click the plus sign again, this time selecting Dashboard from the options. Click the cog icon on the top of the screen and rename the dashboard to Forethought. Save.

Add a heatmap to the dashboard
  1. Click the Add panel button. Select Heatmap.

  2. Click the panel title, then click Edit.

  3. Set the metric so it tracks the average of all our response times via bucket:

    sum(rate(forethought_request_hist_bucket[30s])) by (le)

    Ensure the Legend is also set to {{le}}

  4. Switch to the Axes tab and change the Data Format to Time series buckets.

Save the dashboard

Return to the dashboard and Save the changes.

Additional Resources

Your company has recently set up a monitoring stack, using Prometheus, Alertmanager, and Grafana. While Prometheus and Alertmanager itself has since been set up and configured, and our applications have been instrumented to send out data, we're still left with some remaining configuration tasks. Namely, we've been given the job of getting our Grafana dashboards up and running.

Grafana is located at port 3000 on the monitoring server, and can be accessed directly from the web UI. Use the admin user and admin password to log in, then change the password when prompted. From there, you need to make sure to add Prometheus as a data source.

Once Prometheus has been added as a data source, import in some common-use dashboards, starting with the reccommended Prometheus 2.0 Stats and Grafana metrics dashboard. You also want to pull on the "Node Exporter Full" – the JSON for this dashboard can be found here.

Once finished, it's time to start building out some application metrics there won't be available dashboards for. Create a new dashboard called "Forethought," then create a heatmap based on the forethought_requests_hist_bucket, ensuring you're gathering the average data across all instances of the application. The Y-axis should use the bucket labels, while the X-axis should demonstrate our responses over time.

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Get Started
Who’s going to be learning?