Kubernetes Monitoring with Prometheus

45 minutes
  • 2 Learning Objectives

About this Hands-on Lab

Prometheus is capable of monitoring a wide variety of environments, including Kubernetes. In this lab, you will have the opportunity to work with both Prometheus and Kubernetes as you configure Prometheus monitoring for a Kubernetes cluster. You will use `kube-state-metrics` to provide Kubernetes metric data, and you will configure Prometheus to pull in those metrics.

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Set Up `kube-state-metrics` to Expose Metrics for Your Kubernetes Cluster
  1. Log in to the Kubernetes server.

  2. Install kube-state-metrics:

    git clone https://github.com/kubernetes/kube-state-metrics.git
    cd kube-state-metrics/
    git checkout v1.8.0
    kubectl apply -f kubernetes
  3. Create a NodePort service to expose the metrics service outside the cluster, making it accessible to the Prometheus server:

    cd ..
    vi kube-state-metrics-nodeport-svc.yml
  4. Define the NodePort service in a YAML file:

    kind: Service
    apiVersion: v1
      namespace: kube-system
      name: kube-state-nodeport
        k8s-app: kube-state-metrics
      - protocol: TCP
        port: 8080
        nodePort: 30000
      type: NodePort
  5. Create the NodePort service and test it:

    kubectl apply -f kube-state-metrics-nodeport-svc.yml
    curl localhost:30000/metrics
Configure Prometheus to Scrape Metrics from `kube-state-metrics`
  1. Log in to the Prometheus server.

  2. Edit the Prometheus config:

    sudo vi /etc/prometheus/prometheus.yml
  3. Under the scrape_configs section, add a scrape configuration for Kubernetes:

    - job_name: 'Kubernetes'
      - targets: ['limedrop-kube:30000']
  4. Restart Prometheus to load the new configuration:

    sudo systemctl restart prometheus
  5. Use the expression browser to verify you can see Kubernetes metrics in Prometheus. You can access the expression browser in a web browser at http://<PROMETHEUS_SERVER_PUBLIC_IP>:9090.

  6. Run a query to view some Kubernetes metric data:


Additional Resources

Your company, LimeDrop, is using Kubernetes to orchestrate containers. A Prometheus server is already in place outside the cluster, but the Kubernetes containers are not currently being monitored.

Your task is to use kube-state-metrics to provide metric data for Kubernetes. Then, configure Prometheus to scrape metrics from kube-state-metrics.

Additional info:

  • You can find documentation on kube-state-metrics on GitHub.
  • Use kube-state-metrics version 1.8.0.
  • The Kubernetes server is reachable from the Prometheus server at the hostname limedrop-kube.

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Get Started
Who’s going to be learning?