Now available! Red Hat OpenShift Container Platform for Linux on IBM Z and LinuxONE Learn more

Deploy a custom metric server on Cloud Foundry using Prometheus and Grafana

Together, open source monitoring tools Grafana and Prometheus create a complete solution for gathering, storing, and visualizing metrics. Because Prometheus is a database, deploying on Cloud Foundry isn’t an ideal production solution. However, for development or testing purposes, using Cloud Foundry can ease deployment processes.

In this tutorial, I demonstrate how to deploy a managed Prometheus database and Grafana web application to a Cloud Foundry environment on IBM Cloud. Cloud Foundry provides infrastructure, SSL certifications for HTTP, easy management, and resource management.

For this tutorial, the Grafana deployment configuration is for development purposes. However, it is possible to have a production deployment of Grafana on Cloud Foundry. This simply requires changing the configuration and tuning it for production deployment.

Estimated time

Completing this tutorial should take about 30 minutes.

Prerequisites

  1. Create a free IBM Cloud account.
  2. Install the IBM Cloud CLI. You also have the option of downloading the stand-alone IBM Cloud CLI.

Step 1. Log in to IBM Cloud from your terminal with CLI

  1. Log in to your IBM Cloud account by entering your email address and password.

    If you have multiple accounts, the client will ask you to select which account you would like to connect. Type the account number in and press enter. If the client asks you to select a region, select the one where you have an organization. Your options should look something like this:

    image

    You should now see an output similar to this:

    image

  2. Now let’s target your ibmcloud client to the Default user group.

    ibmcloud target -g Default

  3. After targeting your resource group, you can target the IBM Cloud Foundry organization and space.

    ibmcloud target --cf

  4. Once you’ve completed steps 1-3, you should see a screen similar to the below image:

    image

Step 2. Deploy a Prometheus server

To deploy Prometheus on Cloud Foundry, you need three things:

  1. The Prometheus binary
  2. The Prometheus configuration file
  3. The Cloud Foundry manifests file and command

Cloud Foundry incorporates buildpacks that provide framework and runtime support for apps. Buildpacks determine what dependencies to download and how to configure apps to communicate with bound services. To learn more about buildpacks and its capabilities, see the Cloud Foundry buildpacks documentation.

In this tutorial, I use the binary buildpack to execute Prometheus on Cloud Foundry. Then I configure the Prometheus configuration file and create the manifest file for deployment. Let’s walk through those steps now.

Download the Prometheus server binary

Prometheus has three options for installation:

  1. Pre-compiled binaries
  2. Building Prometheus components from source
  3. Running Prometheus on Docker

I am using a pre-compiled binary for this deployment. Before you begin, make sure to download the latest version of Prometheus. Navigate to the Prometheus section and download the binary which targets the Linux operating system. You can filter operating systems as shown in the figure below.

image

Click on prometheus-\.\-\.tar.gz and download the binary.

You can untar the file by double-clicking on it, using an extractor, or with the tar xzf <filename> command on UNIX systems.

Edit the Prometheus configuration file

In the extracted folder you should see a configuration file named prometheus.yml. Below is a snippet of the file configured for Cloud Foundry. You need to edit the last line - targets: ['localhost:9090'] according to Cloud Foundry network requirements and change the port from 9090 to 8080.

This is what a working configuration file should look like:

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:8080'] # this field is configured

Create the Cloud Foundry manifest file for Prometheus

Manifests can help you automate deploying apps by providing consistency and reproducibility. Both manifests and command line options allow you to override the default attribute values of ibmcloud cf push. For more detailed information about a manifest file, check out the Manifests documentation.

The manifest.yml below is the file that we are using when deploying our Prometheus server. You need to create a manifest.yml file in the folder where prometheus executable is located.

---
applications:
- name: prometheus-test
  instances: 1
  buildpack: https://github.com/cloudfoundry/binary-buildpack.git
  command: ./prometheus --config.file=prometheus.yml --web.listen-address=:8080
  memory: 256M
  random-route: true

Here’s a breakdown of what the above means:

  • The three dashes (- – -) indicate the start of your YAML file.
  • Applications is a YAML array. Each item defines a Cloud Foundry application.
  • Name is the name of your Cloud Foundry application.
  • Instances defines how many instances of this application will run.
  • Buildpack is where you set the buildpack to binary-buildpack for this application.
  • Command is the command that Cloud Foundry will execute when deployment is ready, which is broken down into the following:
    • prometheus is the binary file for your deployment.
    • config.file is the argument to set the Prometheus configuration file that you edit.
    • web.listen-address is the argument to set a network IP and address. :8080 means the Prometheus server will listen to port 8080. The default is 9090, but Cloud Foundry works on 8080 by default.
  • Memory is the field to set the memory for your Cloud Foundry application.
  • Random-route sets a random route on IBM Cloud and is good for development and testing. It provides a random domain name with TLS encryption configured.

With this manifest file, you are now ready to deploy your Prometheus server.

Deploy a Prometheus server to IBM Cloud

Go to the folder where prometheus, prometheus.yml, and manifest.yml are located. Here’s an example of how your folder should look:

image

After entering the folder, run the following command to deploy your Prometheus server on IBM Cloud:

ibmcloud cf push

To see if your application is running, head over to the IBM Cloud dashboard.

  1. Make sure you’re logged in to your IBM Cloud account.
  2. Click the Navigation Menu icon located in the upper left-hand corner.
  3. Click Resource List.
  4. Under Cloud Foundry apps, you should see your Prometheus server application. Click on the name to enter into the service page.
  5. To the right of the name of the application, you should see a Running label with a green dot. This means your application is up and running!
  6. Lastly, click the Visit app URL link on the right of the Running label to open your Prometheus dashboard.

Step 3. Deploy the Grafana client

Grafana is a great tool to help you understand your metrics, no matter where they are stored.

To deploy the Grafana client, I am going to use the same methodology that I used when deploying to the Prometheus server, which means I will:

  1. Download the Grafana binary
  2. Edit the configuration file
  3. Write a manifest file

Download the Grafana binary

Navigate to the Grafana download page and locate the Standalone Linux Binaries section. Copy the commands and run them. You can find example commands for version 6.5.2 below:

wget https://dl.grafana.com/oss/release/grafana-6.5.2.linux-amd64.tar.gz
tar -zxvf grafana-6.5.2.linux-amd64.tar.gz

Now you have a Grafana binary in your file path!

Edit your Grafana configuration file

Under the configuration folder, you find Grafana’s configuration file. The file titled defaults.ini is the main configuration file. This is a large file with more than 700 lines, but for now, I am only interested in line 38:

http_port=3000

Before moving to the next step, you need to change this port from 3000 to 8080.

Create a Cloud Foundry manifest file for Grafana

Now you can create a manifest.yml file in the Grafana folder using the following command:

applications:
- name: grafana
  buildpack: binary_buildpack
  memory: 128m
  command: './bin/grafana-server web'
  random-route: true

Deploy Grafana to IBM Cloud

You are now ready to push your application to IBM Cloud by entering the following command in your Grafana folder:

ibmcloud cf push

To see if your application is running, follow the same steps from above to navigate to the IBM Cloud dashboard. Once you’ve opened your Prometheus dashboard, log in using admin/admin as your credentials.

Summary

Congratulations! You’ve successfully configured an environment to automatically collect metrics with Prometheus using Cloud Foundry. Using a Platform-as-a-service (PaaS) solution underneath minimizes the effort to configure such an environment. If you are interested in focusing on developing more robust applications, visualizing application metrics can be very useful.

To learn more about Prometheus, see Monitoring your apps in Kubernetes with Prometheus and Spring Boot. This is a great tutorial to learn more about monitoring for a Spring Boot application, using Docker and Helm.

Hasan Can Volaka