How to setup Prometheus, Node Exporter and Grafana
Performance monitoring and alerting are very crucial to measure the performance metrics of an application running in a production environment. In this blog post, we are going to talk about two popular open-source tools Prometheus and Grafana. This blog post is aimed at anyone who is trying to learn Prometheus and Grafana or has some experience with these tools.
This is going to be a very beginners guide where we will start from -
- Installation of Prometheus
- How to create and Install Node exporter
- Installing the Grafana
- Setting up the Grafana Dashboard
- Setting up Prometheus and Grafana for Kubernetes on AWS (Advance Use-Case)
- Advantages of Prometheus
- Advantages of Grafana
What is Prometheus? - It is an open-source tool for scrapping the performance metrics of any given hardware resource (CPU, VM, Cloud Virtual Machine, Router, etc.). Prometheus scraps the data with the timestamp which can be stored on a server and accessed using the PromQL. Prometheus has superb support API which makes Prometheus integration with any resource present in the Datacenter. Prometheus architecture is really scalable and 3rd party libraries and makes it more powerful.
What is Grafana? - In Lehman's term Grafana is a User interface for viewing the metrics scraped by Prometheus from various resources. Grafana is an open-source analytics and visualization tool. Grafana does not store any data, but instead, it relies on Prometheus to send the data so that dashboard can be prepared. Also, Grafana is used for sending notifications and mail alerts based on various thresholds. One of the cool features of the Grafana is Grafana Labs where you can go and download the dashboard prepared by other developers so that you do not have to re-invent the wheel.
1. Installation of Prometheus
Prometheus has a various ways of installation for example - pre-compiled binaries, from source, Docker. But to keep the installation simple we are going to use the pre-compiled binaries for installing Prometheus onto my Ubuntu Linux machine.
Goto Download Page of Prometheus and select the
prometheus-x.xx.x.linux-amd64.tar.gzfile for download. (Note- Here x.xx.x is version number)
Extract the download binary file. (Note- Replace x.xx.x with the downloaded version of Prometheus)
1 2tar xvfz prometheus-x.xx.x.linux-amd64.tar.gz
Go into the extracted directory
Start the prometheus server with the command
The Prometheus server should start on port
You can access the Prometheus graph UI by visint
You can access the Prometheus metrics UI by visint
2. How to create and Install Node exporter
Node exporter is responsible for fetching the statistics from various hardware and virtual resources in the format which Prometheus can understand and with the help of the prometheus server those statistics can be exposed on port 9100.
There are many third-party Node exporters which can be used by SREs as well as DevOps based on their application needs. But primarily we look for the following metrics -
- CPU usage
- Memory usage
- Disk usage
- Network usage
2.1 How to Install Node exporter
After installing the Prometheus in the previous step the next package we are going to install is **Node Exporter **. Node exported is used for collecting various hardware and kernel-level metrics of your machine.
Here are the download and installation steps -
Download the binary of Node exporter based on the operating system.(In my case I am using Ubuntu linux machine)
Extract the download node exporter binary file. (Note- Replace x.xx.x with the downloaded version of node exporter)
1 2tar xvfz node_exporter-*.*-amd64.tar.gz
Go into the extracted directory
cd xvfz node_exporter-*.*-amd64.tar.gz
Start the node exported with the command
- Access the Node exporter metrics on the browser with URL -
2.2 Adding node exporter scrap_configs to prometheus as a YAML Configuration
In the previous node export installation, we had Prometheus and node exporter running on the same server. But if you
want to add node exporter for any remote server then you need to define
target hosts inside
the YAML configuration.
Here is an example of
(*Note - In the below configuration you have replaced IP address 220.127.116.11 with your host IP or Hostname)
1 2global: 3 scrape_interval: 15s 4 5scrape_configs: 6 - job_name: node 7 static_configs: 8 - targets: ['localhost:9100','18.104.22.168:9100']
- As you can see I have added two hosts
localhost, 22.214.171.124inside the scrap_config.
- Save the above configuration as
- Start the Prometheus server bypassing
scrap_configwith the help of
1 2./prometheus --config.file=exporter.yml
Verify the Host status-
After adding the remote host into the node exporter configuration verify it by accessing the Prometheus target URL
2.3 Enable and disable the collector
If you have more than one collector configured then node_exporter allows you to enable and disable some of the collectors with run time arguments -
|- - collector.ENTER-THE-NAME-OF-COLLECTOR||Enable collector|
|- - no-collector.ENTER-THE-NAME-OF-COLLECTOR||Disable collector|
Here is an example command for starting the node_exporter with disabled flag -
1 2node_exporter --no-collector.ENTER-THE-NAME-OF-COLLECTOR
For enabling the collector use the following command -
1 2node_exporter --collector.ENTER-THE-NAME-OF-COLLECTOR
3. Installing the Grafana
After installing the Prometheus server and node exporter let's install the Grafana for setting up visualization using Grafana dashboard. Here is the official download link of Grafana, choose the correct Grafana binary based on your operating system. For this blog post, I am going to use ubuntu Linux binary.
Steps for Installing Grafana-
In this installation, we are going to install the latest enterprise edition Grafana.
- Update the package info -
1 2sudo apt-get install -y apt-transport-https 3sudo apt-get install -y software-properties-common wget 4wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
- Add stable repository of Grafana -
1 2echo "deb https://packages.grafana.com/enterprise/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
- Update repository and Install Grafana
1 2sudo apt-get update 3sudo apt-get install grafana-enterprise
- Start the
1 2sudo systemctl daemon-reload 3sudo systemctl start grafana-server 4sudo systemctl status grafana-server
- Configure Grafana to run at boot time-
1 2sudo systemctl enable grafana-server.service 3
- Access the Grafana Dashboard using
The default login Username: Password for accessing the Grafana dashboard is admin: admin
Change the default admin password after the login
4. Setting up the Grafana Dashboard
The next step after installing the Grafana Server would be to set up the Grafana Dashboard. Till now, we have installed the Prometheus Server, Node Exporter as well as Grafana Server but there is a connection between Prometheus and Grafana.
As I mentioned earlier Grafana is an analytics and visualization dashboard, Grafana does not store data, but instead it relies on Datasource connection towards the Prometheus server because Prometheus server is responsible for scraping and storing the data.
4.1 Let's create a Prometheus Datasource inside Grafana Dashboard
- Goto your Grafana Dashboard and click on Gear(Settings Icon)->Data Sources
- Click on Add Data Source
- Select the Prometheus as preferred data source -
- Enter the
hostname or IP addressof the prometheus server
- Save and test data source
4.2 Import Grafana Dashboard from Grafana Labs
Now after settings the data source we can import pre-existing opensource dashboard from Grafana Labs using the Dashboard ID.
- Goto Grafana Dashboard and search for Linux Memory
- Click on the result and then copy the board ID .e.g 2747
- Goto Import and enter the board ID .e.g. 2747
- Click load and then select the data source as Prometheus
Finally Click on Import
You should be able to see the memory dashboard
5. Advantages of Prometheus
Now I have talked about how to install and use Prometheus along with Grafana, let's see what are the advantages of using Prometheus in the landscape of DevOps and SREs -
- Prometheus is a very independent framework that does not impact your existing cloud or in-premise infrastructure.
- It can be customized to suit your infrastructure needs
- To use Prometheus and Node exporter a developer does not have to install all the node exporters available in the market. A Developer can only install the required node exporter and to avoid extra overhead on the hardware resources
- Prometheus node exporter can be customized to suit your infrastructure need, if you do not find a suitable node exporter in the library then Prometheus allows you to build a custom node exporter.
- Prometheus comes along with PromQL and TSDB which make it a more powerful tool for debugging troubleshooting.
- Prometheus has a built in database and you do not have to install the database separately.
- The pull mechanism of Prometheus is very flexible, and it allows you to schedule variable scrappers at different intervals.
- Also, Prometheus is being open-source, it has a wide variety of community support available for developers.
6. Advantages of Grafana
- If you compare Grafana with traditional tools Zabbix and LibreNMS then Grafana allows you to create multiple datasource whereas Zabbix and LibreNMS allow you only to create a single Datasource.
- In the Grafana dashboard you can customize various panels based on your requirement, also you can have integration with multiple data sources in the same Dashboard which is by far the biggest merit of using Grafana.
- In Grafana open-source community you can share your dashboard settings with others, and similarly, you can use other's dashboards by searching it on Grafana Community, so you do not have to re-invent the wheel again.
- Grafana dashboard's user experience is very pleasing and also it allows a developer to put PNG images on the dashboard to make it more appealing.
- Grafana developer community is building a lot of features and various datasource support which makes Grafana more adaptable for various infrastructure monitoring needs.
No doubt Grafana and Prometheus are a really popular frameworks for monitoring, alerting, and data visualization although there is a bit of learning because Prometheus has custom query language PromQL which requires little of practice before you call yourself an expert in the PromQL.
But when you look from DevOps and SREs end you would always like to have Prometheus and Grafana in your arsenal because of various third-party Node exporters, setting up various data sources, customizable dashboards, and panels. I hope this beginners guide will help you to get you started with Prometheus, Grafana, and Node exporter setup, and if you are interested in some advance setup of Prometheus and Grafana then I would recommend this guide on - How to setup Prometheus, Grafana dashboard for Kubernetes monitoring on AWS
Learn more On Prometheus and Grafana -