My notes about Prometheus (1)
Why Prometheus? More accurately, why does monitoring matter? Humans go to hospitals or clinics to diagnose their health. Just like humans, software or applications need monitoring tools to know their health, give an alert, and prevent issues.
Now I hope you understand how important is monitoring! Prometheus is widely used in monitoring because it is not just a visualization tool or an alert system, it also collects data stored in the Prometheus database.
You can find an official source about Prometheus in Prometheus: Up & Running by Brian Brazil. If you are a visual learner, I find this youtube source well illustrating the Prometheus architecture.
Prometheus server
Quite many components, but I often just remember that Prometheus server fetches data about application health and stores it inside, and then provides the output data once requested (it can be requested from Grafana, Prometheus UI, alert manager).
Note: it’s interesting to note that Prometheus storage is local storage in a specified directory.
Fetch data to Prometheus server
(or pull data to Prometheus server)
Pull data is a scraping process via HTTP request (the content of data depends on metrics), and then parse, and ingest data to the database inside Prometheus server. The request won’t work by default, but after all below utilities are satisfied:
Client Library: in order to use Prometheus, there should be someone installed in the Client Library inside the application. This Client Library work inside the application to produce Prometheus text format data once triggered by HTTP request that contains metrics.
Exporters: if the Client Library cannot handle producing expected results, Exporter is used. The exporter is installed nearby but outside of the software application.
Service discovery: Prometheus won’t automatically know where is the targets (where is Client Library, where is Exporter). In a simple baby way, people can put a list of addresses that Prometheus can navigate. But for real-life, Service discovery is needed because:
With dynamic environments you cannot simply provide a list of applications and exporters once, as it will get out of date. This is where service discovery comes in.
Quoted from the book “Prometheus: Up and Running”
Sending data out
Other app communicates with Prometheus via HTTP request, which means that there is a Prometheus API method installed in Prometheus to receive HTTP request.
Typically applications that send the request to Prometheus server is Grafana, Prometheus UI, Alert Manager to query by PromQL.
The difference between Grafana, Prometheus UI vs Alert Manager is that
- Alert Manager doesn’t cost so much time for visualization and it doesn’t have visualization.
- It can automatically query regularly depending on the setting of frequency in the alert rule.
- It can automatically send Alert messages through email, Jira, etc.
