一、引言
Prometheus監控常見服務的原理主要包括服務暴露指標和Prometheus抓取指標。一方面,被監控服務通過自身提供的監控接口或借助Exporter將服務的性能指標等數據以HTTP協議的方式暴露出來;另一方面,Prometheus根據配置好的采集任務,定期去拉取這些服務暴露的指標數據,將其存儲在本地的時間序列數據庫中,之后用戶就可以通過PromQL查詢來分析這些數據,實現對服務的監控。
?二、Prometheus監控MySQL服務
1. 安裝數據庫
這里就不展示怎么安裝啦~
2. 啟動數據庫
[root@elk93 ~]# /etc/init.d/mysql.server start
Starting mysql.server (via systemctl): mysql.server.service.
[root@elk93 ~]#
[root@elk93 ~]# ss -ntl | grep 3306
LISTEN 0 151 *:3306 *:*
LISTEN 0 70 *:33060 *:*
[root@elk93 ~]# # 創建用戶并且授權,這里我就用已經存在的用戶。
mysql> SELECT user,host,plugin FROM mysql.user;
+------------------+-----------+-----------------------+
| user | host | plugin |
+------------------+-----------+-----------------------+
| linux96 | % | mysql_native_password |
| nacos | % | mysql_native_password |
| mysql.infoschema | localhost | caching_sha2_password |
| mysql.session | localhost | caching_sha2_password |
| mysql.sys | localhost | caching_sha2_password |
| root | localhost | caching_sha2_password |
+------------------+-----------+-----------------------+
6 rows in set (0.00 sec)mysql>
mysql> GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO linux96;
Query OK, 0 rows affected (0.00 sec)# 查看權限
mysql> SHOW GRANTS FOR linux96;
+-------------------------------------------------------------------+
| Grants for linux96@% |
+-------------------------------------------------------------------+
| GRANT SELECT, PROCESS, REPLICATION CLIENT ON *.* TO `linux96`@`%` |
| GRANT ALL PRIVILEGES ON `nova_kafka`.* TO `linux96`@`%` |
+-------------------------------------------------------------------+
3.?下載mysql_exporter
[root@elk93 ~]# wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.17.2/mysqld_exporter-0.17.2.linux-amd64.tar.gz3.解壓程序
[root@elk91 ~]# tar xf mysqld_exporter-0.17.2.linux-amd64.tar.gz -C /usr/local/bin/ mysqld_exporter-0.17.2.linux-amd64/mysqld_exporter --strip-components=1mysqld_exporter-0.17.2.linux-amd64/mysqld_exporter顯式指定要解壓的歸檔文件內的具體路徑。即從壓縮包中僅提取 mysqld_exporter-0.17.2.linux-amd64/ 目錄下的 mysqld_exporter 文件。
--strip-components=1移除解壓路徑中的前 1層目錄。[root@elk91 ~]#
[root@elk91 ~]# ll /usr/local/bin/
total 17936
drwxr-xr-x 2 root root 4096 Mar 27 10:56 ./
drwxr-xr-x 18 root root 4096 Mar 24 18:03 ../
-rwxr-xr-x 1 1001 1002 18356306 Feb 26 15:16 mysqld_exporter*
[root@elk91 ~]#
4. 啟動MySQL exporter
# MySQL-exporter可以和要暴露http(s)接口服務在同一個節點,這里分開是為了證明,不在同一個節點也可以。
[root@elk91 ~]# cat .my.cnf
[client]
user=linux96
password=novacao666
host=10.0.0.93
[root@elk91 ~]# [root@elk91 ~]# mysqld_exporter --config.my-cnf="/root/.my.cnf"
time=2025-03-27T03:09:09.709Z level=INFO source=mysqld_exporter.go:239 msg="Starting mysqld_exporter" version="(version=0.17.2, branch=HEAD, revision=e84f4f22f8a11089d5f04ff9bfdc5fc042605773)"
time=2025-03-27T03:09:09.709Z level=INFO source=mysqld_exporter.go:240 msg="Build context" build_context="(go=go1.23.6, platform=linux/amd64, user=root@18b69b4b0fea, date=20250226-07:16:19, tags=unknown)"
time=2025-03-27T03:09:09.709Z level=INFO source=mysqld_exporter.go:252 msg="Scraper enabled" scraper=global_status
......
time=2025-03-27T03:09:09.710Z level=INFO source=tls_config.go:350 msg="TLS is disabled." http2=false address=[::]:9104
5. 測試訪問
# 一般數據量大的都是正確的,如果是幾十幾百就說明我們暴露的數據不太對。
[root@elk93 ~]# curl -s http://10.0.0.91:9104/metrics| wc -l
2573
[root@elk93 ~]#
6. 配置Prometheus監控MySQL項
[root@prometheus-server31 ~]# tail -5 /softwares/prometheus-2.53.4.linux-amd64/prometheus.yml- job_name: linux96-mysql-exporterstatic_configs:- targets: - 10.0.0.91:9104# 熱加載
[root@prometheus-server31 ~]#
[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
7.? 驗證監控結果并導入grafana模板
# 訪問http://10.0.0.31:9090/targets?search=grafana導入模板ID 17320
18949
14057
7362
?三、Prometheus監控zookeeper集群
zookeeper沒有Prometheus提供的exporter組件,但是他自帶metrics暴露方式;打開以下注釋即可。
1. 修改zookeeper集群的配置信息
[root@elk91 ~]# vim /usr/local/zookeeper/conf/zoo.cfg
[root@elk91 ~]#
[root@elk91 ~]# tail -5 /usr/local/zookeeper/conf/zoo.cfg
# zookeeper沒有Prometheus提供的exporter組件,但是他自帶metrics暴露方式;打開以下注釋即可。
# https://prometheus.io Metrics Exporter
metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
metricsProvider.httpHost=0.0.0.0
metricsProvider.httpPort=7000
metricsProvider.exportJvmInfo=true
[root@elk91 ~]# # 拷貝配置文件到其他集群的節點
[root@elk91 ~]# scp /usr/local/zookeeper/conf/zoo.cfg 10.0.0.92:/usr/local/zookeeper/conf/
[root@elk91 ~]# scp /usr/local/zookeeper/conf/zoo.cfg 10.0.0.93:/usr/local/zookeeper/conf/
?2. 重啟zookeeper服務
[root@elk91 ~]# zkServer.sh restart
[root@elk92 ~]# zkServer.sh restart
[root@elk93 ~]# zkServer.sh restart
3. 訪問測試?
http://10.0.0.91:7000/metrics
http://10.0.0.92:7000/metrics
http://10.0.0.93:7000/metrics
4. 配置prometheus監控zookeeper
[root@prometheus-server31 ~]# tail -7 /softwares/prometheus-2.53.4.linux-amd64/prometheus.yml- job_name: linux96-zk-exporterstatic_configs:- targets: - 10.0.0.91:7000- 10.0.0.92:7000- 10.0.0.93:7000
.....
[root@prometheus-server31 ~]# # 熱加載Prometheus
[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#
5. 驗證監控結果并導入grafana模板
5.驗證配置是否生效
http://10.0.0.31:9090/targets?search=
# 看到Prometheus的targets出現zookeeper集群就行6.grafana導入模板ID
10465
四、Prometheus監控kafka集群
1.?啟動kafka集群
[root@elk91 ~]# kafka-server-start.sh -daemon /usr/local/kafka_2.13-3.9.0/config/server.properties
[root@elk91 ~]# ss -ntl | grep 9092
LISTEN 0 50 [::ffff:10.0.0.91]:9092 *:*
[root@elk91 ~]# [root@elk92 ~]# kafka-server-start.sh -daemon /usr/local/kafka_2.13-3.9.0/config/server.properties
[root@elk92 ~]# ss -ntl | grep 9092
LISTEN 0 50 [::ffff:10.0.0.92]:9092 *:*
[root@elk92 ~]# [root@elk93 ~]# kafka-server-start.sh -daemon /usr/local/kafka_2.13-3.9.0/config/server.properties
[root@elk93 ~]# ss -ntl | grep 9092
LISTEN 0 50 [::ffff:10.0.0.93]:9092 *:*
[root@elk93 ~]#
?2. 下載kafka exporter?
wget https://github.com/danielqsj/kafka_exporter/releases/download/v1.9.0/kafka_exporter-1.9.0.linux-amd64.tar.gz3.解壓軟件包
[root@elk92 ~]# tar xf kafka_exporter-1.9.0.linux-amd64.tar.gz -C /usr/local/bin/ kafka_exporter-1.9.0.linux-amd64/kafka_exporter --strip-components=1
[root@elk92 ~]#
[root@elk92 ~]# ll /usr/local/bin/
total 24520
drwxr-xr-x 2 root root 4096 Mar 27 11:44 ./
drwxr-xr-x 14 root root 4096 Mar 18 14:38 ../
-rwxr-xr-x 1 1001 fwupd-refresh 25099148 Feb 17 11:04 kafka_exporter*
[root@elk92 ~]#
3.?啟動kafka exporter
# 這里我們指定了集群信息,和kafka的版本信息
[root@elk92 ~]# kafka_exporter --kafka.server=10.0.0.91:9092 --kafka.server=10.0.0.92:9092 --kafka.server=10.0.0.93:9092 --kafka.version="3.9.0"
4. 訪問kafka exporter的WebUI?
# 出現大量信息即可
http://10.0.0.92:9308/metrics
5.?配置Prometheus監控kafka集群
[root@prometheus-server31 ~]# tail -5 /softwares/prometheus-2.53.4.linux-amd64/prometheus.yml- job_name: linux96-kafka-exporterstatic_configs:- targets: - 10.0.0.92:9308
...
# 熱加載
[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#
6. 驗證監控結果并導入grafana模板
# 出現kafka集群的targets即可
http://10.0.0.31:9090/targets?search=8.grafana導入模板ID
12460
10122
五、prometheus監控nacos服務
nacos服務是自己有提供metrics接口,在配置文件中打開注釋即可
參考鏈接: 監控手冊 | Nacos 官網
1. 啟動nacos集群
[root@elk91 ~]# /usr/local/nacos/bin/startup.sh
[root@elk92 ~]# /usr/local/nacos/bin/startup.sh
[root@elk93 ~]# /usr/local/nacos/bin/startup.sh
訪問測試 :http://10.0.0.93:8848/
?2. 修改nacos服務啟用Metrics接口
# nacos和zk服務都是自己有提供metrics接口,在配置文件中打開注釋即可[root@elk91 ~]# vim /usr/local/nacos/conf/application.properties
...
management.endpoints.web.exposure.include=prometheus,health
...
# 將配置文件拷貝給集群的其他節點
[root@elk91 ~]# scp /usr/local/nacos/conf/application.properties 10.0.0.92:/usr/local/nacos/conf/
[root@elk91 ~]# scp /usr/local/nacos/conf/application.properties 10.0.0.92:/usr/local/nacos/conf/
3. 重啟服務?
[root@elk91 ~]# /usr/local/nacos/bin/shutdown.sh
[root@elk92 ~]# /usr/local/nacos/bin/shutdown.sh
[root@elk93 ~]# /usr/local/nacos/bin/shutdown.sh [root@elk91 ~]# /usr/local/nacos/bin/startup.sh
[root@elk92 ~]# /usr/local/nacos/bin/startup.sh
[root@elk93 ~]# /usr/local/nacos/bin/startup.sh
4.?訪問nacos的WebUI
http://10.0.0.91:8848/actuator/prometheus
http://10.0.0.92:8848/actuator/prometheus
http://10.0.0.93:8848/actuator/prometheus
5.?配置Prometheus監控nacos集群?
# 在Prometheus配置文件中,他默認只會發現到是以/metrics結尾的URI,所以這里我們要指引一下Prometheus。加一行配置文件指定。
[root@prometheus-server31 ~]# tail -8 /softwares/prometheus-2.53.4.linux-amd64/prometheus.yml - job_name: linux96-nacos-exportermetrics_path: "/actuator/prometheus"static_configs:- targets: - 10.0.0.91:8848- 10.0.0.92:8848- 10.0.0.93:8848......# 熱加載Prometheus
[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#
6.?驗證監控結果并導入grafana模板
http://10.0.0.31:9090/targets?search=7.grafana導入模板ID
13221
六、?Prometheus監控docker原生Metrics
參考鏈接:
?? ?https://docs.docker.com/reference/cli/dockerd/#daemon-metrics
?1. 停止docker
[root@elk93 ~]# systemctl stop docker
2. 修改docker啟動腳本啟用Metrics監控指標
[root@elk93 ~]# cat /lib/systemd/system/docker.service
[Unit]
Description=novacao linux Docke Engine
Documentation=https://docs.docker.com
Wants=network-online.target[Service]
Type=notify
ExecStart=/usr/bin/dockerd --metrics-addr 0.0.0.0:5200
[Install]
WantedBy=multi-user.target
[root@elk93 ~]#
[root@elk93 ~]# systemctl daemon-reload
[root@elk93 ~]#
[root@elk93 ~]# systemctl restart docker.service
[root@elk93 ~]#
[root@elk93 ~]# ss -ntl | grep 5200
LISTEN 0 4096 *:5200 *:*
[root@elk93 ~]#
3.?訪問docker的Metrics接口?
http://10.0.0.93:5200/metrics[root@node-exporter41 ~]# curl -s http://10.0.0.93:5200/metrics| wc -l
589
[root@node-exporter41 ~]#
4. prometheus監控docker服務
[root@prometheus-server31 ~]# tail -5 /softwares/prometheus-2.53.4.linux-amd64/prometheus.yml - job_name: linux96-docker-exporterstatic_configs:- targets: - 10.0.0.93:5200[root@prometheus-server31 ~]#
[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#
5.?驗證監控結果并導入grafana模板
http://10.0.0.31:9090/targets?search=8.grafana導入模板ID
9621# 這里Prometheus提供的原生Metrics接口數據顯示不全,所以我們有第二種解決方案。
Prometheus基于cAdvisor監控Docker容器
?1.?cAdvisor是什么?
cAdvisor(Container Advisor)是由Google開源的一款輕量級容器監控工具,主要用于實時收集、分析和展示容器運行時資源使用情況與性能數據,幫助用戶了解容器化應用的行為和資源消耗。
2. 在docker節點拉取鏡像?
[root@elk92 ~]# docker pull gcr.io/cadvisor/cadvisor-amd64:v0.52.1
v0.52.1: Pulling from cadvisor/cadvisor-amd64
44cf07d57ee4: Already exists
.....
Status: Downloaded newer image for gcr.io/cadvisor/cadvisor-amd64:v0.52.1
gcr.io/cadvisor/cadvisor-amd64:v0.52.1
[root@elk92 ~]#
3. 運行cAdvisor?
[root@elk92 ~]# docker run \--volume=/:/rootfs:ro \--volume=/var/run:/var/run:ro \--volume=/sys:/sys:ro \--volume=/var/lib/docker/:/var/lib/docker:ro \--volume=/dev/disk/:/dev/disk:ro \--publish=18080:8080 \--detach=true \--name=cadvisor \--privileged \--device=/dev/kmsg \gcr.io/cadvisor/cadvisor-amd64:v0.52.1[root@elk92 ~]# docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3857cc9ff34e gcr.io/cadvisor/cadvisor-amd64:v0.52.1 "/usr/bin/cadvisor -…" 2 minutes ago Up About a minute (healthy) 0.0.0.0:18080->8080/tcp, :::18080->8080/tcp cadvisor
[root@elk92 ~]#
4. 訪問測試
[root@elk92 ~]# curl -s http://10.0.0.92:18080/metrics | wc -l
4885
[root@elk92 ~]#?
5. 訪問cAdVisor的webUI測試
http://10.0.0.92:18080/containers/
6. 配置Prometheus監控docker容器
[root@prometheus-server31 ~]# tail -6 /softwares/prometheus-2.53.4.linux-amd64/prometheus.yml - job_name: linux96-cadVisor-exporterstatic_configs:- targets: - 10.0.0.92:18080- 10.0.0.93:18080# 熱加載Prometheus
[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#
7.?驗證監控結果并導入grafana模板
http://10.0.0.31:9090/targets?search=6.grafana導入模板ID
11600
七、prometheus監控nginx
方案1>nginx-module-vts
1. 編譯安裝NGINX
1.1 安裝編譯工具
[root@elk91 ~]# apt -y install git wget gcc make zlib1g-dev build-essential libtool openssl libssl-dev1.2 克隆nginx-module-vts模塊# 配置虛擬機VPN
export https_proxy=http://10.0.0.1:7890
export http_proxy=http://10.0.0.1:7890[root@elk91 ~]# env| grep http
https_proxy=http://10.0.0.1:7890
http_proxy=http://10.0.0.1:7890
[root@elk91 ~]# [root@elk91 ~]# git clone https://github.com/vozlt/nginx-module-vts.git
Cloning into 'nginx-module-vts'...
remote: Enumerating objects: 1134, done.
remote: Counting objects: 100% (262/262), done.
remote: Compressing objects: 100% (87/87), done.
remote: Total 1134 (delta 206), reused 175 (delta 175), pack-reused 872 (from 2)
Receiving objects: 100% (1134/1134), 1.22 MiB | 935.00 KiB/s, done.
Resolving deltas: 100% (746/746), done.
[root@elk91 ~]# 1.3 下載nginx軟件包
[root@elk91 ~]# wget http://nginx.org/download/nginx-1.27.3.tar.gz1.4 解壓nginx
[root@elk91 ~]# tar xf nginx-1.27.3.tar.gz 1.5 配置nginx
[root@elk91 ~]# cd nginx-1.27.3/
[root@elk91 nginx-1.27.3]#
[root@elk91 nginx-1.27.3]# ./configure --prefix=/softwares/nginx --with-http_ssl_module --with-http_v2_module --with-http_realip_module --without-http_rewrite_module --with-http_stub_status_module --without-http_gzip_module --with-file-aio --with-stream --with-stream_ssl_module --with-stream_realip_module --add-module=/root/nginx-module-vts1.6 編譯并安裝nginx
make -j 2 && make install
2.?修改nginx的配置文件
[root@elk91 nginx-1.27.3]# vim /softwares/nginx/conf/nginx.conf
...
http {vhost_traffic_status_zone;upstream novacao-promethues {server 10.0.0.31:9090;}...server {...location / {root html;# index index.html index.htm;proxy_pass http://novacao-promethues;}location /nginx_status {vhost_traffic_status_display;vhost_traffic_status_display_format html;}}
}
?3. 啟動NGINX
[root@elk91 nginx-1.27.3]# /softwares/nginx/sbin/nginx -t
nginx: the configuration file /softwares/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /softwares/nginx/conf/nginx.conf test is successful
[root@elk91 nginx-1.27.3]# 1.9 啟動nginx
[root@elk91 nginx-1.27.3]# systemctl disable --now nginx
Synchronizing state of nginx.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable nginx
Removed /etc/systemd/system/multi-user.target.wants/nginx.service.
[root@elk91 nginx-1.27.3]#
[root@elk91 nginx-1.27.3]# /softwares/nginx/sbin/nginx
[root@elk91 nginx-1.27.3]#
4.?訪問nginx的狀態頁面
http://10.0.0.91/nginx_status/
http://10.0.0.91/nginx_status/format/prometheus
5.?修改Prometheus的配置文件
[root@prometheus-server31 ~]# tail -6 /softwares/prometheus-2.53.4.linux-amd64/prometheus.yml - job_name: linux96-nginx-vts-modulesmetrics_path: "/nginx_status/format/prometheus"static_configs:- targets: - 10.0.0.91:80# 熱加載
[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#
6.?驗證監控結果并導入grafana模板
# 查看Prometheus
http://10.0.0.31:9090/targets?search=1.12 導入模板ID
9785
方案2>安裝nginx-vtx-exporter
# 前提也是需要編譯好NGINX
1. 下載nginx-vtx-exporter?
[root@elk92 ~]# wget https://github.com/sysulq/nginx-vts-exporter/releases/download/v0.10.8/nginx-vtx-exporter_0.10.8_linux_amd64.tar.gz[root@elk92 ~]# tar xf nginx-vtx-exporter_0.10.8_linux_amd64.tar.gz -C /usr/local/bin/ nginx-vtx-exporter
[root@elk92 ~]#
[root@elk92 ~]# ll /usr/local/bin/nginx-vtx-exporter
-rwxr-xr-x 1 1001 avahi 7950336 Jul 11 2023 /usr/local/bin/nginx-vtx-exporter*
[root@elk92 ~]#
2.?運行nginx-vtx-exporter
[root@elk92 ~]# nginx-vtx-exporter -nginx.scrape_uri=http://10.0.0.91/nginx_status/format/json
2025/03/27 16:25:19 Starting nginx_vts_exporter (version=, branch=, revision=)
2025/03/27 16:25:19 Build context (go=go1.20.5, user=, date=)
2025/03/27 16:25:19 Starting Server at : :9913
2025/03/27 16:25:19 Metrics endpoint: /metrics
2025/03/27 16:25:19 Metrics namespace: nginx
2025/03/27 16:25:19 Scraping information from : http://10.0.0.91/nginx_status/format/json
3.?訪問nginx exporter的WebUI
http://10.0.0.93:9913/metrics
4.?配置prometheus采集nginx數據
[root@prometheus-server31 ~]# tail -12 /softwares/prometheus-2.53.4.linux-amd64/prometheus.yml - job_name: linux96-nginx-vts-modulesmetrics_path: "/nginx_status/format/prometheus"static_configs:- targets: - 10.0.0.91:80- job_name: "linux96-nginx-vtx-exporter"static_configs:- targets: - 10.0.0.92:9913[root@prometheus-server31 ~]#
[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
5.?驗證監控結果并導入grafana模板
導入grafana模板
2949
八、Prometheus監控redis
1. 部署Redis服務
1.拉取redis鏡像
[root@elk92 ~]# docker pull redis:7.4.2-alpine
7.4.2-alpine: Pulling from library/redis
...........
Digest: sha256:02419de7eddf55aa5bcf49efb74e88fa8d931b4d77c07eff8a6b2144472b6952
Status: Downloaded newer image for redis:7.4.2-alpine
docker.io/library/redis:7.4.2-alpine2.部署redis服務
[root@elk92 ~]# docker run -d --network host --name redis-server redis:7.4.2-alpine
14f6cd2c24f0735bb5bfd189730898b15380101dd11516ad2af3b7776a33a75a
[root@elk92 ~]#
[root@elk92 ~]# ss -ntl | grep 6379
LISTEN 0 511 0.0.0.0:6379 0.0.0.0:*
LISTEN 0 511 [::]:6379 [::]:*
[root@elk92 ~]#
2. 部署redis-exporter
wget https://github.com/oliver006/redis_exporter/releases/download/v1.69.0/redis_exporter-v1.69.0.linux-amd64.tar.gz4.解壓軟件包
[root@elk93 ~]# tar xf redis_exporter-v1.69.0.linux-amd64.tar.gz -C /usr/local/bin/ redis_exporter-v1.69.0.linux-amd64/redis_exporter --strip-components=1
[root@elk93 ~]#
[root@elk93 ~]# ll /usr/local/bin/redis_exporter
-rwxr-xr-x 1 mysql fwupd-refresh 9638072 Mar 15 12:41 /usr/local/bin/redis_exporter*
[root@elk93 ~]#
3. 運行redis exporter?
[root@elk93 ~]# redis_exporter -redis.addr redis://10.0.0.92:6379
INFO[0000] Redis Metrics Exporter v1.69.0 build date: 2025-03-15-04:40:26 sha1: 71dbe37fb14a4ae2537c1790a239dc1e568ffba5 Go: go1.24.1 GOOS: linux GOARCH: amd64
INFO[0000] Providing metrics at :9121/metrics
4. 訪問redis exporter的WebUI
http://10.0.0.93:9121/metrics
5.?配置Prometheus監控redis容器指標?
[root@prometheus-server31 ~]# tail -7 /softwares/prometheus-2.53.4.linux-amd64/prometheus.yml - job_name: linux96-redis-exporterstatic_configs:- targets: - 10.0.0.93:9121 [root@prometheus-server31 ~]#
[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#
6. 驗證監控結果并導入grafana模板
驗證Prometheus配置是否生效
http://10.0.0.31:9090/targets?search=9.grafana導入模板ID
763
14091
14615
彩蛋、 grafana安裝插件
1. 圖表報錯提示
Panel plugin not found: natel-discrete-panel說明需要安裝對應的插件,使用grafana客戶端工具安裝即可。
?2. 安裝指定插件
[root@prometheus-server31 ~]# ll /var/lib/grafana/
total 5236
drwxr-xr-x 5 grafana grafana 4096 Mar 27 16:53 ./
drwxr-xr-x 61 root root 4096 Mar 26 11:59 ../
drwxr-x--- 3 grafana grafana 4096 Mar 26 12:00 alerting/
drwx------ 2 grafana grafana 4096 Mar 26 12:00 csv/
-rw-r----- 1 grafana grafana 5337088 Mar 27 16:53 grafana.db
drwx------ 2 grafana grafana 4096 Mar 26 12:00 png/
[root@prometheus-server31 ~]#
[root@prometheus-server31 ~]# grafana-cli plugins install natel-discrete-panel
? Downloaded and extracted natel-discrete-panel v0.1.1 zip successfully to /var/lib/grafana/plugins/natel-discrete-panelPlease restart Grafana after installing or removing plugins. Refer to Grafana documentation for instructions if necessary.[root@prometheus-server31 ~]#
[root@prometheus-server31 ~]# ll /var/lib/grafana/
total 5240
drwxr-xr-x 6 grafana grafana 4096 Mar 27 16:56 ./
drwxr-xr-x 61 root root 4096 Mar 26 11:59 ../
drwxr-x--- 3 grafana grafana 4096 Mar 26 12:00 alerting/
drwx------ 2 grafana grafana 4096 Mar 26 12:00 csv/
-rw-r----- 1 grafana grafana 5337088 Mar 27 16:53 grafana.db
drwxr-xr-x 3 root root 4096 Mar 27 16:56 plugins/ # 安裝后會將插件存儲在本地。
drwx------ 2 grafana grafana 4096 Mar 26 12:00 png/
[root@prometheus-server31 ~]#
3. 重啟grafana使得插件生效?
[root@prometheus-server31 ~]# systemctl restart grafana-server.service
[root@prometheus-server31 ~]#
九、Prometheus監控MongoDB
1.?部署mongoDB服務
1.下載mongoDB鏡像
[root@elk92 ~]# docker pull mongo:8.0.6-noble
8.0.6-noble: Pulling from library/mongo
.................
Status: Downloaded newer image for mongo:8.0.6-noble
docker.io/library/mongo:8.0.6-noble
[root@elk92 ~]# 2.部署mongoDB服務
[root@elk92 ~]# docker run -d --name mongodb-server --network host mongo:8.0.6-noble
4b0f00dea78bb571c216c344984ced026c1210c94db147fdc9e32f549e3135de
[root@elk92 ~]#
[root@elk92 ~]# docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4b0f00dea78b mongo:8.0.6-noble "docker-entrypoint.s…" 2 seconds ago Up 1 second mongodb-server
[root@elk92 ~]#
[root@elk92 ~]# ss -ntl | grep 27017
LISTEN 0 4096 0.0.0.0:27017 0.0.0.0:*
[root@elk92 ~]#
2. 部署MongoDB的exporter
https://github.com/percona/mongodb_exporter/releases/download/v0.43.1/mongodb_exporter-0.43.1.linux-amd64.tar.gz解壓軟件包
[root@elk91 ~]# tar xf mongodb_exporter-0.43.1.linux-amd64.tar.gz -C /usr/local/bin/ mongodb_exporter-0.43.1.linux-amd64/mongodb_exporter --strip-components=1
[root@elk91 ~]#
[root@elk91 ~]# ll /usr/local/bin/mongodb_exporter
-rwxr-xr-x 1 1001 geoclue 20467864 Dec 13 20:10 /usr/local/bin/mongodb_exporter*
3.?準備MongoDB測試數據
[root@elk91 ~]# vim insert_data.js
// 創建數據庫和集合
db = db.getSiblingDB('testdb');// 插入用戶數據(10條)
for (let i = 1; i <= 10; i++) {db.users.insertOne({name: `user${i}`,email: `user${i}@example.com`,age: Math.floor(Math.random() * 50 + 18),created_at: new Date()});
}// 插入訂單數據(10條)
for (let i = 1; i <= 10; i++) {db.orders.insertOne({user_id: i,product: `product${Math.floor(Math.random() * 5)}`,amount: Math.random() * 100,status: ["pending", "completed", "cancelled"][Math.floor(Math.random() * 3)],created_at: new Date()});
}// 插入產品數據(5條)
const products = ["product0", "product1", "product2", "product3", "product4"];
products.forEach((product, index) => {db.products.insertOne({name: product,price: Math.random() * 50 + 20,stock: Math.floor(Math.random() * 100)});
});// 創建索引(可選)
db.users.createIndex({ email: 1 });
db.orders.createIndex({ user_id: 1 });################################################################## 將腳本復制到容器內
docker cp insert_data.js mongodb-server:/tmp/insert_data.js# 執行腳本(使用 mongosh)
docker exec -it mongodb-server mongosh --quiet /tmp/insert_data.js
4.?運行mongodb-exporter?
mongodb_exporter --mongodb.uri=mongodb://10.0.0.92:27017 --log.level=info --collector.dbstats --collector.diagnosticdata --collector.replicasetstatus --collector.topmetrics --collector.currentopmetrics --collector.exporter-metrics --collector.collstats --collector.indexstats --mongodb.collstats-colls=testdb.users,testdb.orders,testdb.products --mongodb.indexstats-colls=testdb.users,testdb.orders#####################################3驗證mongoDB-exporter的WebUI
http://10.0.0.91:9216/metrics
5. 配置Prometheus監控mongoDB容器
[root@prometheus-server31 ~]# tail -7 /softwares/prometheus-2.53.4.linux-amd64/prometheus.yml - job_name: linux96-mongodb-exporterstatic_configs:- targets: - 10.0.0.91:9216[root@prometheus-server31 ~]#
[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
6.?驗證監控結果并導入grafana模板
grafana導入模板ID
16504當然如果實在找不到,就只能自己定義Dashboard。比如: "mongodb_up"
十、Prometheus監控ElasticSearch集群
保障ES集群狀態正常
[root@elk91 ~]# curl https://10.0.0.91:9200/_cat/nodes -u elastic:123456 -k
10.0.0.93 59 33 45 7.11 2.18 0.75 cdfhilmrstw - elk93
10.0.0.91 78 44 4 0.73 0.51 0.29 cdfhilmrstw * elk91
10.0.0.92 54 31 34 1.49 0.51 0.18 cdfhilmrstw - elk92
1. 下載elasticsearch_exporter服務插件
[root@node-exporter41 ~]# wget https://github.com/prometheus-community/elasticsearch_exporter/releases/download/v1.9.0/elasticsearch_exporter-1.9.0.linux-amd64.tar.gz
2.解壓軟件包
[root@node-exporter41 ~]# tar xf elasticsearch_exporter-1.9.0.linux-amd64.tar.gz -C /usr/local/bin/ elasticsearch_exporter-1.9.0.linux-amd64/elasticsearch_exporter --strip-components=1
[root@node-exporter41 ~]# ll /usr/local/bin/
total 14728
drwxr-xr-x 2 root root 4096 Mar 27 02:42 ./
drwxr-xr-x 10 root root 4096 Feb 16 2024 ../
-rwxr-xr-x 1 1001 fwupd-refresh 15069336 Mar 3 10:01 elasticsearch_exporter*
2.?啟動elasticsearch-exporter
[root@node-exporter41 ~]# elasticsearch_exporter --es.uri="https://elastic:123456@10.0.0.91:9200" --es.ssl-skip-verify
time=2025-03-27T02:42:46.739Z level=INFO source=clusterinfo.go:211 msg="triggering initial cluster info call"
time=2025-03-27T02:42:46.739Z level=INFO source=clusterinfo.go:182 msg="providing consumers with updated cluster info label"
time=2025-03-27T02:42:46.750Z level=INFO source=main.go:244 msg="started cluster info retriever" interval=5m0s
.....
time=2025-03-27T02:14:19.093Z level=INFO source=tls_config.go:350 msg="TLS is disabled." http2=false address=[::]:9114
3. 配置Prometheus的job服務發現
[root@prometheus-server31 ~]# tail -5 /softwares/prometheus-2.53.4.linux-amd64/prometheus.yml- job_name: linux96-es-exporterstatic_configs:- targets: - 10.0.0.41:91145.熱加載Prometheus
[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#
?4.?驗證監控結果并導入grafana模板
驗證Prometheus配置是否生效
http://10.0.0.31:9090/targets?search=7.grafana導入模板ID
14191
9746
十一、小結
Prometheus監控服務分為三種情況。
1. 第一種情況
在Prometheus官網里面提供了exporter的組件,我們只需要部署好組件然后把要監控的服務使用http(s)的方式暴露出去即可,然后Prometheus修改配置文件,配置好服務發現,最后Grafana套用官方的模版ID即可。(當然也可以自己去制作dashboard展示)。
# Prometheus的exporter官網
https://prometheus.io/download/
https://prometheus.io/docs/instrumenting/exporters/
# Grafana的模版官網
https://grafana.com/dashboards2. 第二種情況
有些服務自己本身就支持云原生的Prometheus監控,在自己的配置文件中就支持metrics的接口暴露。只需要在配置文件中修改或者取消注釋好配置即可,然后Prometheus修改配置文件,配置好服務發現,最后Grafana套用官方的模版ID即可。(當然也可以自己去制作dashboard展示)。
例如:nacos,zookeeper3. 第三種情況
第一二種情況都不能達到預期的標準,那么他就會有第三種服務為其metrics提供接口。
例如:docker的cAdvisor
希望這篇文章可以幫助您在生產環境中高效率的解決問題。
最后有關K8S的如何監控可以看我的基于 Operator 部署 Prometheus 實現 K8S 監控-CSDN博客
這篇文章!?