基于Kubernetes的Apache Pulsar云原生架構解析與集群部署指南（下）

文章目錄

k8s安裝部署Pulsar集群
前期準備
- 版本要求
安裝 Pulsar Helm chart
管理pulsar
- Clusters
- Brokers
- Topic

k8s安裝部署Pulsar集群

前期準備

版本要求

Kubernetes 集群，版本 1.14 或更高版本
Helm v3（3.0.2 或更高版本）
數據持久化（本文使用NFS CSI動態供給）

K8s集群信息

[root@master ~]# kubectl get node
NAME      STATUS   ROLES           AGE     VERSION
master    Ready    control-plane   77d     v1.24.2
worker1   Ready    <none>          77d     v1.24.2
worker2   Ready    <none>          77d     v1.24.2
worker3   Ready    <none>          4d18h   v1.24.2

Helm安裝

wget https://get.helm.sh/helm-v3.17.2-linux-amd64.tar.gz
tar xf helm-v3.17.2-linux-amd64.tar.gz
chown root:root helm
mv linux-amd64/helm /usr/local/sbin/
helm version

NFS server部署
本文使用yum 包部署nfs server

nfs server主機安裝nfs相關軟件包

yum install nfs-utils -y

啟動nfs server 相關服務

sudo systemctl start rpcbind
systemctl enable rpcbind
systemctl start nfs-server
systemctl enable nfs-server

配置nfs共享目錄導出
（/nfsdata目錄為主機單獨一塊磁盤的掛載點，需提前配置完成。）

vim /etc/exports
/nfsdata 192.168.61.0/24(rw,sync,no_subtree_check,no_root_squash)

導出共享目錄
exportfs -ra

為數據持久化部署csi storageclass
本文使用nfs csi storageclass，但nfs不具有高可用性，生產環境建議使用ceph，glusterfs分布式的存儲解決方案。
在 Kubernetes 中，NFS CSI（Container Storage Interface）StorageClass 提供了一種標準化的方式來動態供應基于 NFS（Network File System）的持久化存儲卷。通過使用 NFS CSI 驅動程序和相應的 StorageClass，用戶可以更方便地請求、分配和管理持久化存儲資源，而無需手動創建 PersistentVolume（PV）。

nfs csi storageclass安裝部署步驟參考：
https://github.com/kubernetes-csi/csi-driver-nfs

安裝 Pulsar Helm chart

添加 Pulsar repo

helm repo add apache https://pulsar.apache.org/charts
helm repo update

列出可部署的版本，可以看出chart和pulsar的版本并非一一對應。

本文要安裝的pulsar版本為2.10.2，因此選擇pulsar heml對應版本為3.0.0

[root@master pulsar-helm-chart]# helm search repo apache/pulsar --versions
NAME            CHART VERSION   APP VERSION     DESCRIPTION                            
apache/pulsar   4.0.0           4.0.3           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.9.0           4.0.2           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.8.0           4.0.1           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.7.0           4.0.0           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.6.0           3.0.7           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.5.0           3.0.6           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.4.1           3.0.5           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.4.0           3.0.4           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.3.1           3.0.3           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.3.0           3.0.2           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.2.0           3.0.2           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.1.0           3.0.2           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.0.0           2.10.2          Apache Pulsar Helm chart for Kubernetes
apache/pulsar   2.9.4           2.9.3           Apache Pulsar Helm chart for Kubernetes

從helm倉庫中下載指定版本的pulsar helm chart 安裝包

helm pull apache/pulsar --version 3.0.0
pulsar-3.0.0.tgztar xf pulsar-3.0.0.tgz
cd pulsarls -l

總用量 68K

-rw-r--r-- 1 root root  253 10月 21  2022 Chart.lock
drwxr-xr-x 3 root root 4.0K  4月 23 22:01 charts
-rw-r--r-- 1 root root  529 10月 21  2022 Chart.yaml
-rw-r--r-- 1 root root  13K 10月 21  2022 LICENSE
-rw-r--r-- 1 root root  167 10月 21  2022 NOTICE
drwxr-xr-x 2 root root 4.0K  4月 23 22:01 templates
-rw-r--r-- 1 root root  32K  4月 24 10:48 values.yaml

通過修改配置文件指定pulsar版本

vim scripts/pulsar/common_auth.sh
…
PULSAR_VERSION="2.10.2"
…

參考以下內容修改values.yaml文件

namespace: "pulsar"
initialize: true
volumes:persistence: truelocal_storage: false
…
components:# zookeeperzookeeper: true
…# pulsar managerpulsar_manager: true
…
images:zookeeper:repository: apachepulsar/pulsar-all# uses defaultPulsarImageTag when unspecifiedtag: 2.10.2pullPolicy: IfNotPresentbookie:repository: apachepulsar/pulsar-all# uses defaultPulsarImageTag when unspecifiedtag: 2.10.2pullPolicy: IfNotPresent
…pulsar_manager:repository: apachepulsar/pulsar-managertag: v0.3.0pullPolicy: IfNotPresent
hasCommand: false
…
zookeeper:
volumes:persistence: truedata:storageClassName: nfs-csi
…
bookkeeper:
volumes:
persistence: true
journal:name: journalsize: 10Gilocal_storage: falsestorageClassName: nfs-csiledgers:name: ledgerssize: 50Gilocal_storage: falsestorageClassName: nfs-csi
…
kube-prometheus-stack:enabled: false
…

使用 Pulsar Helm將 Pulsar 集群安裝到 Kubernetes。

helm install \--values values.yaml \--set initialize=true \--version 3.0.0 \pulsar01 apache/pulsar

查看pod的狀態

[root@master pulsar]# kubectl get pod -n pulsar 
NAME                                      READY   STATUS      RESTARTS      AGE
pulsar01-bookie-0                         1/1     Running     0             13h
pulsar01-bookie-1                         1/1     Running     0             13h
pulsar01-bookie-2                         1/1     Running     0             13h
pulsar01-bookie-init-zb9gt                0/1     Completed   0             13h
pulsar01-broker-0                         1/1     Running     3 (12h ago)   13h
pulsar01-broker-1                         1/1     Running     3 (12h ago)   13h
pulsar01-broker-2                         1/1     Running     1 (13h ago)   13h
pulsar01-proxy-0                          1/1     Running     0             13h
pulsar01-pulsar-init-6zcx5                0/1     Completed   0             13h
pulsar01-pulsar-manager-f856cd9c6-fzj72   1/1     Running     0             33m
pulsar01-recovery-0                       1/1     Running     0             13h
pulsar01-toolset-0                        1/1     Running     0             13h
pulsar01-zookeeper-0                      1/1     Running     0             13h
pulsar01-zookeeper-1                      1/1     Running     0             13h
pulsar01-zookeeper-2                      1/1     Running     0             13h

查看svc

[root@master pulsar]# kubectl get svc -n pulsar 
NAME                      TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
pulsar01-bookie           ClusterIP      None            <none>        3181/TCP,8000/TCP                     13h
pulsar01-broker           ClusterIP      None            <none>        8080/TCP,6650/TCP                     13h
pulsar01-proxy            LoadBalancer   10.96.97.171    <pending>     80:31682/TCP,6650:30689/TCP           13h
pulsar01-pulsar-manager   LoadBalancer   10.100.16.216   <pending>     9527:31991/TCP                        33m
pulsar01-recovery         ClusterIP      None            <none>        8000/TCP                              13h
pulsar01-toolset          ClusterIP      None            <none>        <none>                                13h
pulsar01-zookeeper        ClusterIP      None            <none>        8000/TCP,2888/TCP,3888/TCP,2181/TCP   13h

查看pvc

[root@master pulsar]# kubectl get pvc -n pulsar 
NAME                                           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pulsar01-bookie-journal-pulsar01-bookie-0      Bound    pvc-c86bb6dd-d98e-4b0f-b9f0-c67c169a5255   10Gi       RWO            nfs-csi        13h
pulsar01-bookie-journal-pulsar01-bookie-1      Bound    pvc-e7c2935d-d4cf-4cad-8d63-5dc87dbcae5a   10Gi       RWO            nfs-csi        13h
pulsar01-bookie-journal-pulsar01-bookie-2      Bound    pvc-9dfa73ec-8512-495a-ad6d-d3141a97128d   10Gi       RWO            nfs-csi        13h
pulsar01-bookie-journal-pulsar01-bookie-3      Bound    pvc-1f038b9a-4d0d-46c8-89ca-dc8f630ade76   10Gi       RWO            nfs-csi        13h
pulsar01-bookie-ledgers-pulsar01-bookie-0      Bound    pvc-8fcb5205-eef2-4903-90e4-68499746c8d8   50Gi       RWO            nfs-csi        13h
pulsar01-bookie-ledgers-pulsar01-bookie-1      Bound    pvc-4412bca2-4314-4fd6-9fb0-85bf1472fd53   50Gi       RWO            nfs-csi        13h
pulsar01-bookie-ledgers-pulsar01-bookie-2      Bound    pvc-a0fae489-99f9-4f39-ba7d-a543d2691502   50Gi       RWO            nfs-csi        13h
pulsar01-bookie-ledgers-pulsar01-bookie-3      Bound    pvc-0124e56b-0c28-4b24-92be-019acb8331a8   50Gi       RWO            nfs-csi        13h
pulsar01-zookeeper-data-pulsar01-zookeeper-0   Bound    pvc-81219d17-27f6-4ee0-864c-9f1a296f85cd   20Gi       RWO            nfs-csi        13h
pulsar01-zookeeper-data-pulsar01-zookeeper-1   Bound    pvc-8571b0c0-9ad9-4a5c-9582-0a81fda4c6ff   20Gi       RWO            nfs-csi        13h
pulsar01-zookeeper-data-pulsar01-zookeeper-2   Bound    pvc-ab41f541-c9a1-475b-ab07-cb1b90b47c87   20Gi       RWO            nfs-csi        13h

在修改values.yaml后可以使用upgrade更新部署

[root@master pulsar]# helm upgrade --values values.yaml --set initialize=true --version 3.0.0 pulsar01 apache/pulsar
Release "pulsar01" has been upgraded. Happy Helming!
NAME: pulsar01
LAST DEPLOYED: Thu Apr 24 10:48:19 2025
NAMESPACE: default
STATUS: deployed
REVISION: 4
TEST SUITE: None

c如何刪除集群

[root@master pulsar]# helm list
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
pulsar01        default         4               2025-04-24 10:48:19.873604446 +0800 CST deployed        pulsar-3.0.0    2.10.2    [root@master pulsar]# helm uninstall pulsar01 --namespace pulsar
release "pulsar01 " uninstalled[root@master pulsar]# helm list --namespace pulsar
NAME    NAMESPACE       REVISION        UPDATED STATUS  CHART   APP VERSION

管理pulsar

Clusters

獲取現有集群的配置

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin clusters get pulsar01
{"serviceUrl" : "http://pulsar01-broker.pulsar.svc.cluster.local:8080/","serviceUrlTls" : "https://pulsar01-broker.pulsar.svc.cluster.local:8443/","brokerServiceUrl" : "pulsar://pulsar01-broker.pulsar.svc.cluster.local:6650/","brokerServiceUrlTls" : "pulsar+ssl://pulsar01-broker.pulsar.svc.cluster.local:6651/","brokerClientTlsEnabled" : false,"tlsAllowInsecureConnection" : false,"brokerClientTlsEnabledWithKeyStore" : false,"brokerClientTlsTrustStoreType" : "JKS"
}

獲取 Pulsar實例中所有集群的列表

I have no name!@pulsar01-toolset-0:/pulsar/bin$./pulsar-admin clusters list
pulsar01I have no name!@pulsar01-toolset-0:/pulsar/bin$ curl -X GET "http://pulsar01-proxy.pulsar.svc.cluster.local/admin/v2/clusters"v2/clusters"
["pulsar01"]
Tenants

列出所有租戶

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin tenants list
public
pulsarI have no name!@pulsar01-toolset-0:/pulsar/bin$ curl -X GET "http://pulsar01-proxy.pulsar.svc.cluster.local/admin/v2/tenants"
["public","pulsar"]

新創建1個名為my-tenant的租戶

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin tenants create my-tenant

創建租戶時，您可以選擇使用-r/–admin-roles 標志分配管理員角色，并使用-c/–

allowed-clusters標志分配集群。您可以以逗號分隔的列表形式指定多個值。
I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin tenants create my-tenant2 -r role1 -c pulsar01curl -X PUT "http://pulsar01-proxy/admin/v2/tenants/my-tenant3" \
> -H "Content-Type: application/json" \
> -d '{
>   "allowedClusters": ["pulsar01"],
>   "adminRoles": ["role1"]
> }'

獲取現有租戶的配置。

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin tenants get my-tenant2                                                
{"adminRoles" : [ "role1" ],"allowedClusters" : [ "pulsar01" ]
}I have no name!@pulsar01-toolset-0:/pulsar$ curl -X GET "http://pulsar01-proxy/admin/v2/tenants/my-tenant3"
{"adminRoles":["role1"],"allowedClusters":["pulsar01"]}

刪除指定租戶

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin tenants delete my-tenantI have no name!@pulsar01-toolset-0:/pulsar$ curl -X DELETE "http://pulsar01-proxy/admin/v2/tenants/my-tenant"

Brokers

可以通過以下方式管理broker：

工具brokers的命令pulsar-admin
/admin/v2/brokers管理REST API的端點

列出所有活躍的broker

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin brokers list use 
pulsar01-broker-2.pulsar01-broker.pulsar.svc.cluster.local:8080
pulsar01-broker-1.pulsar01-broker.pulsar.svc.cluster.local:8080
pulsar01-broker-0.pulsar01-broker.pulsar.svc.cluster.local:8080curl -s -X GET "http://pulsar01-proxy.pulsar/admin/v2/brokers/pulsar01"    
["pulsar01-broker-2.pulsar01-broker.pulsar.svc.cluster.local:8080","pulsar01-broker-1.pulsar01-broker.pulsar.svc.cluster.local:8080","pulsar01-broker-0.pulsar01-broker.pulsar.svc.cluster.local:8080"]

列出指定broker擁有和服務的所有命名空間。

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin brokers namespaces use --url pulsar01-broker-2.pulsar01-broker.pulsar.svc.cluster.local:8080
pulsar/pulsar01/pulsar01-broker-2.pulsar01-broker.pulsar.svc.cluster.local:8080/0x00000000_0xffffffff    [broker_assignment=shared is_controlled=false is_active=true]
pulsar/pulsar01-broker-2.pulsar01-broker.pulsar.svc.cluster.local:8080/0x00000000_0xffffffff    [broker_assignment=shared is_controlled=false is_active=true]
public/functions/0x40000000_0x80000000    [broker_assignment=shared is_controlled=false is_active=true]
public/default/0x20000000_0x30000000    [broker_assignment=shared is_controlled=false is_active=true]

獲取broker的監控指標數據

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin broker-stats monitoring-metrics

此命令輸出的內容較多，為json格式，建議使用jq格式化，更易閱讀。
涵蓋了 Pulsar Broker 的多個關鍵方面，包括：

Managed Ledger 緩存性能。
Topic 加載時間。
ZooKeeper 交互延遲。
JVM 性能和資源使用。
Broker 客戶端連接狀態。
負載均衡資源使用。
查看broker的健康狀態

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin brokers healthcheck
Ok

查看broker的版本：

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin brokers version    
2.10.2

Topic

創建topic

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin topics create persistent://public/default/my-topic-1

創建3分區的topic

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin topics create-partitioned-topic persistent://public/default/my-partitioned-topic --partitions 3

查看topic狀態

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin topics stats persistent://public/default/my-topic-1I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin topics  partitioned-stats persistent://public/default/my-partitioned-topic --per-partition

Topic增加分區

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin topics update-partitioned-topic persistent://public/default/my-partitioned-topic --partitions 6

分區數量應與預期的吞吐量需求相匹配。例如：

如果你預計每秒需要處理 100,000 條消息，并且單個分區的吞吐量為 10,000 條/秒，則至少需要 10 個分區。
如果你的集群有 5 個 Broker，那么 10 個分區可以均勻分布在這些 Broker 上，每個 Broker 處理 2 個分區
查看指定topic的ledgerId

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin topics info-internal persistent://public/default/my-topic
{"version": 7,"creationDate": "2025-04-23T14:29:59.347Z","modificationDate": "2025-04-25T03:54:25.812Z","ledgers": [{"ledgerId": 11,"entries": 36,"size": 1870,"isOffloaded": false},{"ledgerId": 18,"entries": 1,"size": 48,"isOffloaded": false},{"ledgerId": 28,"isOffloaded": false}],