最近在接觸kafka,想著在自己的電腦安裝一套環境,docker也能行,但是還是想裝一些原生的軟件試試看,因此便想著整理一下brew的命令,這命令確實是方便,不需要下載tar包亂八七糟的東西,一鍵安裝
brew
Brew全稱叫Homebrew,是Mac系統上的軟件包管理工具;
常用命令
- brew services【查看所有brew管理的軟件】
> brew services
Name Status User File
redis started a58 ~/Library/LaunchAgents/homebrew.mxcl.redis.plist
zookeeper none
none代表沒有啟動
- brew services start xxx【啟動brew管理的軟件】
a58@192 kafka % brew services start zookeeper
==> Successfully started `zookeeper` (label: homebrew.mxcl.zookeeper)
- breew services stop 軟件名【停止】
- brew services restart 軟件名【重啟】
- brew upgrade 軟件名【更新軟件】
- brew list 或 brew services 【查看已安裝的軟件包列表】
- brew list --versions 軟件名【查看已安裝的軟件包版本】
- brew search 軟件名【搜索云端的軟件】
- brew install 軟件名【安裝】
- brew uninstall【卸載】
- cd /opt/homebrew/etc【已安裝軟件的配置文件位置】
brew安裝的軟件配置,一般都在這個地方,比如kafka的話,是在
/opt/homebrew/etc/kafka
目錄下
cd /opt/homebrew/bin【已安裝軟件的支持的命令】
brew安裝的軟件支持的命令,一般都在這個地方,在/opt/homebrew/bin
目錄,我們可以看一下kafka
有哪些命令
a58@192 bin % ls -l |grep kafka
lrwxr-xr-x 1 a58 admin 47 4 18 22:27 connect-distributed -> ../Cellar/kafka/3.3.1_1/bin/connect-distributed
lrwxr-xr-x 1 a58 admin 48 4 18 22:27 connect-mirror-maker -> ../Cellar/kafka/3.3.1_1/bin/connect-mirror-maker
lrwxr-xr-x 1 a58 admin 46 4 18 22:27 connect-standalone -> ../Cellar/kafka/3.3.1_1/bin/connect-standalone
lrwxr-xr-x 1 a58 admin 38 4 18 22:27 kafka-acls -> ../Cellar/kafka/3.3.1_1/bin/kafka-acls
lrwxr-xr-x 1 a58 admin 53 4 18 22:27 kafka-broker-api-versions -> ../Cellar/kafka/3.3.1_1/bin/kafka-broker-api-versions
lrwxr-xr-x 1 a58 admin 41 4 18 22:27 kafka-cluster -> ../Cellar/kafka/3.3.1_1/bin/kafka-cluster
lrwxr-xr-x 1 a58 admin 41 4 18 22:27 kafka-configs -> ../Cellar/kafka/3.3.1_1/bin/kafka-configs
lrwxr-xr-x 1 a58 admin 50 4 18 22:27 kafka-console-consumer -> ../Cellar/kafka/3.3.1_1/bin/kafka-console-consumer
lrwxr-xr-x 1 a58 admin 50 4 18 22:27 kafka-console-producer -> ../Cellar/kafka/3.3.1_1/bin/kafka-console-producer
lrwxr-xr-x 1 a58 admin 49 4 18 22:27 kafka-consumer-groups -> ../Cellar/kafka/3.3.1_1/bin/kafka-consumer-groups
lrwxr-xr-x 1 a58 admin 52 4 18 22:27 kafka-consumer-perf-test -> ../Cellar/kafka/3.3.1_1/bin/kafka-consumer-perf-test
lrwxr-xr-x 1 a58 admin 51 4 18 22:27 kafka-delegation-tokens -> ../Cellar/kafka/3.3.1_1/bin/kafka-delegation-tokens
lrwxr-xr-x 1 a58 admin 48 4 18 22:27 kafka-delete-records -> ../Cellar/kafka/3.3.1_1/bin/kafka-delete-records
lrwxr-xr-x 1 a58 admin 42 4 18 22:27 kafka-dump-log -> ../Cellar/kafka/3.3.1_1/bin/kafka-dump-log
lrwxr-xr-x 1 a58 admin 42 4 18 22:27 kafka-features -> ../Cellar/kafka/3.3.1_1/bin/kafka-features
lrwxr-xr-x 1 a58 admin 45 4 18 22:27 kafka-get-offsets -> ../Cellar/kafka/3.3.1_1/bin/kafka-get-offsets
lrwxr-xr-x 1 a58 admin 49 4 18 22:27 kafka-leader-election -> ../Cellar/kafka/3.3.1_1/bin/kafka-leader-election
lrwxr-xr-x 1 a58 admin 42 4 18 22:27 kafka-log-dirs -> ../Cellar/kafka/3.3.1_1/bin/kafka-log-dirs
lrwxr-xr-x 1 a58 admin 49 4 18 22:27 kafka-metadata-quorum -> ../Cellar/kafka/3.3.1_1/bin/kafka-metadata-quorum
lrwxr-xr-x 1 a58 admin 48 4 18 22:27 kafka-metadata-shell -> ../Cellar/kafka/3.3.1_1/bin/kafka-metadata-shell
lrwxr-xr-x 1 a58 admin 46 4 18 22:27 kafka-mirror-maker -> ../Cellar/kafka/3.3.1_1/bin/kafka-mirror-maker
lrwxr-xr-x 1 a58 admin 52 4 18 22:27 kafka-producer-perf-test -> ../Cellar/kafka/3.3.1_1/bin/kafka-producer-perf-test
lrwxr-xr-x 1 a58 admin 53 4 18 22:27 kafka-reassign-partitions -> ../Cellar/kafka/3.3.1_1/bin/kafka-reassign-partitions
lrwxr-xr-x 1 a58 admin 54 4 18 22:27 kafka-replica-verification -> ../Cellar/kafka/3.3.1_1/bin/kafka-replica-verification
lrwxr-xr-x 1 a58 admin 43 4 18 22:27 kafka-run-class -> ../Cellar/kafka/3.3.1_1/bin/kafka-run-class
lrwxr-xr-x 1 a58 admin 46 4 18 22:27 kafka-server-start -> ../Cellar/kafka/3.3.1_1/bin/kafka-server-start
lrwxr-xr-x 1 a58 admin 45 4 18 22:27 kafka-server-stop -> ../Cellar/kafka/3.3.1_1/bin/kafka-server-stop
lrwxr-xr-x 1 a58 admin 41 4 18 22:27 kafka-storage -> ../Cellar/kafka/3.3.1_1/bin/kafka-storage
lrwxr-xr-x 1 a58 admin 59 4 18 22:27 kafka-streams-application-reset -> ../Cellar/kafka/3.3.1_1/bin/kafka-streams-application-reset
lrwxr-xr-x 1 a58 admin 40 4 18 22:27 kafka-topics -> ../Cellar/kafka/3.3.1_1/bin/kafka-topics
lrwxr-xr-x 1 a58 admin 46 4 18 22:27 kafka-transactions -> ../Cellar/kafka/3.3.1_1/bin/kafka-transactions
lrwxr-xr-x 1 a58 admin 53 4 18 22:27 kafka-verifiable-consumer -> ../Cellar/kafka/3.3.1_1/bin/kafka-verifiable-consumer
lrwxr-xr-x 1 a58 admin 53 4 18 22:27 kafka-verifiable-producer -> ../Cellar/kafka/3.3.1_1/bin/kafka-verifiable-producer
lrwxr-xr-x 1 a58 admin 35 4 18 22:27 trogdor -> ../Cellar/kafka/3.3.1_1/bin/trogdor
lrwxr-xr-x 1 a58 admin 56 4 18 22:27 zookeeper-security-migration -> ../Cellar/kafka/3.3.1_1/bin/zookeeper-security-migration
lrwxr-xr-x 1 a58 admin 50 4 18 22:27 zookeeper-server-start -> ../Cellar/kafka/3.3.1_1/bin/zookeeper-server-start
lrwxr-xr-x 1 a58 admin 49 4 18 22:27 zookeeper-server-stop -> ../Cellar/kafka/3.3.1_1/bin/zookeeper-server-stop
lrwxr-xr-x 1 a58 admin 43 4 18 22:27 zookeeper-shell -> ../Cellar/kafka/3.3.1_1/bin/zookeeper-shell
1.安裝一套kafka環境
kafka4.0版本就不再需要zk做注冊中心了,但是我使用brew安裝發現版本是3.4.4,因此需要單獨安裝一下zookeeper;
1. 安裝kafka和zookeeper
brew install kafka
安裝完kafka之后,有必要就修改一下kafka的配置,端口9092,如果不修改則直接忽略【配置文件上面已提到過了】
是否具體需要zookeeper配合,其實只需要看一下server.properties文件中有沒有zookeeper.connect=localhost:2181
brew install zookeeper
Name Status User File
kafka started a58 ~/Library/LaunchAgents/homebrew.mxcl.kafka.plist
zookeeper none
2.啟動kafka和zookeeper
brew services start zookeeper
Name Status User File
grafana none
kafka started a58 ~/Library/LaunchAgents/homebrew.mxcl.kafka.plist
zookeeper started a58 ~/Library/LaunchAgents/homebrew.mxcl.zookeeper.plist
3.kafka小試牛刀
上面一開始我們就提到,brew安裝的軟件命令都在/opt/homebrew/bin
下,且使用ls -l |grep kafka
就可以查看到相關的命令;
那么我們試一下吧,看看能不能完成一次生產消費的case
1.創建一個topic=leo
a58@192 bin % kafka-topics --create --bootstrap-server localhost:9092 --topic leo
Created topic leo.
在查看一下是否真的有了
a58@192 bin % kafka-topics --list --bootstrap-server localhost:9092
leo
好,成功后就再繼續創一個vicky
,方便后續使用;
- kafka-topics : 管理
Kafka
主題,涵蓋創建、刪除、查看; - –create:表明要執行創建動作,具體什么動作看前面的主命令
- –bootstrap-server localhost:9092:指定
Kafka
集群的初始連接地址來作為命令的執行方 - –topic:主題名,此處表示要創建的
topic
名 - –list:此處表明列出所有
topic
列表
2.生產kafka消息
kafka-console-producer --bootstrap-server localhost:9092 --topic leo
輸入這段命令后會進入一個控制臺:輸入消息內容回車即發送成功了,如下圖,我發送了3條消息:
3.消費kafka消息
再開一個控制臺,執行下面的消費命令;
這里需要注意一個問題,kafka是支持按照指定偏移量offset
進行消費的,即我們可以有選擇的消費;
1.從offset=0開始消費topic
kafka-console-consumer --bootstrap-server localhost:9092 --topic leo --from-beginning
123
456
789
這命令意味著,不管生產方何時發送的消息,只要這個命令下去則從頭開始消費
2.從當前執行完命令后開始消費topic
kafka-console-consumer --bootstrap-server localhost:9092 --topic leo
3.從偏移量offset=[num]時消費
kafka-console-consumer --bootstrap-server localhost:9092 --topic leo --partition 0 --offset 5
這個命令,是必須要指定分區的:--partition 0
;我是本地環境,只有默認的分區,可以缺省,但生產環境不能缺省;
4.使用消費組
我加上一個命令:--group one
即指定消費組id=one
kafka-console-consumer --bootstrap-server localhost:9092 --topic leo --group one
生產者生產消息,當前消費者的控制臺,會收到消息;
假設我重新再開一個mac
控制臺,命令也是:
kafka-console-consumer --bootstrap-server localhost:9092 --topic leo --group one
即此時有兩個mac
消費者控制臺在消費,輸入以下的命令,會發現,只有其中的一個控制臺可以接收到消息,且只要一開始接收到消息的控制臺不關閉,則會一直接收消息,另一個控制臺消費者一點機會都沒有;
這里我提個問題,如果不指定group的情況下,開兩個消費者消費控制臺,會怎么樣?
假設把groupId改一下:two
kafka-console-consumer --bootstrap-server localhost:9092 --topic leo --group two
此時生產者控制臺繼續輸入內容回車發送消息,我們會發現兩個消費者消費的消息內容都是一樣的;
為什么會這樣呢,等我后面寫文章講一下kafka就知道了!