1.概念
etcd 是由GO語言編寫的分布式的、可靠的鍵值存儲系統,主要用于分布式系統中關鍵數據的存儲和服務發現。
2.核心概念
節點(Node)
每個運行 etcd 的實例被稱為一個節點。一個或多個節點可以組成一個集群。
集群(Cluster)
由多個節點組成的集合,這些節點共同工作以提供一致的數據存儲服務。通過 Raft 共識算法確保集群中各節點間數據的一致性。
鍵值對(Key-Value Pair)
etcd 存儲的基本單位是鍵值對,其中鍵和值都是字節數組。鍵用于唯一標識存儲的數據項,而值則包含實際的數據內容。
3.etcd
集群準備
節點 | IP地址 | 操作系統版本 | etcd版本 |
---|---|---|---|
etcd-node1 | 192.168.100.5 | Ubuntu 24.04.2 LTS | v3.6.4 |
etcd-node2 | 192.168.100.6 | Ubuntu 24.04.2 LTS | v3.6.4 |
etcd-node3 | 192.168.100.7 | Ubuntu 24.04.2 LTS | v3.6.4 |
3.1 配置IP地址
配置etcd-node1節點IP
sudo cat /etc/netplan/ens32-cloud-init.yaml
network:version: 2ethernets:ens32:dhcp4: falseaddresses:- "192.168.100.5/24"routes:- to: defaultvia: 192.168.100.254nameservers:addresses:- 114.114.114.114
sudo netplan apply
配置etcd-node2節點IP
sudo cat /etc/netplan/ens32-cloud-init.yaml
network:version: 2ethernets:ens32:dhcp4: falseaddresses:- "192.168.100.6/24"routes:- to: defaultvia: 192.168.100.254nameservers:addresses:- 114.114.114.114
sudo netplan apply
配置etcd-node3節點IP
sudo cat /etc/netplan/ens32-cloud-init.yaml
network:version: 2ethernets:ens32:dhcp4: falseaddresses:- "192.168.100.7/24"routes:- to: defaultvia: 192.168.100.254nameservers:addresses:- 114.114.114.114
sudo netplan apply
3.2 配置主機名
配置etcd-node1節點主機名
sudo hostnamectl set-hostname etcd-node1
配置etcd-node2節點主機名
sudo hostnamectl set-hostname etcd-node2
配置etcd-node3節點主機名
sudo hostnamectl set-hostname etcd-node3
3.3 配置主機名與IP解析
3個節點均需要執行
sudo cat >> /etc/hosts <<EOF
192.168.100.5 etcd-node1
192.168.100.6 etcd-node2
192.168.100.7 etcd-node3
EOF
3.4 關閉防火墻
3個節點均需要執行
sudo ufw stop
sudo ufw status
3.5 時鐘同步
3個節點均需要執行
sudo apt install chrony
sudo sed -i '/pool.*ubuntu\.pool\.ntp\.org/s/^/# /' /etc/chrony/chrony.conf
sudo sed -i 's/^pool ntp\.ubuntu\.com.*$/server ntp.aliyun.com iburst/' /etc/chrony/chrony.conf
sudo systemctl restart chrony
sudo chronyc sources
3.6 配置節點互信
3個節點均需要執行
sudo ssh-keygen
sudo cp /root/.ssh/id_rsa.pub /root/.ssh/authorized_keys
在etcd-node1節點上執行
sudo for i in 5 6 7
> do
> scp -r /root/.ssh 192.168.100.$i:/root/
> done
4.etcd
集群部署
4.1 下載etcd
3個節點均需要執行
sudo wget https://github.com/etcd-io/etcd/releases/download/v3.6.4/etcd-v3.6.4-linux-amd64.tar.gz
4.2解壓etcd
3個節點均需要執行
sudo tar xzvf etcd-v3.6.4-linux-amd64.tar.gz -C /usr/local
sudo ln -s /usr/local/etcd-v3.6.4-linux-amd64/ /usr/local/etcd
4.3復制解壓文件至系統標準可執行文件路徑中
3個節點均需要執行
sudo cp /usr/local/etcd/etcd* /usr/local/bin/
4.4創建etcd
用戶
3個節點均需要執行
sudo useradd --system --shell /bin/false --home-dir /var/lib/etcd etcd
4.5創建數據目錄
3個節點均需要執行
sudo mkdir -p /var/lib/etcd
sudo mkdir /var/lib/etcd/default.etcd
sudo chown -R etcd:etcd /var/lib/etcd /usr/local/etcd
4.6 創建etcd
配置文件
配置etcd-node1節點配置文件
sudo cat > /usr/local/etcd/etcd.conf <<EOF
ETCD_NAME="etcd-node1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.100.5:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.100.5:2379,http://127.0.0.1:2379"#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.100.5:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.100.5:2379"
ETCD_INITIAL_CLUSTER="etcd-node1=https://192.168.100.5:2380,etcd-node2=https://192.168.100.6:2380,etcd-node3=https://192.168.100.7:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF
配置etcd-node2節點配置文件
sudo cat > /usr/local/etcd/etcd.conf <<EOF
ETCD_NAME="etcd-node2"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.100.6:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.100.6:2379,http://127.0.0.1:2379"#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.100.6:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.100.6:2379"
ETCD_INITIAL_CLUSTER="etcd-node1=https://192.168.100.5:2380,etcd-node2=https://192.168.100.6:2380,etcd-node3=https://192.168.100.7:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF
配置etcd-node3節點配置文件
sudo cat > /usr/local/etcd/etcd.conf <<EOF
ETCD_NAME="etcd-node3"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.100.7:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.100.7:2379,http://127.0.0.1:2379"#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.100.7:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.100.7:2379"
ETCD_INITIAL_CLUSTER="etcd-node1=https://192.168.100.5:2380,etcd-node2=https://192.168.100.6:2380,etcd-node3=https://192.168.100.7:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF
相關參數說明
參數 | 說明 |
---|---|
ETCD_NAME | 當前etcd節點名稱 |
ETCD_DATA_DIR | 數據存儲目錄 |
ETCD_LISTEN_CLIENT_URLS | 當前節點通過該地址監聽客戶端發送的信息 |
ETCD_LISTEN_PEER_URLS | 當前節點通過該地址監聽集群其他節點發送的信息 |
ETCD_INITIAL_ADVERTISE_PEER_URLS | 集群的其他節點通過該地址與當前節點通信 |
ETCD_ADVERTISE_CLIENT_URLS | 客戶端通過該地址與當前節點通信 |
ETCD_INITIAL_CLUSTER | 當前集群的所有節點信息,當前節點根據此信息與其他節點取得聯系 |
ETCD_INITIAL_CLUSTER_TOKEN | 用于區分不同的集群,同一集群的所有節點配置相同的值 |
ETCD_INITIAL_CLUSTER_STATE | 本次是否為新建集群,取值為 new 或者 existing |
4.7下載cfssl
證書生成工具
在etcd-node1節點上下載cfssl
wget https://github.com/cloudflare/cfssl/releases/download/v1.6.5/cfssl_1.6.5_linux_amd64
在etcd-node1節點上下載cfssljson
wget https://github.com/cloudflare/cfssl/releases/download/v1.6.5/cfssljson_1.6.5_linux_amd64
4.8授權并移至系統標準可執行文件路徑中
在etcd-node1節點上執行
chmod +x cfssl_1.6.5_linux_amd64chmod +x cfssljson_1.6.5_linux_amd64mv cfssl_1.6.5_linux_amd64 /usr/local/bin/cfssljsonmv cfssljson_1.6.5_linux_amd64 /usr/local/bin/cfssljson
4.9創建CA證書
在etcd-node1節點上配置CA證書策略
mkdir /usr/local/etcd/ssl
cat > /usr/local/etcd/ssl/ca-config.json <<EOF
{"signing": {"default": {"expiry": "87600h"},"profiles": {"etcd-server": {"usages": ["signing","key encipherment","client auth","server auth"],"expiry": "87600h"}}}
}
EOF
在etcd-node1節點上配置CA證書請求文件
cat > /usr/local/etcd/ssl/ca-csr.json <<EOF
{"CN": "My etcd CA","key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","L": "LANZHOU","O": "LZ","ST": "LANZHOU","OU": "CN"}],"ca": {"expiry": "87600h"}
}
EOF
在etcd-node1節點上生成CA證書
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
4.10創建etcd證書
在etcd-node1節點上配置etcd請求文件
cat > /usr/local/etcd/ssl/etcd-server.json <<EOF
{"CN": "etcd","hosts": ["127.0.0.1","192.168.100.5","192.168.100.6","192.168.100.7"],"key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","L": "LANZHOU","ST": "LANZHOU","OU": "CN"}]
}
EOF
在etcd-node1節點上生成 Etcd 證書
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd-server etcd-server.json | cfssljson -bare etcd-server
說明:
參數 | 說明 |
---|---|
-ca-key | 指定CA證書機構的私鑰 |
-config | 指定CA證書策略 |
-profile | 指定使用CA證書策略 |
etcd-server.pem | 證書/公鑰 |
etcd-server-key.pem | 私鑰 |
4.7創建 systemd 服務
3個節點均需要執行
sudo cat > /usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target[Service]
Type=notify
EnvironmentFile=-/usr/local/etcd/etcd.conf
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/local/bin/etcd \--cert-file=/usr/local/etcd/ssl/etcd-server.pem \--key-file=/usr/local/etcd/ssl/etcd-server-key.pem \--trusted-ca-file=/usr/local/etcd/ssl/ca.pem \--peer-cert-file=/usr/local/etcd/ssl/etcd-server.pem \--peer-key-file=/usr/local/etcd/ssl/etcd-server-key.pem \--peer-trusted-ca-file=/usr/local/etcd/ssl/ca.pem \--peer-client-cert-auth \--client-cert-auth
Restart=on-failure
RestartSec=5
LimitNOFILE=65536[Install]
WantedBy=multi-user.target
EOF
4.8 啟動etcd
3個節點均需要執行
sudo systemctl daemon-reload
sudo systemctl enable etcd
sudo systemctl start etcd
sudo systemctl status etcd
4.9查看集群成員
任意節點執行
sudo etcdctl member list
4.10查看集群節點健康狀態
ETCDCTL_API=3 /usr/local/bin/etcdctl --write-out=table --cacert=/usr/local/etcd/ssl/ca.pem --cert=/usr/local/etcd/ssl/etcd-server.pem --key=/usr/local/etcd/ssl/etcd-server-key.pem --endpoints=https://192.168.100.5:2379,https://192.168.100.6:2379,https://192.168.100.7:2379 endpoint health