1、需求

遷移部分新集群沒有的索引和數據

?

2、工具elasticdump

Elasticdump 的工作原理是將輸入發送到輸出 。兩者都可以是 elasticsearch URL 或 File

?

2.1 mac安裝

前置：已經安裝有npm

npm install elasticdump -g

（-g 全局）

報錯：

npm WARN ERESOLVE overriding peer dependency

npm WARN While resolving: requestretry@7.1.0

npm WARN Found: peer request@“2..” from the root project

npm WARN npm WARN Could not resolve dependency:

npm WARN peer request@“2..” from the root project

npm ERR! code EACCES

npm ERR! syscall mkdir

npm ERR! path /Users/miuye/.npm/_cacache/content-v2/sha512/cd/26

npm ERR! errno EACCES

npm ERR!

npm ERR! Your cache folder contains root-owned files, due to a bug in

npm ERR! previous versions of npm which has since been addressed.

npm ERR!

npm ERR! To permanently fix this problem, please run:

npm ERR! sudo chown -R 501:20 “/Users/miuye/.npm”

問題

??1、依賴沖突 2、權限不足

解決

# 升級相關包
sudo npm install requestretry@latest -g

# sudo重新執行
sudo npm install elasticdump -g

2.2 elasticdump文檔

https://github.com/taskrabbit/elasticsearch-dump

?

3、遷移

遷移mapping

elasticdump --input=“http://username:password@testing.es.com:9200/my_index” --output=“http://username:password@staging.es.com:9200/$INDEX_NAME” --type=mapping

遷移data

elasticdump --input=“http://username:password@testing.es.com:9200/my_index” --output=“http://username:password@staging.es.com:9200/$INDEX_NAME” --type=data

【注意】：鑒權的用戶名和密碼寫在url中，以@分割，無鑒權就只寫es url

??如果目的es中已經存在對應資源，會報錯，可以設置--skip-existing=true，如果資源存在，不報錯

??我使用的腳本進行對比和遷移，如果需要新集群的配置信息和新集群保持一致，可以先遷移setting

#!/bin/bash# 源集群和目標集群的配置
SOURCE_ES_URL="http://testing.es.com:9200"
TARGET_ES_URL="http://staging.es.com:9200"
SOURCE_USERNAME="testing"
SOURCE_PASSWORD="123456"
TARGET_USERNAME="staging"
TARGET_PASSWORD="123456"# 獲取源集群中所有索引的名稱
INDEXES=$(curl -u $SOURCE_USERNAME:$SOURCE_PASSWORD -s -X GET "$SOURCE_ES_URL/_cat/indices?h=index&s=index")while IFS= read -r INDEX_NAME; do# 只遷移以_dev結尾的if [[ "$INDEX_NAME" == *_dev ]]; then# 查詢新集群索引INDEX_EXISTS=$(curl -u $TARGET_USERNAME:$TARGET_PASSWORD -s -o /dev/null -w "%{http_code}" -X GET "$TARGET_ES_URL/$INDEX_NAME")# 檢查 curl 命令是否成功if [ $? -ne 0 ]; thenecho "Failed to check index existence for $INDEX_NAME"continuefiif [ "$INDEX_EXISTS" -eq 200 ]; thenecho "Index $INDEX_NAME exists, jump"elif [ "$INDEX_EXISTS" -eq 404 ]; thenecho "Index $INDEX_NAME not exists, transfer..."# 遷移結構elasticdump --input="http://$SOURCE_USERNAME:$SOURCE_PASSWORD@testing.es.com:9200/$INDEX_NAME" --output="http://$TARGET_USERNAME:$TARGET_PASSWORD@staging.es.com:9200/$INDEX_NAME" --type=mapping# 遷移數據elasticdump --input="http://$SOURCE_USERNAME:$SOURCE_PASSWORD@testing.es.com:9200/$INDEX_NAME" --output="http://$TARGET_USERNAME:$TARGET_PASSWORD@staging.es.com:9200/$INDEX_NAME" --type=dataelseecho "Unknown $INDEX_EXISTS for $INDEX_NAME"fifi
done <<< "$INDEXES"