Easysearch 可以使用 AWS S3 作為遠程存儲庫，進行索引的快照（Snapshot）備份和恢復。同時，Easysearch 內置了 S3 插件，無需額外安裝。以下是完整的配置和操作步驟。
1. 在 AWS S3 上創建存儲桶
- 登錄 AWS 控制臺，進入 S3 服務。
- 創建一個新存儲桶（例如
easysearch-backups）。 - 啟用版本控制（可選，但推薦）。
- 權限配置：確保 IAM 角色具有訪問 S3 的權限。
{
"Version": "2012-10-17",
"Statement": [{"Action": ["s3:ListBucket"],"Effect": "Allow","Resource": ["arn:aws:s3:::s3-bucket-name"]},{"Action": ["s3:GetObject","s3:PutObject","s3:DeleteObject"],"Effect": "Allow","Resource": ["arn:aws:s3:::s3-bucket-name/*"]}
]
}
2. 在 Console 上注冊 S3 作為快照存儲庫
使用 Console DevTools 或 API
在 Easysearch 的 DevTools 執行：
PUT _snapshot/my_s3_repository
{"type": "s3","settings": {"bucket": "easysearch-backups","base_path": ""}
}
注意：
bucket需要填寫你的 S3 存儲桶名稱。region需要替換成你的 AWS S3 所在區域，SDK 默認美東區。- 如果 Bucket 在中國區，還需添加
endpoint: https://s3.<region>.amazonaws.com.cn參數。
3. 創建快照
一旦 my_s3_repository 注冊完成，就可以創建快照：
PUT _snapshot/my_s3_repository/my_snapshot_001
{"indices": "my_index","include_global_state": false
}
查看當前存儲的快照：
GET _snapshot/my_s3_repository/_all
4. 從 AWS S3 還原快照
當你需要恢復索引時：
POST _snapshot/my_s3_repository/my_snapshot_001/_restore
{"indices": "my_index","rename_pattern": "my_index","rename_replacement": "restored_my_index"
}
說明：這會從
my_snapshot_001還原my_index，但以restored_my_index命名，避免與現有索引沖突。
如果要直接覆蓋原索引（確保 my_index 為空或已刪除）：
POST _snapshot/my_s3_repository/my_snapshot_001/_restore
{"indices": "my_index","ignore_unavailable": true,"include_global_state": false
}
5. 可能的錯誤與解決方案
| 錯誤信息 | 可能原因 | 解決方案 |
|---|---|---|
repository_s3 plugin not installed | 沒有安裝 repository-s3 插件 | 運行 bin/elasticsearch-plugin install repository-s3 并重啟 |
NoSuchBucket | S3 存儲桶不存在 | 確保 S3 存儲桶名稱正確 |
AccessDenied | 權限不足 | 確保 S3 存儲桶策略正確，檢查 IAM 角色 |
index_closed_exception | 目標索引已關閉 | 先 POST my_index/_open 再恢復 |
index_already_exists_exception | 目標索引已存在 | 先 DELETE my_index 再恢復 |
6. 快照恢復常見錯誤排查
報錯 1：無法連接到 S3
{"error": {"root_cause": [{"type": "repository_verification_exception","reason": "[my_s3_repository] path [/] is not accessible on master node"}],"type": "repository_verification_exception","reason": "[my_s3_repository] path [/] is not accessible on master node","caused_by": {"type": "i_o_exception","reason": "Unable to upload object [//tests-sXkmh3q5ThCCIX2VJp609g/master.dat] using a single upload","caused_by": {"type": "sdk_client_exception","reason": "Failed to connect to service endpoint: ","caused_by": {"type": "socket_timeout_exception","reason": "Connect timed out"}}}},"status": 500
}
解決方案：
- 在 keystore 中添加 AWS 憑證：
sudo ./bin/easysearch-keystore add s3.client.default.access_key sudo ./bin/easysearch-keystore add s3.client.default.secret_key - 如果運行在 EC2 上，確保實例掛載了 IAM Role。
{"error": {"root_cause": [{"type": "repository_verification_exception","reason": "[my_s3_repositor1] path is not accessible on master node"}],"type": "repository_verification_exception","reason": "[my_s3_repositor1] path is not accessible on master node","caused_by": {"type": "i_o_exception","reason": "Unable to upload object [tests-sUUzs-mTSZeYw1qk372DkQ/master.dat] using a single upload","caused_by": {"type": "sdk_client_exception","reason": "The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/"}}},"status": 500
}
報錯 2：索引已存在，無法恢復
{"error": {"root_cause": [{"type": "snapshot_restore_exception","reason": "[my_s3_repository:1/9gIDCgSySwKzQqEYvaGM_w] cannot restore index [my_index] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"}],"type": "snapshot_restore_exception","reason": "[my_s3_repository:1/9gIDCgSySwKzQqEYvaGM_w] cannot restore index [my_index] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"},"status": 500
}
解決方案：
- 刪除現有索引后恢復：
DELETE /my_index - 關閉索引后恢復：
POST /my_index/_close - 恢復為新的索引名稱：
POST _snapshot/my_s3_repository/1/_restore {"indices": "my_index","rename_pattern": "my_index","rename_replacement": "restored_my_index" }
報錯 3：權限錯誤
{"error": {"root_cause": [{"type": "security_exception","reason": "no permissions for [] and User [name=admin, external_roles=[admin]]"}],"type": "security_exception","reason": "no permissions for [] and User [name=admin, external_roles=[admin]]"},"status": 403
}
解決方案：
- 確保用戶有
manage_snapshots角色權限。 - 排除
.security索引或全局狀態，否則無法恢復。
POST _snapshot/my_s3_repositor1/snapshot_002/_restore
{"indices": "-.security","ignore_unavailable": true,"include_global_state": false
}
📌 存儲庫（Repository）管理 API
存儲庫用于存儲快照，Elasticsearch 支持 AWS S3、GCS、本地等存儲。
1?? 查看所有已注冊的存儲庫
GET _snapshot/_all
示例返回：
{"my_s3_repository": {"type": "s3","settings": {"bucket": "es-snapshots-bucket","region": "us-east-1"}}
}
2?? 查看特定存儲庫信息
GET _snapshot/my_s3_repository
3?? 創建存儲庫（AWS S3 示例）
PUT _snapshot/my_s3_repository
{"type": "s3","settings": {"bucket": "es-snapshots-bucket",}
}
4?? 刪除存儲庫
DELETE _snapshot/my_s3_repository
? 刪除存儲庫不會刪除快照，需要手動刪除快照！
📌 快照（Snapshot）管理 API
快照用于備份和恢復索引數據。
1?? 創建快照
備份特定索引
PUT _snapshot/my_s3_repository/snapshot_001
{"indices": "my_index","include_global_state": false
}
備份所有索引
PUT _snapshot/my_s3_repository/snapshot_002
{"include_global_state": true
}
2?? 查看所有快照
GET _snapshot/my_s3_repository/_all
3?? 查看特定快照信息
GET _snapshot/my_s3_repository/snapshot_001
4?? 刪除快照
DELETE _snapshot/my_s3_repository/snapshot_001
📌 快照恢復（Restore）API
恢復已備份的索引。
1?? 還原單個索引
POST _snapshot/my_s3_repository/snapshot_001/_restore
{"indices": "my_index","ignore_unavailable": true,"include_global_state": false
}
2?? 還原索引并重命名
POST _snapshot/my_s3_repository/snapshot_001/_restore
{"indices": "my_index","rename_pattern": "my_index","rename_replacement": "restored_my_index"
}
3?? 還原所有索引
POST _snapshot/my_s3_repository/snapshot_002/_restore
📌 快照狀態 API
查詢快照的執行狀態。
1?? 查看當前快照任務
GET _snapshot/_status
2?? 查看特定快照狀態
GET _snapshot/my_s3_repository/snapshot_001/_status
| API | 用途 |
|---|---|
GET _snapshot/_all | 查看所有存儲庫 |
GET _snapshot/my_s3_repository | 查看特定存儲庫 |
PUT _snapshot/my_s3_repository | 創建存儲庫 |
DELETE _snapshot/my_s3_repository | 刪除存儲庫 |
PUT _snapshot/my_s3_repository/snapshot_001 | 創建快照 |
GET _snapshot/my_s3_repository/_all | 查看所有快照 |
GET _snapshot/my_s3_repository/snapshot_001 | 查看快照詳情 |
DELETE _snapshot/my_s3_repository/snapshot_001 | 刪除快照 |
POST _snapshot/my_s3_repository/snapshot_001/_restore | 還原快照 |
GET _snapshot/_status | 查看快照狀態 |
🚀 通過本文，你可以高效地使用 AWS S3 進行 Easysearch 快照備份和恢復，并排查可能的錯誤，確保集群數據安全無憂！
