文章目錄
- 簡介
- 常用請求
- 創建映射
- 寫入數據
- 查詢數據
- 其他
- 安裝
- Docker方式
- 安裝OpenSearch
- 安裝OpenSearchDashboard
- Docker-Compose方式
- Docker-Compose安裝
- 1.設置主機環境
- 2.下載docker-compose.yml文件
- 3.啟動docker-compose
- 4.驗證
- 問題
- 問題1:IPv4 forwarding is disabled. Networking will not work
- 問題2:OpenSearch Dashboards 啟用中文界面
簡介
官網:
-
https://opensearch.org/
-
https://opensearch.org/docs/2.7/
這里我以2.7版本示例
前提docker已完成安裝
我的業務數據是個json如下:
{"title": "The Wind Rises","created_on": "2023-11-22T09:28:48+00:00","type": "order","user": "eddie","price": 24.99
}
當添加到opensearch時,OpenSearch 會添加一些元數據如下:
{"_index": "<index-name>","_type": "_doc","_id": "<document-id>","_version": 1,"_source": {"title": "The Wind Rises","created_on": "2023-11-22T09:28:48+00:00","type": "order","user": "eddie","price": 24.99}
}
常用請求
在Dev Tools執行
創建映射
PUT laker-index
{"mappings": {"properties": {"title": {"type": "text"},"created_on": {"type": "date"},"type": {"type": "keyword"},"user": {"type": "keyword"},"price": {"type": "half_float"}}}
}
寫入數據
POST _bulk
{"index":{"_index":"laker-index","_id":0}}
{"title":"The Wind Rises today is nice day","created_on":"2023-11-21T09:28:48+00:00","type":"order","user":"eddie","price":24.99}
{"index":{"_index":"laker-index","_id":1}}
{"title":"The Wind Rises","created_on":"2023-11-22T09:28:48+00:00","type":"sale","user":"eddie","price":4.99}
{"index":{"_index":"laker-index","_id":2}}
{"title":"The Wind Rises hello ","created_on":"2023-11-23T09:28:48+00:00","type":"order","user":"laker","price":34.99}
{"index":{"_index":"laker-index","_id":3}}
{"title":"The Wind Rises laker","created_on":"2023-11-22T09:28:48+00:00","type":"sale","user":"laker","price":14.99}
{"index":{"_index":"laker-index","_id":4}}
{"title":"The Wind Rises laker","created_on":"2023-11-21T09:28:48+00:00","type":"order","user":"laker","price":44.99}
查詢數據
GET laker-index/_search
{"size": 1,"query": {"match": {"title": "Rises"}}
}
其他
# 創建映射 mapping
PUT sample-index1
{"mappings": {"properties": {"year": { "type" : "text" },"age": { "type" : "integer" },"director":{ "type" : "text" },"ip_address" : {"type" : "ip","ignore_malformed": true},"timestamp": {"type": "date","format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"},"value": {"type": "double"}}}
}
# 將映射添加到現有索引
POST sample-index1/_mapping
{"properties": {"year": { "type" : "text" },"age": { "type" : "integer" },"director":{ "type" : "text" }}
}# 新增文檔
PUT /<index-name>/_doc/<document-id>
{"title": "The Wind Rises","release_date": "2013-07-20"
}
# 搜索索引
GET /<index-name>/_search?q=wind
# 刪除文檔
DELETE /<index-name>/_doc/<document-id>
# 查看集群設置
GET _cluster/settings?include_defaults=true
GET _cluster/settings
# 索引列表
GET _cat/indicesgreen open .opensearch-observability JbwykWeAQTeyjbiEue73xw 1 0 0 0 208b 208b
yellow open security-auditlog-2023.11.22 n_D4yJoKS4-YKLI1vpyQ9w 1 1 256 0 831.7kb 831.7kb
green open .kibana_92668751_admin_1 RG79AlpgTiOhAoT2uifInw 1 0 1 0 5.1kb 5.1kb
green open .kibana_1 d88uN7czQ2eGli3Kmqcg5A 1 0 0 0 208b 208b
green open .opendistro_security JG9gJ3hARWmngXidVL89mw 1 0 10 0 43.5kb 43.5kb# 系統狀態
GET _cat/health?v
# 節點信息
GET _cat/nodes?v# 獲取映射
GET <index>/_mapping{"sample-index1" : {"mappings" : {"year" : {"full_name" : "year","mapping" : {"year" : {"type" : "text"}}},"age" : {"full_name" : "age","mapping" : {"age" : {"type" : "integer"}}}}}
}# 獲取所有索引的所有映射
GET _mapping
# 獲取已安裝的插件
GET _cat/plugins
127296e3f35f opensearch-alerting 2.7.0.0
127296e3f35f opensearch-anomaly-detection 2.7.0.0
127296e3f35f opensearch-asynchronous-search 2.7.0.0
127296e3f35f opensearch-cross-cluster-replication 2.7.0.0
127296e3f35f opensearch-geospatial 2.7.0.0
127296e3f35f opensearch-index-management 2.7.0.0
127296e3f35f opensearch-job-scheduler 2.7.0.0
127296e3f35f opensearch-knn 2.7.0.0
127296e3f35f opensearch-ml 2.7.0.0
127296e3f35f opensearch-neural-search 2.7.0.0
127296e3f35f opensearch-notifications 2.7.0.0
127296e3f35f opensearch-notifications-core 2.7.0.0
127296e3f35f opensearch-observability 2.7.0.0
127296e3f35f opensearch-performance-analyzer 2.7.0.0
127296e3f35f opensearch-reports-scheduler 2.7.0.0
127296e3f35f opensearch-security 2.7.0.0
127296e3f35f opensearch-security-analytics 2.7.0.0
127296e3f35f opensearch-sql 2.7.0.0
索引模板
PUT _index_template/<template name>
# 創建一個名為的模板daily_logs,并將其應用于名稱與模式匹配的任何新索引logs-2020-01-*,并將其添加到my_logs別名中:
PUT _index_template/daily_logs
{"index_patterns": ["logs-2020-01-*"],"template": {"aliases": {"my_logs": {}},"settings": {"number_of_shards": 2,"number_of_replicas": 1},"mappings": {"properties": {"timestamp": {"type": "date","format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"},"value": {"type": "double"}}}}
}
# 如果您創建名為 的索引logs-2020-01-01,您可以看到它具有模板中的映射和設置:
PUT logs-2020-01-01
GET logs-2020-01-01
{"logs-2020-01-01": {"aliases": {"my_logs": {}},"mappings": {"properties": {"timestamp": {"type": "date","format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"},"value": {"type": "double"}}},"settings": {"index": {"creation_date": "1578107970779","number_of_shards": "2","number_of_replicas": "1","uuid": "U1vMDMOHSAuS2IzPcPHpOA","version": {"created": "7010199"},"provided_name": "logs-2020-01-01"}}}
}
安裝
Docker方式
安裝OpenSearch
1.拉取鏡像
docker pull opensearchproject/opensearch:2.7.0
2.啟動容器并暴露端口
docker run --name dev -d -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" -e "network.host=0.0.0.0" -e "plugins.security.disabled=false" opensearchproject/opensearch:2.7.0
- discovery.type=single-node 單節點模式
- network.host=0.0.0.0 在外部IP也能訪問
- plugins.security.disabled=true 禁用安全套件,就不用密碼了(這里開發環境可以設置為true)
3.向端口 9200 發送請求。默認用戶名和密碼為admin
。
curl https://localhost:9200 -ku 'admin:admin'
結果如下:
{"name" : "039dd59a68de","cluster_name" : "docker-cluster","cluster_uuid" : "rUa90awRQe2KKP7VxBmwxg","version" : {"distribution" : "opensearch","number" : "2.7.0","build_type" : "tar","build_hash" : "b7a6e09e492b1e965d827525f7863b366ef0e304","build_date" : "2023-04-27T21:43:09.523336706Z","build_snapshot" : false,"lucene_version" : "9.5.0","minimum_wire_compatibility_version" : "7.10.0","minimum_index_compatibility_version" : "7.0.0"},"tagline" : "The OpenSearch Project: https://opensearch.org/"
}
也可以這樣配置
# 復雜docker容器中的配置到本地
docker cp 039dd59a68de:/usr/share/opensearch/config/opensearch.yml .
# vim opensearch.yml 修改內容
network.host: 0.0.0.0
discovery.type: single-node
plugins.security.ssl.http.enabled: false
plugins.security.disabled=true
# 再將文件復制到容器
docker cp ./opensearch.yml 039dd59a68de:/usr/share/opensearch/config/opensearch.yml
# 重新啟動docker容器:
docker restart 039dd59a68de
# 查看日志
docker logs -f 039dd59a68de
安裝OpenSearchDashboard
1.拉取鏡像
docker pull opensearchproject/opensearch-dashboards:2.7.0
2.啟動容器并暴露端口
docker run --name dev-dash -d -p 5601:5601 -e "server.host=0.0.0.0" -e 'OPENSEARCH_HOSTS=["https://10.1.77.53:9200"]' opensearchproject/opensearch-dashboards:2.7.0# 直接禁用安全套件,沒有密碼
docker run --name dev-dash -d -p 5601:5601 -e "server.host=0.0.0.0" -e 'OPENSEARCH_HOSTS=["http://10.1.77.102:9200"]' -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchproject/opensearch-dashboards:2.7.0
- server.host=0.0.0.0 允許外部IP訪問
- opensearch.hosts=[“http://10.1.77.53:9200”] 配置opensearch地址
也可以這樣去改配置
# 復雜docker容器中的配置到本地 docker cp 40301ac292a7:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml . # vim opensearch_dashboards.yml 修改內容 opensearch.hosts: [https://10.1.77.53:9200] opensearch.username: admin opensearch.password: admin server.host: '0.0.0.0' # 再將文件復制到容器 docker cp ./opensearch_dashboards.yml 40301ac292a7:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml # 重新啟動docker容器: docker restart 40301ac292a7 # 查看日志 docker logs -f 40301ac292a7
3.訪問http://ip:5601/ 。默認用戶名和密碼為admin
。
Docker-Compose方式
Docker-Compose安裝
# 根據需要替換其中的版本,這里是 v2.21.0
sudo curl -L https://github.com/docker/compose/releases/download/v2.21.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
# 將可執行權限應用于二進制文件
sudo chmod +x /usr/local/bin/docker-compose
# 創建軟鏈
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
# 測試是否安裝成功
docker-compose --version
- Docker Compose version v2.21.0# 常用命令
# 自動完成包括構建鏡像,(重新)創建服務,啟動服務
docker-compose up
# 后臺執行
docker-compose up -d
# 此命令將會停止 up 命令所啟動的容器,并移除網絡
docker-compose down
# -v刪除所有卷的選項
docker-compose down -vdocker-compose start
docker-compose stop
docker-compose version
1.設置主機環境
vm.max_map_count
至少設置為 262144
# 檢測當前值
cat /proc/sys/vm/max_map_count
# 要增加該值,請將以下行添加到/etc/sysctl.conf
vim /etc/sysctl.conf
vm.max_map_count=262144
# 然后運行sudo sysctl -p重新加載
sudo sysctl -p
2.下載docker-compose.yml文件
- docker-compose.yml
mkdir -p /user/laker/opensearch
cd /user/laker/opensearch
wget https://opensearch.org/samples/docker-compose.yml
3.啟動docker-compose
# 啟動完要看日志,等1-2分鐘就差不多了,注意看日志問題
docker-compose up
4.驗證
訪問http://ip:5601/ 。默認用戶名和密碼為admin
。
問題
問題1:IPv4 forwarding is disabled. Networking will not work
執行docker run時有個提醒
WARNING: IPv4 forwarding is disabled. Networking will not work.
這個警告提示 Docker 在你的主機上禁用了 IPv4 轉發,這可能導致 Docker 容器無法正常工作。IPv4 轉發是指將進入的網絡流量從一個網絡接口轉發到另一個網絡接口的功能。
為了解決這個問題,你可以啟用 IPv4 轉發。
解決方案:
-
打開 IPv4 轉發:
sudo sysctl -w net.ipv4.ip_forward=1
-
若要使該更改永久生效,編輯
/etc/sysctl.conf
文件,將net.ipv4.ip_forward
的值設置為1
。
問題2:OpenSearch Dashboards 啟用中文界面
在 opensearch_dashboards.yml中修改配置
#設置中文
i18n.locale: "zh-CN"