Elaticsearch學習

Elaticsearch

索引

1、索引創建

PUT /index_v1
{"settings": {"number_of_shards": 3,"number_of_replicas": 1},"mappings": {"properties": {"aaa": {"type": "keyword","store": true},   "hhh": {"type": "keyword","store": true}}}
}

2、索引別名

person_info_v1為索引名稱，person_info為索引要創建的別名

put /person_info_v1/_alias/person_info

查詢語法

1、minimum_should_match

bool查詢也可以用 minimum_should_match，如果配置成數字 3，則表示查詢關鍵詞被分詞器分成 3 個及其以下的term 時， they are all required（條件都需要滿足才能符合查詢要求）

對于被analyzer分解出來的每一個term都會構造成一個should的bool query的查詢,每個term變成一個term query子句。例如"query": “how not to be”，被解析成： { “bool”: { “should”: [ { “term”: { “body”: “how”}}, { “term”: { “body”: “not”}}, { “term”: { “body”: “to”}}, { “term”: { “body”: “be”}} ],

2、查詢分詞效果

anlyzer后面是分詞器，有ik_smart，ik_max_word等，text后面是想要查看分詞效果的詞

POST _analyze
{"analyzer":"ik_max_word","text":"李四"}

3、must和should混合使用

must是數據庫中AND的意思，should是數據庫中OR的意思，使用的時候不能簡單的QueryBuilders.boolQuery.must().should()，要向下面這樣使用

QueryBuilders.boolQuery().must(QueryBuilders.termQuery("is_deleted", DELETE_FLAG)).must(QueryBuilders.boolQuery().should(QueryBuilders.matchQuery("person_name", keywordVal).operator(Operator.AND).analyzer("ik_max_word") ));

Operato.AND表示查詢分詞要和es中的索引都匹配上才行，比如索引中內容是張三三，分詞效果是張和三三，查詢內容是張三，分詞是張和三，那這個時候就查詢不到結果，查詢內容改成張三三，分詞效果是張和三三，就和索引中的分詞都匹配上了，可以查詢出內容。這樣做的原因是防止你輸入張三的時候把李三也查出來。如果不顯示的聲明Operator.AND，那會默認使用Operator.OR,這樣的話輸入張三，就會把李三也查出來，因為張三分詞是張和三，只要三匹配了，就會查出來

4、查詢索引中數據大小

GET /my-index-000001/_stats

5、字段匹配度排序

比如有個person_name字段，正常查詢的時候按照_score排序，查詢張建的時候，張建建的分值比張建的分值大，導致排序的時候張建建排在張建之前，但是按照常理來說，張建應該排在張建建之前，這就涉及到es的分詞器以及分值計算問題了

解決方法是在person_name字段中設置一個子字段，不分詞

"person_name": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart","store": true,"index_options": "docs","fields": {"raw": { "type": "keyword", "store": true }}}

查詢的時候，使用match_parse精確查詢子字段并用boost設置較大的權重，使用match模糊查詢person_name字段

查詢語句

1、短語匹配

{"query": {"bool": {"should": [{"match_phrase": {"person_name.raw": {"query": "張建建","boost": 10}}},{"match": {"person_name": {"query": "張建建"}}}]}}
}

java代碼

BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
queryBuilder.should(QueryBuilders.matchPhraseQuery("person_name.raw",keywordVal).boost(4));
queryBuilder.should(QueryBuilders.matchQuery("person_name", keywordVal).operator(Operator.AND).analyzer("ik_max_word"));

2、查詢所有

/_search
{"query": {"match_all": {}}
}

3、查詢數量

/_count
{"query": {"match_all": {}}
}

4、排序

{"query": {"match": {"ent_name": "杭州乾元"}},"sort": [{"est_date": {"order": "asc"}}]
}

5、nested查詢

 {"query": {"bool": {"filter": [{"nested": {"query": {"bool": {"filter": [{"term": {"clues.clue_id": {"value": "xxx","boost": 1}}}],"boost": 1}},"path": "clues","score_mode": "none","boost": 1}}],"boost": 1}}
}

6、字段+nested

{"query": {"bool": {"filter": [{"terms": {"_id": ["xxx"],"boost": 1}},{"nested": {"query": {"bool": {"filter": [{"terms": {"clues.clue_code": ["xxx"],"boost": 1}}],"adjust_pure_negative": true,"boost": 1}},"path": "clues","ignore_unmapped": false,"score_mode": "none","boost": 1}}],"adjust_pure_negative": true,"boost": 1}}
}

7、nested字段為空條件查詢

{"query": {"bool": {"must_not": [{"nested": {"path": "tags","query": {"exists": {"field": "tags"}}}}]}}
}

8、案件數據為空，但是線索不為空的數據

{"query": {"bool": {"filter": [{"bool": {"should": [{"bool": {"must_not": [{"exists": {"field": "case_type"}}],"adjust_pure_negative": true,"boost": 1}}],"adjust_pure_negative": true,"boost": 1}},{"range": {"clue_num": {"from": "0","to": null,"include_lower": false,"include_upper": true,"boost": 1}}}]}}
}

刪除

刪除索引中的全部數據

POST /my_index/_delete_by_query
{"query": {"match_all": {}}
}

命令行刪除：

curl -u elastic:'xxxx' -XPOST 'ip:port/medical_institution/_delete_by_query?refresh&slices=5&pretty' -H 'Content-Type: application/json' -d'{  "query": {    "match_all": {}  }}'

插入

POST /person_info_test_v1/_doc/
{"person_name": "張建芬"
}

更新

1、數據更新

（1）nested更新

POST  http://ip:port/case_info/_update_by_query
{"script": {"source": "ctx._source.clues[0].clue_state = 2","lang": "painless"},"query": {"bool": {"filter": [{"nested": {"query": {"bool": {"filter": [{"term": {"clues.clue_id": {"value": "xxx","boost": 1}}}],"boost": 1}},"path": "clues","score_mode": "none","boost": 1}}],"boost": 1}}
}

（2）nested字段置空

{"script": {"source": "ctx._source.clues = []","lang": "painless"},"query": {"term": {"_id": "xxx"}}
}

（3）多條件更新

POST  http://ip:port/case_info/_update_by_query
{"script": {"source": "ctx._source.obj_code = 'xxx'","lang": "painless"},"query": {"bool": {"filter": [{"term": {"case_type": "check_action"}},{"term": {"obj_code": "xxx"}}]}}
}

（4）數組(nested)字段更新

#更新為空的字段
{"script": {"source": "def tags= ctx._source.tags;def newTag=params.tagInfo; if (tags == null) {  ctx._source.tags = params.tagInfo;}","lang": "painless","params": {"tagInfo": [{"tag_code": "case_xzcf_basic_0001","tag_value": "簡易程序"},{"tag_code": "case_xzcf_basic_0002","tag_value": "立案階段"},{"tag_code": "case_xzcf_basic_0003","tag_value": "無文書"}]}},"query": {"term": {"_id": "0e978d6afb74b52a322d7aa8fbfbddf8"}}
}
#將不為空的字段置為空
{"script": {"source": "def tags= ctx._source.tags;def newTag=params.tagInfo;  ctx._source.tags = params.tagInfo;","lang": "painless","params": {"tagInfo": []}},"query": {"bool": {"must": [{"nested": {"path": "tags","query": {"exists": {"field": "tags"}}}}]}}
}

2、更新配置參數

PUT http://ip:port/case_info/_settings
{"refresh_interval": "1s"
}

訪問

1、在linux中加密訪問

#elastic是用戶名，xxx是密碼
curl ip:port -u elastic:'xxx'

2、ES健康狀態查看

curl http://localhost:9200/_cat/health?v -u elastic:'xxx'

ES問題處理

一、數據插入失敗

1、提示只讀

] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"index [person_info_v1] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

解決方法：首先查看磁盤空間是否被占滿了，如果磁盤空間夠用，則執行以下語句，將索引只讀狀態置為false

/indexname/_settings    PUT
{"index": {"blocks": {"read_only_allow_delete": "false"}}
}{"index": {"refresh_interval": "1s"}
}

2、cpu占用過高

在網頁上輸入以下地址

http://ip:port/_nodes/hotthreads

問題處理

一、數據插入失敗

1、提示只讀

] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"index [person_info_v1] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

解決方法：首先查看磁盤空間是否被占滿了，如果磁盤空間夠用，則執行以下語句，將索引只讀狀態置為false

/indexname/_settings    PUT
{"index": {"blocks": {"read_only_allow_delete": "false"}}
}{"index": {"refresh_interval": "1s"}
}

2、cpu占用過高

在網頁上輸入以下地址

http://ip:port/_nodes/hotthreads

查詢出的內容搜索cpu usage by thread即可