Elasticsearch-DSL高級查詢操作

一、禁用元數據和過濾數據

1、禁用元數據_source

GET product/_search
{"_source": false, "query": {"match_all": {}}
}

查詢結果不顯示元數據
禁用之前:

{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 5,"relation" : "eq"},"max_score" : 1.0,"hits" : [{"_index" : "product","_type" : "_doc","_id" : "1","_score" : 1.0,"_source" : {"name" : "xiaomi phone","desc" : "shouji zhong de zhandouji","date" : "2021-06-01","price" : 3999,"tags" : ["xingjiabi","fashao","buka"]}},{"_index" : "product","_type" : "_doc","_id" : "2","_score" : 1.0,"_source" : {"name" : "xiaomi nfc phone","desc" : "zhichi quangongneng nfc,shouji zhong de jianjiji","date" : "2021-06-02","price" : 4999,"tags" : ["xingjiabi","fashao","gongjiaoka"]}},{"_index" : "product","_type" : "_doc","_id" : "3","_score" : 1.0,"_source" : {"name" : "nfc phone","desc" : "shouji zhong de hongzhaji","date" : "2021-06-03","price" : 2999,"tags" : ["xingjiabi","fashao","menjinka"]}},{"_index" : "product","_type" : "_doc","_id" : "4","_score" : 1.0,"_source" : {"name" : "xiaomi erji","desc" : "erji zhong de huangmenji","date" : "2021-04-15","price" : 999,"tags" : ["low","bufangshui","yinzhicha"]}},{"_index" : "product","_type" : "_doc","_id" : "5","_score" : 1.0,"_source" : {"name" : "hongmi erji","desc" : "erji zhong de kendeji 2021-06-01","date" : "2021-04-16","price" : 399,"tags" : ["lowbee","xuhangduan","zhiliangx"]}}]}
}

禁用之后:

{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 5,"relation" : "eq"},"max_score" : 1.0,"hits" : [{"_index" : "product","_type" : "_doc","_id" : "1","_score" : 1.0},{"_index" : "product","_type" : "_doc","_id" : "2","_score" : 1.0},{"_index" : "product","_type" : "_doc","_id" : "3","_score" : 1.0},{"_index" : "product","_type" : "_doc","_id" : "4","_score" : 1.0},{"_index" : "product","_type" : "_doc","_id" : "5","_score" : 1.0}]}
}

2、數據源過濾器

Including：結果中返回哪些field
Excluding：結果中不要返回哪些field，不返回的field不代表不能通過該字段進行檢索，因為元數據不存在不代表索引不存在

兩種實現方式，
1：在創建索引的時候，mapping中配置；
這樣配置映射，在查詢的時候只顯示name和price，不顯示desc和tags

PUT product2
{"mappings": {"_source": {"includes": ["name","price"],"excludes": ["desc","tags"]}}
}

查看映射信息:GET product2/_mapping

{"product2" : {"mappings" : {"_source" : {"includes" : ["name","price"],"excludes" : ["desc","tags"]},"properties" : {"desc" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"name" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"owner" : {"properties" : {"age" : {"type" : "long"},"name" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"sex" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}}}},"price" : {"type" : "long"},"tags" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}}}}}
}

插入數據:

PUT /product2/_doc/1
{"owner":{"name":"zhangsan","sex":"男","age":18},"name": "hongmi erji","desc": "erji zhong de kendeji","price": 399,"tags": ["lowbee","xuhangduan","zhiliangx"]
}

查詢數據:
GET product2/_search
可以看到查詢的結果沒有上面excludes的數據

{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1,"relation" : "eq"},"max_score" : 1.0,"hits" : [{"_index" : "product2","_type" : "_doc","_id" : "1","_score" : 1.0,"_source" : {"price" : 399,"name" : "hongmi erji"}}]}
}

2：在寫get search查詢的時候指定；
基于上面的測試數據，先DELETE product2刪除索引再重新PUT /product2/_doc/1創建索引直接自動映射。
兩種寫法:
1.“_source”: 直接寫展示的字段,
只展示owner.name和owner.sex

GET product2/_search
{"_source": ["owner.name","owner.sex"], "query": {"match_all": {}}
}

結果:

{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1,"relation" : "eq"},"max_score" : 1.0,"hits" : [{"_index" : "product2","_type" : "_doc","_id" : "1","_score" : 1.0,"_source" : {"owner" : {"sex" : "男","name" : "zhangsan"}}}]}
}

2.source里用includes和excludes

GET product2/_search
{"_source": {"includes": ["owner.*","name"],"excludes": ["name", "desc","price"]},"query": {"match_all": {}}
}

結果:

{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1,"relation" : "eq"},"max_score" : 1.0,"hits" : [{"_index" : "product2","_type" : "_doc","_id" : "1","_score" : 1.0,"_source" : {"owner" : {"sex" : "男","name" : "zhangsan","age" : 18}}}]}
}

二、query string search

1.查看索引的結構
GET product/_mapping

2.查詢索引的數據默認10條
GET product/_search

3.查詢索引的數據限制條數20條
GET /product/_search?size=20

4.查詢name分詞后含有nfc的數據
GET /product/_search?q=name:nfc

5.查詢前20條數據并且按照價格降序排列
GET /product/_search?from=0&size=20&sort=price:desc

6.createtime的數據類型是date，不會索引，所以這里是精準匹配createtime:2020-08-19的數據
GET /product/_search?q=createtime:2020-08-19

7.查詢所有text分詞后的詞條中包含炮這個單詞的
GET /product/_search?q=炮

三、全文檢索-Fulltext query

查詢模板:

GET index/_search
{"query": {"match": {"field": "searchContent"}}
}

造測試數據:
put mapping 就像關系型數據庫的表結構：ddl語句

PUT product
{"mappings" : {"properties" : {"createtime" : {"type" : "date"},"date" : {"type" : "date"},"desc" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}},"analyzer":"ik_max_word"},"lv" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"name" : {"type" : "text","analyzer":"ik_max_word","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"price" : {"type" : "long"},"tags" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"type" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}}}}
}

插入數據:就像關系型數據庫的insert

PUT /product/_doc/1
{"name" : "小米手機","desc" :  "手機中的戰斗機","price" :  3999,"lv":"旗艦機","type":"手機","createtime":"2020-10-01T08:00:00Z","tags": [ "性價比", "發燒", "不卡頓" ]
}
PUT /product/_doc/2
{"name" : "小米NFC手機","desc" :  "支持全功能NFC，手機中的滑翔機","price" :  4999,"lv":"旗艦機","type":"手機","createtime":"2020-05-21T08:00:00Z","tags": [ "性價比", "發燒", "公交卡" ]
}
PUT /product/_doc/3
{"name" : "NFC手機","desc" :  "手機中的轟炸機","price" :  2999,"lv":"高端機","type":"手機","createtime":"2020-06-20","tags": [ "性價比", "快充", "門禁卡" ]
}
PUT /product/_doc/4
{"name" : "小米耳機","desc" :  "耳機中的黃燜雞","price" :  999,"lv":"百元機","type":"耳機","createtime":"2020-06-23","tags": [ "降噪", "防水", "藍牙" ]
}
PUT /product/_doc/5
{"name" : "紅米耳機","desc" :  "耳機中的肯德基","price" :  399,"type":"耳機","lv":"百元機","createtime":"2020-07-20","tags": [ "防火", "低音炮", "聽聲辨位" ]
}
PUT /product/_doc/6
{"name" : "小米手機10","desc" :  "充電賊快掉電更快，超級無敵望遠鏡，高刷電競屏","price" :  "","lv":"旗艦機","type":"手機","createtime":"2020-07-27","tags": [ "120HZ刷新率", "120W快充", "120倍變焦" ]
}
PUT /product/_doc/7
{"name" : "挨炮 SE2","desc" :  "除了CPU，一無是處","price" :  "3299","lv":"旗艦機","type":"手機","createtime":"2020-07-21","tags": [ "割韭菜", "割韭菜", "割新韭菜" ]
}
PUT /product/_doc/8
{"name" : "XS Max","desc" :  "聽說要出新款12手機了，終于可以換掉手中的4S了","price" :  4399,"lv":"旗艦機","type":"手機","createtime":"2020-08-19","tags": [ "5V1A", "4G全網通", "大" ]
}
PUT /product/_doc/9
{"name" : "小米電視","desc" :  "70寸性價比只選，不要一萬八，要不要八千八，只要兩千九百九十八","price" :  2998,"lv":"高端機","type":"耳機","createtime":"2020-08-16","tags": [ "巨饃", "家庭影院", "游戲" ]
}
PUT /product/_doc/10
{"name" : "紅米電視","desc" :  "我比上邊那個更劃算，我也2998，我也70寸，但是我更好看","price" :  2999,"type":"電視","lv":"高端機","createtime":"2020-08-28","tags": [ "大片", "藍光8K", "超薄" ]
}
PUT /product/_doc/11
{"name": "紅米電視","desc": "我比上邊那個更劃算，我也2998，我也70寸，但是我更好看","price": 2998,"type": "電視","lv": "高端機","createtime": "2020-08-28","tags": ["大片","藍光8K","超薄"]
}

在這里插入圖片描述

在構造mapping映射的時候，對text類型的字段指定了"analyzer":"ik_max_word"分詞器，這里用的是IK分詞器，插入數據會對該字段進行分詞，建立倒排索引。*“type” : “keyword”*是用來后續精準查詢的時候通過field.keyword來精準匹配。

1、query->match->text類型字段
進行全文搜索，會對查詢的文本進行分詞。
query match 這個name會被分詞 name是txt類型會被分詞所以搜索條件被分詞后會和這個查詢字段的詞項進行匹配匹配到的都返回
查詢條件和索引中的字段數據都會進行分詞后進行匹配按照score返回

GET product/_search?_source=false
{"query": {"match": {"name": "NFC手機"}}
}

query->match->text.keyword類型字段
name是text類型字段，name.keyword做為查詢條件不會進行分詞，直接和索引數據中的name進行匹配，id為3的數據可以查詢匹配。

GET product/_search
{"query": {"match": {"name.keyword": "NFC手機"}}
}

2、query->match_all查詢全部數據
默認查詢返回10條，這里指定20條，禁用元數據不返回太多

GET product/_search?size=20&_source=false
{"query": {"match_all": {}}
}

3、query->multi_match 多個字段匹配
多個字段匹配 name或者desc 包含 query中的任意一個就行，name或者desc分詞后的數據包含手機就返回

GET product/_search?size=20&_source=false
{"query": {"multi_match": {"query": "手機","fields": ["name","desc"]}}
}

4、query->match_phrase 短語查詢
搜索與指定短語匹配的文檔，保留短語中詞語的相對位置。
name的分詞器是ik_max_word,看下name會被分為哪些詞

GET _analyze
{"analyzer": "ik_max_word","text": "小米NFC手機"
}
結果:
{"tokens" : [{"token" : "小米","start_offset" : 0,"end_offset" : 2,"type" : "CN_WORD","position" : 0},{"token" : "nfc","start_offset" : 2,"end_offset" : 5,"type" : "ENGLISH","position" : 1},{"token" : "手機","start_offset" : 5,"end_offset" : 7,"type" : "CN_WORD","position" : 2}]
}GET _analyze
{"analyzer": "ik_max_word","text": "NFC手機"
}結果:
{"tokens" : [{"token" : "nfc","start_offset" : 0,"end_offset" : 3,"type" : "ENGLISH","position" : 0},{"token" : "手機","start_offset" : 3,"end_offset" : 5,"type" : "CN_WORD","position" : 1}]
}

短語查詢索引里面name字段要有NFC手機這個短語順序不能顛倒，NFC手機會被分為nfc 手機
分詞后能和索引字段name分詞后的數據匹配到且順序不亂就可以做為結果展示

GET product/_search
{"query": {"match_phrase": {"name": "NFC手機"}}
}

結果:

{"took" : 5,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 2,"relation" : "eq"},"max_score" : 2.8616219,"hits" : [{"_index" : "product","_type" : "_doc","_id" : "3","_score" : 2.8616219},{"_index" : "product","_type" : "_doc","_id" : "2","_score" : 2.4492486}]}
}

5、Term 對字段進行精確匹配。

GET /my_index/_search
{"query": {  // "query"定義查詢條件"term": { // "term"查詢執行精確匹配"field_name": "exact_value" // "field_name"是要匹配的字段; "exact_value"是精確查詢的精確值，通常用于keyword標簽或其他不分析的文本字段}}
}

6、Bool 多條件組合查詢
組合多個查詢條件，支持must（必須）、should（至少一個）和must_not（必須不）關鍵字。
match支持全文檢索，對查詢條件分詞然后匹配索引中的分詞后的詞項
term精準查詢，不會分詞檢索，和非text類型或者text.keyword使用
range gte大于等于lte小于等于
minimum_should_match should默認至少滿足一個，這里表示至少滿足的數量自己控制

GET product/_search
{"query": {"bool": {"must": [{"match": {"name": "手機"}},{"match": {"desc": "手機"}}],"should": [{"term": {"type.keyword": {"value": "手機"}}},{"range": {"price": {"gte": 100,"lte": 300}}}],"minimum_should_match": 2,"must_not": [{"range": {"price": {"gte": 2999,"lte": 4500}}}]}}
}

filter：條件過濾查詢，過濾滿足條件的數據不計算相關度得分

GET product/_search
{"query": {"bool": {"filter": [{"term": {"type.keyword": {"value": "手機"}}}]}}
}

7、terms
索引中tags含有性價比或者大片任意一個就行

GET product/_search
{"query": {"terms": {"tags.keyword": [ "性價比", "大片" ],"boost": 2.0}}
}

8、constant_score 意為固定得分
避免算分提高性能

GET product/_search
{"query": {"constant_score": {"filter": {"term": {"type.keyword": "手機"}},"boost": 1.2}}
}

9、(must或者filter)和should組合這時should滿足0也行如果should單用要至少滿足一個

GET product/_search
{"query": {"bool": {"filter": [{"range": {"price": {"gte": 10,"lte": 4000}}}],"should": [{"match": {"name": "哈哈哈哈哈哈哈哈哈哈哈哈"}},{"range": {"price": {"gte": 4001,"lte": 9000}}}],"minimum_should_match": 1}}
}