1 普通聚合分析
1.1 直接聚合統計
(1) 計算每個tag下的文檔數量, 請求語法:
GET book_shop/it_book/_search
{
"size": 0, // 不顯示命中(hits)的所有文檔信息
"aggs": {
"group_by_tags": {// 聚合結果的名稱, 需要自定義(復制時請去掉此注釋)
"terms": {
"field": "tags"
}
}
}
}
(2) 發生錯誤:
說明: 索引book_shop的mapping映射是ES自動創建的, 它把tag解析成了text類型, 在發起對tag的聚合請求后, 將拋出如下錯誤:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [tags] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [......]
},
"status": 400
}
(3) 錯誤分析:
錯誤信息: Set fielddata=true on [xxxx] ......
錯誤分析: 默認情況下, Elasticsearch 對 text 類型的字段(field)禁用了 fielddata;
text 類型的字段在創建索引時會進行分詞處理, 而聚合操作必須基于字段的原始值進行分析;
所以如果要對 text 類型的字段進行聚合操作, 就需要存儲其原始值 —— 創建mapping時指定fielddata=true, 以便通過反轉倒排索引(即正排索引)將索引數據加載至內存中.
(4) 解決方案一: 對text類型的字段開啟fielddata屬性:
將要分組統計的text field(即tags)的fielddata設置為true:
PUT book_shop/_mapping/it_book
{
"properties": {
"tags": {
"type": "text",
"fielddata": true
}
}
}
再次統計, 得到的結果如下:
{
"took": 153,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 4,
"max_score": 0.0,
"hits": []
},
"aggregations": {
"group_by_tags": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 6,
"buckets": [
{
"key": "java",
"doc_count": 3
},
{
"key": "程",
"doc_count": 2
},
......
]
}
}
}
(5) 解決方法二: 使用內置keyword字段:
開啟fielddata將占用大量的內存.
Elasticsearch 5.x 版本開始支持通過text的內置字段keyword作精確查詢、聚合分析:
GET shop/it_book/_search
{
"size": 0,
"aggs": {
"group_by_tags": {
"terms": {
"field": "tags.keyword"// 使用text類型的內置keyword字段
}
}
}
}
1.2 先檢索, 再聚合
(1) 統計name中含有“jvm”的圖書中每個tag的文檔數量, 請求語法:
GET book_shop/it_book/_search
{
"query": {
"match": { "name": "jvm" }
},
"aggs": {
"group_by_tags": { // 聚合結果的名稱, 需要自定義. 下面使用內置的keyword字段:
"terms": { "field": "tags.keyword" }
}
}
}
(2) 響應結果:
{
"took" : 7,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 0.64072424,
"hits" : [
{
"_index" : "book_shop",
"_type" : "it_book",
"_id" : "2",
"_score" : 0.64072424,
"_source" : {
"name" : "深入理解Java虛擬機：JVM高級特性與最佳實踐",
"author" : "周志明",
"category" : "編程語言",
"desc" : "Java圖書領域公認的經典著作",
"price" : 79.0,
"date" : "2013-10-01",
"publisher" : "機械工業出版社",
"tags" : [
"Java",
"虛擬機",
"最佳實踐"
]
}
}
]
},
"aggregations" : {
"group_by_tags" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "Java",
"doc_count" : 1
},
{
"key" : "最佳實踐",
"doc_count" : 1
},
{
"key" : "虛擬機",
"doc_count" : 1
}
]
}
}
}
1.3 擴展: fielddata和keyword的聚合比較
為某個 text 類型的字段開啟fielddata字段后, 聚合分析操作會對這個字段的所有分詞分別進行聚合, 獲得的結果大多數情況下并不符合我們的需求.
使用keyword內置字段, 不會對相關的分詞進行聚合, 結果可能更有用.
—— 推薦使用text類型字段的內置keyword進行聚合操作.
2 嵌套聚合
2.1 先分組, 再聚合統計
(1) 先按tags分組, 再計算每個tag下圖書的平均價格, 請求語法:
GET book_shop/it_book/_search
{
"size": 0,
"aggs": {
"group_by_tags": {
"terms": { "field": "tags.keyword" },
"aggs": {
"avg_price": {
"avg": { "field": "price" }
}
}
}
}
}
(2) 響應結果:
"hits" : {
"total" : 3,
"max_score" : 0.0,
"hits" : [ ]
},
"aggregations" : {
"group_by_tags" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "Java",
"doc_count" : 3,
"avg_price" : {
"value" : 102.33333333333333
}
},
{
"key" : "編程語言",
"doc_count" : 2,
"avg_price" : {
"value" : 114.0
}
},
......
]
}
}
2.2 先分組, 再統計, 最后排序
(1) 計算每個tag下圖書的平均價格, 再按平均價格降序排序, 查詢語法:
GET book_shop/it_book/_search
{
"size": 0,
"aggs": {
"all_tags": {
"terms": {
"field": "tags.keyword",
"order": { "avg_price": "desc" } // 根據下述統計的結果排序
},
"aggs": {
"avg_price": {
"avg": { "field": "price" }
}
}
}
}
}
(2) 響應結果:
與#2.1節內容相似, 區別在于按照價格排序顯示了.
2.3 先分組, 組內再分組, 然后統計、排序
(1) 先按價格區間分組, 組內再按tags分組, 計算每個tags組的平均價格, 查詢語法:
GET book_shop/it_book/_search
{
"size": 0,
"aggs": {
"group_by_price": {
"range": {
"field": "price",
"ranges": [
{ "from": 00, "to": 100 },
{ "from": 100, "to": 150 }
]
},
"aggs": {
"group_by_tags": {
"terms": { "field": "tags.keyword" },
"aggs": {
"avg_price": {
"avg": { "field": "price" }
}
}
}
}
}
}
}
(2) 響應結果:
"hits" : {
"total" : 3,
"max_score" : 0.0,
"hits" : [ ]
},
"aggregations" : {
"group_by_price" : {
"buckets" : [
{
"key" : "0.0-100.0", // 區間0.0-100.0
"from" : 0.0,
"to" : 100.0,
"doc_count" : 1, // 共查找到了3條文檔
"group_by_tags" : { // 對tags分組聚合
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "Java",
"doc_count" : 1,
"avg_price" : {
"value" : 79.0
}
},
......
]
}
},
{
"key" : "100.0-150.0",
"from" : 100.0,
"to" : 150.0,
"doc_count" : 2,
"group_by_tags" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "Java",
"doc_count" : 2,
"avg_price" : {
"value" : 114.0
}
},
......
}
]
}
}
]
}
}
