【elasticsearch】慢查詢替代查詢審計的嘗試

使用了es有兩年了，突然發現一個，es沒有查詢審計日志，某個用戶查詢了某個索引的審計。
找了官方文檔和社區的回復都是說使用slow log替代慢查詢。
嘗試一下。

參考鏈接1：https://discuss.elastic.co/t/does-elasticsearch-capture-audit-logs-for-query-dsl-eql-and-sql-or-not/339398/8
參考鏈接2：https://www.elastic.co/guide/en/elasticsearch/reference/7.17/auditing-search-queries.html

前置條件

elasticsearch： 7.17.13
操作系統：linux7.9
es加密配置，只有加密后才可以使用slow log

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate 
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12 
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12

普通查詢

 curl -u elastic:AAA320 localhost:9200/sq-20240506/_search

普通插入

curl -u elastic:AAA320 -H 'Content-Type: application/json' -X POST localhost:9200/sq-20240506/_doc/1 -d '
{
"firstname": "Krishna",
"lastname": "kumar"
}'

上述查詢和插入沒有任何slow log信息。

設置慢查詢

這里設置trace的慢查詢為0ms，會捕獲所有查詢。

curl -u elastic:AAA320 -H 'Content-Type: application/json'  -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{"index.search.slowlog.threshold.fetch.debug" : "500ms","index.search.slowlog.threshold.fetch.info" : "800ms","index.search.slowlog.threshold.fetch.trace" : "0ms","index.search.slowlog.threshold.fetch.warn" : "1s","index.search.slowlog.threshold.query.debug" : "2s","index.search.slowlog.threshold.query.info" : "5s","index.search.slowlog.threshold.query.trace" : "0ms","index.search.slowlog.threshold.query.warn" : "10s"
}'

查看日志

慢查詢日志，存放再es的日志文件夾中，elasticsearch_index_search_slowlog.log,這里可以看到TRACE級別的日志，可以看到分query和fetch兩種，有節點信息、索引信息、分片以及耗時，但是我還是覺得這里缺少了查詢用戶，查詢來源信息，目前還是無法滿足我的實際排查需求。

] tailf elasticsearch_index_search_slowlog.log
[2024-05-06T16:33:29,528][TRACE][i.s.s.query              ] [node-1] [sq-20240506][0] took[179.4micros], took_millis[0], total_hits[1 hits], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[1], source[{}], id[], 
[2024-05-06T16:33:29,529][TRACE][i.s.s.fetch              ] [node-1] [sq-20240506][0] took[3.3ms], took_millis[3], total_hits[1 hits], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[1], source[{}], id[],

取消慢查詢

curl -u elastic:AAA320 -H 'Content-Type: application/json' -XPUT 'http://localhost:9200/_all/_settings' -d '{"index.search.slowlog.threshold.fetch.debug": null,"index.search.slowlog.threshold.fetch.info": null,"index.search.slowlog.threshold.fetch.trace": null,"index.search.slowlog.threshold.fetch.warn": null,"index.search.slowlog.threshold.query.debug": null,"index.search.slowlog.threshold.query.info": null,"index.search.slowlog.threshold.query.trace": null,"index.search.slowlog.threshold.query.warn": null
}'