進入docker中的logstash 容器內:
jelex@bogon ~ % docker exec -it 7ee8960c99a31e607f346b2802419b8b819cc860863bc283cb7483bc03ba1420 /bin/sh
$ pwd
/usr/share/logstash
$ ls
bin CONTRIBUTORS Gemfile jdk logstash-core modules tools x-pack
config data Gemfile.lock lib logstash-core-plugin-api pipeline vendor
$ cd bin
$ ls
benchmark.bat ingest-convert.bat logstash-keystore logstash-plugin.bat pqrepair.bat
benchmark.sh ingest-convert.sh logstash-keystore.bat pqcheck ruby
cpdump logstash logstash.lib.sh pqcheck.bat setup.bat
dependencies-report logstash.bat logstash-plugin pqrepair system-install
安裝插件:
$ logstash-plugin install logstash-codec-json_lines
logstash優化:官方的建議是提高每次批處理的數量,調節傳輸間歇時間。當batch.size增大,es處理的事件數就會變少,寫入也就越快了
vim /etc/logstash/pipeline.yml
pipeline線程數,官方建議是等于CPU內核數
pipeline.workers: 24
實際output時的線程數
pipeline.output.workers: 24
每次發送的事件數
pipeline.batch.size: 10000
發送延時(ms)
pipeline.batch.delay: 10