Difference between revisions of "Logstash"
Jump to navigation
Jump to search
Tags: Mobile web edit, Mobile edit |
|||
| (30 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| − | Logstash is a light-weight, open-source, server-side data processing pipeline that allows you to collect data from a variety of sources, transform it on the fly, and send it to your desired destination. It is most often used as a data pipeline for Elasticsearch, an open-source analytics and search engine. Logstash integrates with Elasticsearch and has over 200 pre-built open-source plugins that can help to index your data. | + | [[wikipedia:Logstash]] is a light-weight, open-source, server-side data processing pipeline that allows you to collect data from a variety of sources, transform it on the fly, and send it to your desired destination. It is most often used as a data pipeline for Elasticsearch, an open-source analytics and search engine. Logstash integrates with Elasticsearch and has over 200 pre-built open-source plugins that can help to index your data. |
| + | Plugins: https://www.elastic.co/guide/en/logstash/current/output-plugins.html | ||
| + | * [[Kafka]], [[XMMP]], [[Zabbix]] | ||
| + | == Installation == | ||
| + | * https://www.elastic.co/guide/en/logstash/current/installing-logstash.html | ||
| + | Linux: | ||
| + | {{ELK repo}} | ||
| + | [[sudo apt-get update]] && sudo [[apt-get install logstash]] | ||
| − | + | [[macOS]] installation: | |
| − | * [[ | + | * <code>[[brew install logstash]]</code> |
| − | * [[ | + | * <code>[[brew cask install]] homebrew/cask-versions/adoptopenjdk8</code> |
| − | * [[ | + | |
| + | Usage: | ||
| + | * <code>[[logstash (command)]]</code> | ||
== Docker Logstash == | == Docker Logstash == | ||
| + | Official Logstash docker image is around 800 MB size. | ||
[[docker pull]] docker.elastic.co/logstash/logstash:7.8.0 | [[docker pull]] docker.elastic.co/logstash/logstash:7.8.0 | ||
| − | + | [[Logstash: docker run|docker run]] --rm -it -v ~/pipeline/:/usr/share/logstash/pipeline/ docker.elastic.co/logstash/logstash:7.8.0 | |
* Docker Logstash configurations: | * Docker Logstash configurations: | ||
| − | ** Config: <code>/usr/share/logstash/config/ | + | ** Config: <code>[[/usr/share/logstash/config/logstash.yml]]</code> |
** Pipeline configurations: <code>/usr/share/logstash/pipeline/</code> | ** Pipeline configurations: <code>/usr/share/logstash/pipeline/</code> | ||
| Line 20: | Line 30: | ||
* Review homepage: https://www.elastic.co/logstash | * Review homepage: https://www.elastic.co/logstash | ||
* Review [[Logstash logs]] | * Review [[Logstash logs]] | ||
| + | * docker run --log-driver=syslog --log-opt syslog-address=tcp://<logstash-system-ip>:5000 hello-world | ||
| + | |||
| + | == Related terms == | ||
| + | * [[MetricBeat]] | ||
| + | * [[Lumberjack protocol]] | ||
| + | * [[Filebeat]] | ||
| + | * [[Vector]] | ||
| + | * [[Fluent Bit]] use [[Logstash]] format to ingest the logs | ||
== See also == | == See also == | ||
| + | * {{Logstash}} | ||
* {{ELK}} | * {{ELK}} | ||
| − | + | * {{logs}} | |
[[Category:Logging]] | [[Category:Logging]] | ||
Latest revision as of 18:37, 15 December 2023
wikipedia:Logstash is a light-weight, open-source, server-side data processing pipeline that allows you to collect data from a variety of sources, transform it on the fly, and send it to your desired destination. It is most often used as a data pipeline for Elasticsearch, an open-source analytics and search engine. Logstash integrates with Elasticsearch and has over 200 pre-built open-source plugins that can help to index your data.
Plugins: https://www.elastic.co/guide/en/logstash/current/output-plugins.html
Installation[edit]
Linux:
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - sudo apt-get install apt-transport-https echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list sudo apt-get update && sudo apt-get install logstash
macOS installation:
brew install logstashbrew cask install homebrew/cask-versions/adoptopenjdk8
Usage:
Docker Logstash[edit]
Official Logstash docker image is around 800 MB size.
docker pull docker.elastic.co/logstash/logstash:7.8.0 docker run --rm -it -v ~/pipeline/:/usr/share/logstash/pipeline/ docker.elastic.co/logstash/logstash:7.8.0
- Docker Logstash configurations:
- Config:
/usr/share/logstash/config/logstash.yml - Pipeline configurations:
/usr/share/logstash/pipeline/
- Config:
Activities[edit]
- Review homepage: https://www.elastic.co/logstash
- Review Logstash logs
- docker run --log-driver=syslog --log-opt syslog-address=tcp://<logstash-system-ip>:5000 hello-world
Related terms[edit]
- MetricBeat
- Lumberjack protocol
- Filebeat
- Vector
- Fluent Bit use Logstash format to ingest the logs
See also[edit]
- Logstash,
logstash (command), logstash.yml, Logstash: docker run, Logstash changelog - Elastic: ELK,
Elasticsearch,Logstash,Kibana, Installation, AWS Elasticsearch, Elastic SIEM, Elastic Beats,metricbeat,filebeat,journalbeat, Elastisearch Service , Search guard, Elasticsearch logs, curator, ILM, Lumberjack protocol,aws_elasticsearch_domain, KQL,elasticsearch.yml, elasticsearch-plugin, elasticsearch-certutil, Elasticsearch release notes/changelog - Logs, Log collector, log management, log explorer, Linux logging,
docker logs,minikube logs, Vector, Logstash, Filebeat, promtail, logfmt, Elasticsearch, fluentd, Mezmo (LogDNA), Scalyr, Loggly, Loki,tlog, cockpit, NXLog, Winston, Amazon CloudWatch Logs Insights, Logz.io, Logflare
Advertising:
