Difference between revisions of "Logstash"

From wikieduonline
Jump to navigation Jump to search
 
(19 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Logstash is a light-weight, open-source, server-side data processing pipeline that allows you to collect data from a variety of sources, transform it on the fly, and send it to your desired destination. It is most often used as a data pipeline for Elasticsearch, an open-source analytics and search engine. Logstash integrates with Elasticsearch and has over 200 pre-built open-source plugins that can help to index your data.
+
[[wikipedia:Logstash]] is a light-weight, open-source, server-side data processing pipeline that allows you to collect data from a variety of sources, transform it on the fly, and send it to your desired destination. It is most often used as a data pipeline for Elasticsearch, an open-source analytics and search engine. Logstash integrates with Elasticsearch and has over 200 pre-built open-source plugins that can help to index your data.
  
 
Plugins: https://www.elastic.co/guide/en/logstash/current/output-plugins.html
 
Plugins: https://www.elastic.co/guide/en/logstash/current/output-plugins.html
* [[Kafka]]  
+
* [[Kafka]], [[XMMP]], [[Zabbix]]
* [[XMMP]]
 
* [[Zabbix]]
 
  
* MacOS installation:  
+
== Installation ==
** <code>[[brew install]] logstash</code>
+
* https://www.elastic.co/guide/en/logstash/current/installing-logstash.html
** <code>[[brew cask install]] homebrew/cask-versions/adoptopenjdk8</code>
+
Linux:
 +
{{ELK repo}}
 +
[[sudo apt-get update]] && sudo [[apt-get install logstash]]
  
[[logstash (command)]]
+
[[macOS]] installation:
 +
* <code>[[brew install logstash]]</code>
 +
* <code>[[brew cask install]] homebrew/cask-versions/adoptopenjdk8</code>
 +
 
 +
Usage:
 +
* <code>[[logstash (command)]]</code>
  
 
== Docker Logstash ==
 
== Docker Logstash ==
Line 19: Line 24:
  
 
* Docker Logstash configurations:  
 
* Docker Logstash configurations:  
** Config: <code>/usr/share/logstash/config/[[logstash.yml]]</code>
+
** Config: <code>[[/usr/share/logstash/config/logstash.yml]]</code>
 
** Pipeline configurations: <code>/usr/share/logstash/pipeline/</code>
 
** Pipeline configurations: <code>/usr/share/logstash/pipeline/</code>
  
Line 25: Line 30:
 
* Review homepage: https://www.elastic.co/logstash
 
* Review homepage: https://www.elastic.co/logstash
 
* Review [[Logstash logs]]
 
* Review [[Logstash logs]]
 
+
* docker run --log-driver=syslog --log-opt syslog-address=tcp://<logstash-system-ip>:5000 hello-world
  
 
== Related terms ==
 
== Related terms ==
Line 31: Line 36:
 
* [[Lumberjack protocol]]
 
* [[Lumberjack protocol]]
 
* [[Filebeat]]
 
* [[Filebeat]]
 +
* [[Vector]]
 +
* [[Fluent Bit]] use [[Logstash]] format to ingest the logs
  
 
== See also ==
 
== See also ==

Latest revision as of 18:37, 15 December 2023

wikipedia:Logstash is a light-weight, open-source, server-side data processing pipeline that allows you to collect data from a variety of sources, transform it on the fly, and send it to your desired destination. It is most often used as a data pipeline for Elasticsearch, an open-source analytics and search engine. Logstash integrates with Elasticsearch and has over 200 pre-built open-source plugins that can help to index your data.

Plugins: https://www.elastic.co/guide/en/logstash/current/output-plugins.html

Installation[edit]

Linux:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo apt-get install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list
sudo apt-get update && sudo apt-get install logstash

macOS installation:

Usage:

Docker Logstash[edit]

Official Logstash docker image is around 800 MB size.

docker pull docker.elastic.co/logstash/logstash:7.8.0
docker run --rm -it -v ~/pipeline/:/usr/share/logstash/pipeline/ docker.elastic.co/logstash/logstash:7.8.0

Activities[edit]

Related terms[edit]

See also[edit]

Advertising: