Often, customers ask me to explain or describe the ELK stack. I figured that it would be easier to describe, at a high-level, the ELK stack in simple terms. As a bundle, "ELK" stands for Elasticsearch, Logstash, Kibana. However, the term "ELK Stack" is nowadays called the Elastic Stack. "ELK" is the acronym for three open source projects: Elasticsearch, Logstash, and Kibana.
ELK Stack is often used in-house by organizations because it's simple to install and run. The stack does takes time and skill to maintain and scale it when the volume of data/queries goes increases. The now branded “Elastic Stack” is the next evolution of ELK.
The ELK Stack is a log management platform; a collection of three open-source products — Elasticsearch, Logstash, and Kibana — from Elastic: https://www.elastic.co/elk-stack.
- Elastic is a search and analytics engine. Elasticsearch is a NoSQL database that is based on the Lucene search engine.
- Logstash is a server-side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then sends it to a "stash" like Elasticsearch. Logstash is a log pipeline tool that accepts inputs from various sources, executes different transformations, and exports the data to various targets.
- Kibana is a visualization layer that works on top of Elasticsearch. Kibana lets users visualize data with charts and graphs in Elasticsearch.
Together, these three different open source products are most commonly used in log analysis in IT environments (though there are many more use cases for this stack including business intelligence, security and compliance, and web analytics.
Logstash collects and parses logs
- Elasticsearch indexes and stores the information
- Kibana then presents the data in visualizations that provide actionable insights into one’s environment
Logstash collects and pushes logs to the Elasticsearch database. Kibana then takes the data and visualizes it in graphical form.