Elastic Observability

Essential tool for all my devops needs

Pros:

I've been using Elasticsearch since early days, with very different things in mind. I started with simple text search - with some additional tweaks, stemming and other cool features it helped us drive enormous traffic to our website. I can't imaging pulling it off so easily with any other tool. Every day I use it for web server log analytics. Search and great visualizations make it an absolute essential in work my toolset. We also run a lot of algorithm analytics on top of our Elasticsearch cluster. If you're looking for managed options check AWS Elasticsearch Service, or the recently introduced Elastic Cloud.

Cons:

My only concern with Elasticsearch is that it might get expensive to run pretty quickly. But with a certain amount of effort put into optimization it's gonna be worth it.

Logstash works great with Elasticsearch and Kibana, making it easy to ingest data and transform it

Comments: It is a really good product and does what it intends to do well, as long as you are not intimidated by the terminal and configuration files.

Pros:

Logstash does a great job of ingesting data, transforming it and then storing in a 'stash' like elasticsearch. It can virtually ingest any data as long as you have the time and patience to set it all up. We use it as part of the process of collecting all logs from a multitude of sources using filebeat (generally application and system logs) on different operating systems, shipping to logstash then transforming and storing that in elasticsearch and having Kibana for dashboards and analysis. It works great for this use case.

Cons:

It can be tricky to set up as it is mostly just a command line tool with configuration files. The recent log4j issues also make you think again about all this log shipping, transformation and ingesting in a different way. If there is similar vulnerabilities in the future, it could end up happening way down in your process in non-public components due to the way logs are shipped and processed down the line.

Review for Elasticsearch

Pros:

Elastic Search is easy to install Easy to use as there are loads of documentations present online Easy to scale up when the need arises Uses REST FULL API which is light weight.

Cons:

monitoring of Elastic Search are complex like wise administration Installation of elastic search on windows OS is not straight forward

Elastic - Very fast moving company and product

Pros:

Elasticsearch has turned into a solution that you'd typically integrate into another to a fully customizable standalone solution. It has everything in one spot now, with hardly a need to use 3rd party plugins/extensions.

Cons:

The licensing models are very confusing. There is a big push to get on their hosted SaaS offering but that itself is also not very clear.

Great for indexing a large amount of data

Comments: We're ElasticSearch mainly to index large amount of logs from several servers. Its makes it very easy for us to index and search logs. Logstash sends the logs and with Kibana we access to logs and create nice dashboards. But, you have to manage your indexes. For our log an index is created every day which we reindex monthly to a new index, then we do a forcemerge and after that we delete the daily indexes. This keeps the number of shards low. If we don't do this we run into problems because if to many shards. But, you can schedule this via curl and with every new version of Kibana/ElasticSearch you can do more and more via the GUI. For us this is the number one tool to index and search gigabytes of logs on a daily base and we're able to keep months of logs and still be able to search through it.

Pros:

Indexing large amount of data Searching data Very scalable Creating a cluster is very easy Ability to send commands via CURL to the API Creating snapshots of your data

Cons:

Managing indexes can be a bit of a pain Sometimes issues with indexes becoming read only