Elastic Search Reviews

We are primarily using it for monitoring. It is used for server monitoring.

I really like the visualization that you can do within it. That's really handy. Product-wise, it is a very good and stable product.

They should improve its documentation. Their official documentation is not very informative. They can also improve their technical support. They don't help you much with the customized stuff.

They also need to add more visuals. Currently, they have line charts, bar charts, and things like that, and they can add more types of visuals. 

They should also improve the alerts. They are not very simple to use and are a bit complex. They could add more options to the alerting system.

I have been using this solution for one year.

Stability-wise, it is very good. Once the data starts coming in, it is very stable. I didn't find any big glitches in it.

We contacted their technical support once. I didn't find them very good. They are there just to provide documentation and stuff. They don't help you much with the customized stuff. They could improve that. I would rate them a two out of five.

It is complex because it is not Windows-based. It is Linux-based, so one must know Linux to deploy it properly. It is not a product that you can install with just multiple clicks. You need to understand it.

It seems good in terms of return on investment. It is a monitoring solution, and it triggers alerts before something happens. For example, it triggers an alert when the space in Windows reaches an 80% limit. I would say it is a good investment. We are able to fix things before they go wrong. If we didn't have Elasticsearch, things would go wrong, and we would be spending more time fixing them later on.

I would advise others to first know Linux because it would most probably be on Linux. If you're good at Linux, you will be good at this as well.

I would rate ELK Elasticsearch an eight out of ten.

We are developing a SIEM application that is similar to QRadar, ArcSight, or Splunk. This application uses Elasticsearch as its search engine because we want to retrieve information fast. We are just using the basic search engine part of Elasticsearch. We have developed lots of things on top of Elasticsearch, such as security, correlation, reporting, etc.

The search speed is most valuable and important.

Its licensing needs to be improved. They don't offer a perpetual license. They want to know how many nodes you will be using, and they ask for an annual subscription. Otherwise, they don't give you permission to use it. Our customers are generally military or police departments or customers without connection to the internet. Therefore, this model is not suitable for us. This subscription-based model is not the best for OEM vendors. 

Another annoying thing about Elasticsearch is its roadmap. We are developing something, and then they say, "Okay. We have removed that feature in this release," and when we are adapting to that release, they say, "Okay. We have removed that one as well." We don't know what they will remove in the next version. They are not looking for backward compatibility from the customers' perspective. They just remove a feature and say, "Okay. We've removed this one."

In terms of new features, it should have an ODBC driver so that you can search and integrate this product with existing BI tools and reporting tools. Currently, you need to go for third parties, such as CData, in order to achieve this. ODBC driver is the most important feature required. 

Its Community Edition does not have security features. For example, you cannot authenticate with a username and password. It should have security features. They might have put it in the latest release.

I have been using this solution since version 1.0.

For a one-node installation, it is easy. You can do it and retrieve information fast, but when you are trying to scale up, everything becomes complicated. If you want to deal with several terabytes of data, you should read whitepapers or case studies or get proper consultancy from Elasticsearch. Otherwise, you will lose data. I know many customers who lost their data and could not recover it. It is not like you store everything and search for everything, and it is just instant. It is not like that. You should do your homework very intensively. It looks easy, but when you scale up, it gets complicated.

We got 60 days of development consultancy with them. Until we sign the agreement, they were quick and prompt. After the signature it changed. Overall experience, we are not satisfied with the development consultancy.

We switched from SQL Server to Elasticsearch. For our application, we wanted the information very fast without locking everything. In SQL Server or Oracle, that would not have been possible. Deleting is also very difficult in SQL Server.

Its initial setup is straightforward. There were no problems.

We are using the Community Edition because Elasticsearch's licensing model is not flexible or suitable for us. They ask for an annual subscription. We also got the development consultancy from Elasticsearch for 60 days or something like that, but they were just trying to do the same trick. That's why we didn't purchase it. We are just using the Community Edition.

We evaluated other products and chose Elasticsearch because the data that we are collecting is unstructured. Every log has a different structure.

The most important thing to keep in mind is that it is not as they advertise on their site. If you want to scale up and are looking for a big deployment, you must read everything. You also need support from the company itself. 

I would rate ELK Elasticsearch a seven out of ten.

I'm involved in architecting and implementing Elasticsearch-based solutions, catering to various use cases including IIoT, cybersecurity, IT Ops, and general logging and monitoring.

The intention of this article is not to compare AWS Elasticsearch with Elastic ELK Elasticsearch and at the end declare the winner. Elasticsearch by itself is one of the coolest and versatile Big Data stacks out there. If you are planning to use it in your organization or trying to evaluate if it is the right stack for your product/ solution, this article offers some insights from an architect's perspective.

I'm not the right person to answer this question as I'm the service provider. My clients are the right people to answer.

The Spaces feature in Kibana is really useful. I can ingest all data and then offer multi-tenancy on a single stack to various departments (internal) or customers (external). This feature isn't available in AWS Elasticsearch, and Machine Learning isn't available either.

Other useful features such as Canvas (used to create live infographics) and Lens (used to explore and create visualisations using a drag-and-drop feature) are available only in Elastic's ELK Elasticsearch.

In the last 18 months Elastic has really caught up and also gone way beyond AWS by putting together all the missing components that make ELK Elasticsearch the most comprehensive stack in the entire Big Data ecosystem. Comprehensive because one stack addresses all of the three essential technical components of an end-to-end system: collect, store and visualise terabytes (and even petabytes) of structured or semi-structured data at ease.

Enhance the Spaces feature to make it fully multi-tenant by enabling role-based access control (RBAC) at a Space level rather than overall Kibana or stack level like it is currently.

Elastic needs to work on their Machine Learning offering because currently they have been trying to make it a black box which doesn't work for a serious user (a Data Scientist) as it doesn't give any control over the underlying algorithm. It's like a point-and-click camera vs a DSLR. The offering started with a single/ univariate anomaly detection on time-series data. Now, they have a multivariate which is good, but beyond this, we cannot build any other Machine Learning models, like traditional supervised models. Anomaly detection uses mostly unsupervised algorithms and also it is a very broad problem space for a black box to solve it fully.

Make index’s metadata searchable (or referenceable in search queries).

5 years

Elastic ELK Elasticsearch is one of the most stable Big Data engines and the simplest to maintain and scale. Redundancy is built into the design so there is no single point of failure. We can configure a DR easily and if something goes wrong, we can restore the system into a brand new cluster in hours.

Elasticsearch by itself is 100% scalable as scalability is built into the design like any Big Data system. We just have to add more nodes, and it scales horizontally and then redistributes the data into the new nodes, and the cluster becomes faster and agile automatically. Cross-cluster replication comes with a Platinum license. But this feature is highly exceptional and not a common need.

I have worked with all the flavours of Elasticsearch viz. Elastic.co's ELK which is popularly known as the ELK stack (pronounced as 'yelk'), AWS Elasticsearch and Open Distro plugins for Elasticsearch.

All (including Solr that comes with Hadoop) are built on a common underlying technology, Apache Lucene. The difference is the added features that I call 'batteries included'. To be precise, Elastic's ELK Elasticsearch, unlike others, comes with free enterprise-grade apps (called plugins in Kibana) and a bunch of cool and useful Kibana features. It also features a good deal of engineering automation conveniences built into the stack.

Did you know that the original founders of Elasticsearch are the folks at Elastic.co, the company that has recently transitioned to an open-core philosophy by design. But since AWS took the initial lead and started offering the stack as AWS Elasticsearch service it became more popular and a preferred option for the uninformed. Elastic, on the other hand, was busy innovating and adding more muscle to the stack that it is no more limited to being just the fastest search engine on the planet. In fact, the keyword ‘search’ in Elasticsearch is not relevant anymore and, moreover, it is misleading.

Initial setup is indeed straightforward and fast because it will mostly be a single-node cluster. But as the data volume grows and we start seeing a performance lag, the stack requires scaling (by adding more nodes) and a professional intervention for doing the right capacity design and configuration fine tuning.

It is always a good idea to engage a professional vendor to implement it right the first time and save yourself a lot of time in experimenting and trying to figure out the optimisation hacks and how-to’s all by yourself.

A stack like Elasticsearch that enables heavy lifting of the data effortlessly comes with its intrinsic yet obvious ROI. If one is not able to realise the ROI it means either the data is bad (garbage in, garbage out) or the stack is not implemented properly.

The basic license is free, and it comes with a lot of features that aren't supposed to be free! With a Gold license, we get Alerting (called Watcher) and some modest enterprise features. Note that if alerting is a must feature for you, you can install open-source alerting plugins like Open Distro Alerting or ElastAlert and avoid the Gold license cost. Active Directory integration, SAML, SSO, Machine Learning etc. come with Platinum license. The licensing is per-node and per-annum basis for an on-premise installation and for Cloud Elastic-managed service the cost is baked into the hourly pay-as-you-go fee. Kibana does not have a license, so it's free.

If you don't want alerting, Active Directory or LDAP integration and are good with native authentication, the basic license will suffice. The basic license also comes with many internal stack features, which are free. For example, data segregation into hot and warm storage, automatic configuration, and rolling over the index after achieving a certain size limit. 

SIEM (Security Information and Event Management) app is free. Also is another cool app called Uptime that helps us monitor the uptime of servers and web services. We can do this without any third-party licensing cost. Just turn on the apps, ingest data using Beats and the apps will start thriving. Over time they become mission critical to your business.

For example, the SIEM app will automatically populate the dashboards and allow us to monitor network traffic, successful logins, unsuccessful login attempts, and anomalous security events. All that comes off the shelf and is free. You'll pay a lot, on the other hand, for a traditional SIEM like ArcSight or LogRhythm.

Another free app called Infrastructure (formerly known as Metrics) helps monitor the server infrastructure by configuring light-weight data collectors called MetricBeats (for Windows systems) and AuditBeats (for Linux systems). The Beats will start pumping in all the system performance metrics into the stack and help monitor the memory, CPU and disk utilization.

I have worked with all the flavours of Elasticsearch viz. Elastic.co's ELK which is popularly known as the ELK stack (pronounced as 'yelk'), AWS Elasticsearch and Open Distro plugins for Elasticsearch.

All (including Solr that comes with Hadoop) are built on a common underlying technology- Apache Lucene. The difference is the added features that I call 'batteries included'. To be precise, Elastic's ELK, unlike the others, comes with free enterprise-grade apps (called plugins in Kibana), a bunch of cool and useful Kibana features, and a good deal of engineering automation built into the stack.

Moreover, the original founders of Elasticsearch are the folks at Elastic.co, the company that's built on open-core philosophy. But AWS took the initial lead and offered the stack as AWS Elasticsearch service catering mostly to search-engine use cases. But ELK, with all its goodness, is much more than a search engine! In fact, the keyword search in Elasticsearch is very misleading.

You can spin up Elastic ELK Elasticsearch fully-managed service either on AWS, GCP, or Azure, or have your own on-premises installation and dockerize it. Whereas the AWS Elasticsearch is available only on AWS. That's the hosting difference.

Elastic ELK Elasticsearch comes with a support-only subscription, and there are a lot of updates happening. Kibana is constantly improved and there’s a new release every two weeks.

Our primary use case of this solution is for monitoring our logs and infrastructure. We are customers of ELK and I'm a system administrator. 

A positive feature of ELK is that it directly interacts with Elasticsearch. The UI is very nice, and performance wise it's quite good too. A key feature is that this is a reasonably priced monitoring solution.

We run this solution on multiple servers. ELK has three lanes which comprise a single package made up of Elasticsearch, Logstash, and Kibana. To my mind, this is not efficient because we have to individually deploy the different applications. In contrast, we're able to deploy Splunk with a singe application. Implementing the dashboards is also quite difficult. With Splunk and Nagios it's much easier to directly interact with Elasticsearch. I'd like to see some additional features in the front end which currently make it a bit difficult to implement and it should be simplified.

I've been using this solution for six months. 

This solution is stable. 

This is a scalable solution, we have eight to 10 users. We had initially planned to expand use of ELK because of its cheap price and the services that are included, but given the difficulty with implementation we've decided to go with Nagios instead. 

The technical support people are very knowledgeable but the response time is quite slow which is not very good. 

The initial setup of ELK is more difficult than the setup of other monitoring applications. I was able to carry out the deployment alone. 

For anyone looking to implement a monitoring product with almost no cost or at a cheaper price, I would suggest the ELK stack. However, it does require a high skill set because of the difficulty with implementation. 

I would rate this solution a six out of 10. 

I run the function to review the usage for the team and for the organization itself.

We use this product internally and then some of our business relationships with the other businesses that we have, they get their data from our data. It's more for collaborative data reporting that we have with them.

The most valuable feature is the out of the box Kibana. You plug it in and start the basic analysis on the data out of the box. This also gives a quick way to check the data and the models to figure out what fits the needs.

There are a few things that did not work for us. 

When doing a search in a bigger setup, with a huge amount of data where there are several things coming in, it has to be on top of the index that we search. 

There could be a way to do a more distributed kind of search. For example, if I have multiple indexes across my applications and if I want to do a correlation between the searches, it is very difficult. From a usage perspective, this is the primary challenge.

I would like to be able to do correlations between multiple indexes. There is a limit on the number of indexes that I can query or do. I can do an all-index search, but it's not theoretically okay on practical terms we cannot do that.

In the next release, I would like to have a correlation between multiple indexes and to be able to save the memory to the disk once we have built the index and it's running.

Once the system is up, it will start building that in memory.

We need to be able to distribute it across or save it to have a faster load time.

We don't make many changes to the data that we are creating, but we would like archived reports and to be able to retrieve those reports to see what is going on. That would be helpful.

Also, if you provide a customer with a report or some archived queries, that the customer is looking at when they are creating, at first it will be slow while putting up their data or subsequently doing it. I want it to be up and running efficiently. 

If the memory could be saved and put back into memory as it is, then starts working it would reduce the load time then it will be more efficient from a cost perspective and it will optimize resource usage.

I have been familiar with this product for approximately four years.

ELK Elasticsearch is stable.

It's scalable, but there are some limitations.

If you are scaling a bit too quickly, you tend to break the applications into different indexes. 

The limitations come in when getting the correlation between the applications or the logs.

It is difficult to get the correlations once the indexes have been split.

We are using the open-source version, that is installed on-premises.

We have not worried about technical support, but the community is good.

Before ELK, we used another solution for internal usage, and also, we used Splunk for different use cases in a different organization altogether.

It wasn't a switch per se, it was a different organization with a different use case.

The initial setup is simple, not too difficult. 

Getting the index, doing your models, and putting the data in, correctly, is done more on a trial and error basis. You have to start early and plan it well to get it right.

We are using the open-source version. 

We are not looking into the subscription because it's on-premises in-house.

For anyone who is looking into implementing this solution, the only tip is to get your models for the type of actual use that you are looking at upfront in order to have a good run.

I would rate ELK Elasticsearch a seven out of ten.

We try to detect malicious files by the logs. The logs are all centralized including all our PCs, our callers, our servers, Linux, windows, Polaris names. We scan everything. Then we have pre-defined specific use cases that allow us to identify if there is an attack on the machine or indirectly by the endpoint. On top of that, we can check with users as we're not directly dealing with the configuration, so we can follow up on the alerts we receive. On top of that, we have the systems in place that allow us to detect if certain inexcusable items are on the system, such as malicious files. We can do this because we also retrieve the log files of the identifiers.

The fact that you can dump any type of format in the database without any specific reformatting is fantastic. It makes it very flexible in collecting information and that saves us a lot of time because otherwise, we would really need to define specifically what we're looking for and reformat everything. With this solution, that's not necessary. We can directly, and in a really standard raw format, dump the data into the database. Only afterwards do we need to define what specifically we're looking for, however, at that point, it's not a big deal to actually add an additional log and to collect additional information. 

The solution is very scalable. 

There's lots of processing power. You can actually just add machines to get more performance if you need to. It's pretty flexible and very easy to add another log. It's not like 'oh, no, it's going to be so much extra data'. That's not a problem for the machine. It can handle it.

The solution has quite a steep learning curve. The usability and general user-friendliness could be improved. However, that is kind of typical with products that have a lot of flexibility, or a lot of capabilities. Sometimes having more choices makes things more complex. It makes it difficult to configure it, though. It's kind of a bitter pill that you have to swallow in the beginning and you really have to get through it. 

Once you begin to understand the concepts and how to actually look for data it's a very pleasant solution, but the learning curve is very steep in the beginning, to the point that they could improve it to make it a bit less intimidating to start. There needs to be a bit more intuition behind the architecture and the data search.

This solution has been used for at least five years at the company.

It's very stable. The only thing that might happen is that sometimes when you do a search it will stress the machine a bit too much. If that happens, then it's a matter of, if you do it the wrong way, the machine gets stressed and then it slows down. However, it will not crash. It almost never crashes. You'll simply figure out that the machine is overwhelmed and take the stress off. 

The problem, occasionally, is that it may become unresponsive, but it isn't really unresponsive, it's just that the system is overloaded. That can only happen if you do your database search in the wrong way. That's why, especially when you have a lot of data and are really concentrating a lot of data on a few machines, you have to be careful of what you're doing. 

It's a very nice tool but you have to be a bit aware of how to deal with this, especially when you have a lot of data and you have limited processing capacity. If you have unlimited processing capacity you can do whatever you want with it. I personally can say that I've never seen a machine crash.

The scalability of the product is good. It's our key system that generates alerts and does surveillance on a security level. This product is extensively used in our organization.

We have people of course, from the server team that makes sure that the logs get collected. And then we have the people that actually deal with the configuration of the ELK as well. That is a team of five or six people that we use now. Then, of course, we have all the teams that follow up on the alerts, and there, I would say, we have two or three different teams, which is between 10 and 20 people. That's just part of the people that work with the solution.

I work on part of the team that deals with technical support issues. There's a good community around the solution. This is because the product is actually open-source. With a lot of typical issues, you can simply Google questions and you will find the answer. Of course, we do have a support contract with the company. I don't deal directly with that, however. We contact them directly if we really need to and we have maintenance contracts with them. Unfortunately, I can't really speak to how good or bad they are because I've never called them myself.

Before we switched over to this, we used it in combination with an end product called QRadar, but both of them together were time-consuming. 

It's easy to install the servers, that's not really the problem. The difficulty is afterward. Users need to understand how to explore the data.

The server setup is the easy part. Even, let's say, moving the log into the machine or into the database is no problem. However, then you have all this data and you will really struggle to understand the information. That is sometimes not always obvious at the outset. In order to do that in an effective way, it requires a little bit of manipulating.

To install the servers, a minimum installation takes me a day or more. It's for the most part usually pretty fast.

I myself have already had quite a lot of experience with the product. Therefore, I can set it up myself.  Most customers or most IT departments will struggle to set it up due to the difficult learning curve in the beginning. 

I would definitely recommend most users or companies, at least for the beginning, to get help troubleshooting problems. It will help them understand a little bit more about the steep learning curve. It really makes things much easier, and much more effective. 

I have used different products myself due to the nature of my work. I'm a security consultant. I have been working with different customers who use different solutions, which means that I have used other things and can evaluate and compare them for clients.

I've worked with Splunk, for example. Splunk, for instance, on the level of data mining and inquiring, might be easier. It's a bit more intuitive. The downside of it is as soon as you start collecting a lot of data, it becomes extremely expensive to use Splunk. It's a very good product. However, typically, with the need to collect as many logs and as much data as possible, Splunk becomes expensive, and you can't put it in a budget easily. It's simply out of budget for many as soon as they start clicking. Also, the purpose of a security system is not the same.

With Splunk, some will not add additional logs because they don't often have the budget, especially when it immediately means that you're going to need to increase your costs enormously. That's not the purpose of a security system. For the system to be effective you must be able to have good surveillance and that means that you should not hesitate in adding your logs. Still, when the costs double, people hesitate and if they don't have the budget and cut the logs, things can get through. Fortunately, with ELK, you don't have that issue. With ELK you don't pay for gigabytes, or terabytes or the data that you use. That's the main advantage compared to Splunk. But Splunk, it has a less steep learning curve.

I'm just using it as a customer

We tend to use the latest versions of the solution. We try to upgrade it on a regular basis.

I'd advise other companies considering implementing the solution to get a team in that knows the product and try to take advantage of their knowledge. It will help reduce the pain of the learning curve.

I'd rate the solution eight out of ten.

I would not give it a ten because of the steep learning curve. I know what the product is, but many do not, and for them it will be quite difficult to get started without becoming very frustrated in the process. 

We use Elasticsearch as an alternative to Splunk. It is basically for log monitoring.

It's probably a cost-efficient alternative to Splunk. The search interface is nearly the same. When it comes to visualizations, Elastic is a bit better than Splunk.

Elastic Search needs better guides for developers. Better guides for development.

I have been using it for a year.

I would rate the stability an eight out of ten. 

It's fairly scalable. I would rate the scalability of this solution a ten out of ten. 

There are around five end users using it in my team. 

Till date, we did not have any issues with  customer service and support. Like, initially, we had issues in accessing the portal. But that was the only issue, but it was resolved pretty quick.

The initial setup is fairly simple. Initially, it was on-prem, but right now, it's on the cloud.

It is pretty easy to integrate as well.

It's like, when someone is buidling products for scale, it reduces the time to market.

I would rate the pricing a seven out of ten, with one being high price and ten being low price. It could be cheaper for certain use cases, but since it gets the job done, no complaints for the pricing. 

Overall, I would rate it a nine out of ten. I would definitely recommend it to other users. 

The tool's stability and performance are good. 

Elastic Search needs to improve its technical support. It should be customer-friendly and have good support. 

I have been using the product for a year. 

The tool is stable; I rate it an eight to nine out of ten. 

The product is scalable, and I rate it a ten out of ten. My company has three users. We use it regularly. 

You need three resources to handle the deployment. 

The tool is not expensive. Its licensing costs are yearly. 

I rate Elastic Search an eight out of ten. You can use the product if you are looking for value for money.