Elastic Search and AWS Glue compete in the data processing and analytics category. Elastic Search holds an advantage in log monitoring and data visualization, while AWS Glue is superior in data processing and integration with AWS services.
Features: Elastic Search stands out with its integration of Elasticsearch, Logstash, and Kibana for robust log monitoring. It provides comprehensive search functionality and support for key-value pair storage. Its flexibility is enhanced with anomaly detection and multi-language search queries. AWS Glue offers seamless integration with AWS services, simple ETL processes using Spark, and automation features like Glue Studio for minimal coding. It excels in data cataloging and real-time data streaming.
Room for Improvement: Elastic Search could improve its security in the open-source version, enhance machine learning and alerting features, and refine its licensing model. AWS Glue faces high costs, complex integration issues, and limited native services. Enhancements are needed for advanced transformations, multi-cloud capabilities, and a more user-friendly experience.
Ease of Deployment and Customer Service: Elastic Search offers flexibility in on-premises deployments but relies on third-party consulting for direct support. AWS Glue benefits from wide adoption in the AWS ecosystem but lacks substantial support for customized implementations, relying on AWS's technical team. Elastic Search provides community-driven resources, whereas AWS Glue integrates seamlessly within AWS environments.
Pricing and ROI: Elastic Search is open-source with cost-effective options, though backend management can be high for enterprise features. It serves as an alternative to more expensive solutions with variable ROI based on implementation. AWS Glue's pay-as-you-go model emphasizes scalability but can be costly with extensive usage, aligning with modern cloud practices and offering perceived value for streamlined ETL processes.
I advocate using Glue in such cases.
We have not purchased any licensed products, and our use of Elastic Search is purely open-source, contributing positively to our ROI.
It is stable, and we do not encounter critical issues like server downtime, which could result in data loss.
The main benefits observed from using Elastic Search include improvements in operational efficiency, along with cost, time, and resource savings.
Upgrades occur every four months, and new developments coincide with version updates.
It is beneficial to upgrade jobs, and we conduct extensive testing in development before migrating to production.
It can easily handle data from one terabyte to 100 terabytes or more, scaling nicely with larger datasets.
I would rate its scalability a ten.
I can actually add more storage and memory because I host it in the cloud.
I would rate the scalability of Elasticsearch as an eight.
AWS Glue is highly stable, and I would rate its stability as nine.
The data transfer sometimes exceeded the bandwidth limits without proper notification, which caused issues.
The stability of Elasticsearch was very high.
Migrating jobs from version 3.0 to 4.0 can present compatibility issues.
With AWS, I gather data from multiple sources, clean it up, normalize it, de-duplicate it, and make it presentable.
A more user-friendly and simpler process would help speed up the deployment process.
This can create problems for new developers because they have to quickly switch to another version.
It is primarily based on Unix or Linux-based operating systems and cannot be easily configured in Windows systems.
The consistency and stability of Elasticsearch are commendable, and they should keep up the good work.
Costing depends on resource usage, and cost optimization may involve redesigning jobs for flexibility.
AWS charges based on runtime, which can be quite pricey.
The smallest cost for a project is around €700, while the largest can reach up to €7,000 based on the scale of the usage.
We used the open-source version of Elasticsearch, which was free.
For ETL, I feel the performance is excellent. If I create jobs in a standard way, the performance is great, and maintenance is also seamless.
AWS Glue's most valuable features include its transformation capabilities, which provide data quality and shape for processing in ML or AI models.
I think if I'm working with big data, common languages like Python work quite nicely, which is advantageous.
Elastic Search makes handling large data volumes efficient and supports complex search operations.
The most valuable feature of Elasticsearch was the quick search capability, allowing us to search by any criteria needed.
Aggregation is faster than querying directly from a database, like Postgres or Vertica.
AWS Glue is a serverless cloud data integration tool that facilitates the discovery, preparation, movement, and integration of data from multiple sources for machine learning (ML), analytics, and application development. The solution includes additional productivity and data ops tooling for running jobs, implementing business workflows, and authoring.
AWS Glue allows users to connect to more than 70 diverse data sources and manage data in a centralized data catalog. The solution facilitates visual creation, running, and monitoring of extract, transform, and load (ETL) pipelines to load data into users' data lakes. This Amazon product seamlessly integrates with other native applications of the brand and allows users to search and query cataloged data using Amazon EMR, Amazon Athena, and Amazon Redshift Spectrum.
The solution also utilizes application programming interface (API) operations to transform users' data, create runtime logs, store job logic, and create notifications for monitoring job runs. The console of AWS Glue connects all of these services into a managed application, facilitating the monitoring and operational processes. The solution also performs provisioning and management of the resources required to run users' workloads in order to minimize manual work time for organizations.
AWS Glue Features
AWS Glue groups its features into four categories - discover, prepare, integrate, and transform. Within those groups are the following features:
AWS Glue Benefits
AWS Glue offers a wide range of benefits for its users. These benefits include:
Reviews from Real Users
Mustapha A., a cloud data engineer at Jems Groupe, likes AWS Glue because it is a product that is great for serverless data transformations.
Liana I., CEO at Quark Technologies SRL, describes AWS Glue as a highly scalable, reliable, and beneficial pay-as-you-go pricing model.
Elasticsearch is a prominent open-source search and analytics engine known for its scalability, reliability, and straightforward management. It's a favored choice among enterprises for real-time data search, analysis, and visualization. Open-source Elasticsearch is free, offering a comprehensive feature set and scalability. It allows full control over deployments but requires managing and maintaining the infrastructure. On the other hand, Elastic Cloud provides a managed service with features like automated provisioning, high availability, security, and global reach.
Elasticsearch excels in handling time-sensitive data and complex search requirements across large datasets. Its scalability allows it to handle growing data volumes efficiently, maintaining high performance and fast response times. Integrated with Kibana, Elasticsearch enables powerful data visualization, providing real-time insights crucial for data-driven decision-making.
Elastic Cloud reduces operational overhead and improves scalability and performance, though it comes with associated costs. It is available on your preferred cloud provider — AWS, Azure, or Google Cloud. Customers who want to manage the software themselves, whether on public, private, or hybrid cloud, can download the Elastic Stack.
At its core, Elasticsearch is renowned for its full-text search capabilities, capable of performing complex queries and supporting features like fuzzy matching and auto-complete.
Peer reviews from various professionals highlight its strengths and weaknesses. Pros include its detection and correlation features, flexibility, cloud-readiness, extensibility, and efficient search capabilities. However, users have noted challenges like steep learning curves, data analysis limitations, and integration complexities. The platform is generally viewed as stable and scalable, with varying degrees of satisfaction regarding its usability and feature set.
In summary, Elasticsearch stands out for its high-speed search, scalability, and versatile analytics, making it a go-to solution for organizations managing large datasets. Its adaptability to different enterprise needs, robust community support, and continuous development keep it at the forefront of enterprise search and analytics solutions. However, potential users should be aware of its learning curve and the need for skilled personnel for optimization.
We monitor all Cloud Data Integration reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.
