AWS Glue Reviews

We use AWS Glue for handling data-intensive tasks such as data lake creation, log analysis, machine learning pipelines, data warehouse population for analytics, and real-time data integration with AWS Lambda.

AWS Glue has increased efficiency and time saving. It simplifies and automates data pipeline processes, enabling faster data processing and analysis.

The AWS Glue Data Catalog provides metadata management and schema discovery. AWS Glue simplifies data transformation with automatic schema detection, incremental data updates, and integration with other AWS services.

It enables us to analyze data stored in Amazon S3 using SQL, which is manageable and cost-effective.

AWS Glue should be more reliable and faster in processing. Enhancing the speed of data processing would be beneficial.

I have been using AWS Glue for more than one year.

AWS Glue is generally considered stable and reliable for data integration, especially for larger scale production environments. Its serverless architecture and integration with other AWS services contribute to its stability.

AWS Glue is scalable because of its serverless nature, which allows for easy scaling without needing to manage any infrastructure.

The technical support from AWS is very reliable. I would rate it nine out of ten.

I did not use any other cloud data integration solutions before AWS Glue.

The initial setup process for AWS Glue involved setting up Glue resources, creating roles and permissions, and developing ETL scripts and jobs. It took about half an hour.

The deployment process required three developers.

While specific data on time and cost savings was not provided, AWS Glue's benefits include increased efficiency and time-saving.

The approximate cost for ETL jobs is about 0.44 USD, which is mostly covered by the company. Employees do not purchase AWS Glue solutions individually.

AWS Glue is highly recommended for data engineers due to its ability to build and maintain data pipelines, ensure data quality and integrity, and its integration with UI tools. It offers data preparation, machine learning integration, and governance.

I would rate it a ten out of ten.

In my role as the global lead for AWS solutions and offerings, we work with various clients, including large-scale clients, to adopt and implement AWS cloud offerings. 

Our primary focus revolves around cloud lift-and-shift migration, modernization, re-platforming, rehosting, data architecture, design strategy, and implementing generative AI-specific solutions across different industries such as banking, capital insurance, energy utilities, manufacturing, automotive, semiconductor, and aerospace and defense. 

For example, we have implemented AWS Glue at several client locations, utilizing its serverless data integration capabilities during the data discovery process, enterprise transformation, cleansing, transforming, and centralizing data.

AWS Glue has significantly improved our data quality, enhancing the data by removing duplicates and providing timely and efficient insights. 

It also aids in real-time data processing, reducing effort and cost due to its serverless architecture. These features ensure we maintain the highest level of scalability, reliability, and security compliance.

AWS Glue is fully managed, providing an easy-to-use integration environment to create, run, and monitor ETL jobs. It's broadly compatible and seamlessly integrates with other AWS services like Amazon S3, Redshift, and Athena. It's flexible with data integration, manages various data formats (JSON, ORC, CSV, etc.), and is serverless, eliminating the need for infrastructure management.

Its ease of use, cost-effectiveness, and highly secure architecture are some of the most valuable features.

There could be an enhanced way of managing pure metadata management or data cataloging. 

Additionally, while it covers a wide range of integrations with AWS services, integrating with certain additional or legacy products is not seamless and can be complex. 

Increasing support for more programming languages and improving advanced analytics capabilities could also be beneficial.

We have been working with AWS Glue for almost three-plus years now.

We haven't faced any stability issues with AWS Glue. It is a scalable solution, provided that the right design principles and workload management are implemented.

AWS Glue is a scalable solution due to its serverless architecture and efficient design.

My team handles interactions with AWS for technical support, ensuring our design architectures are scalable, flexible, and well-integrated. We often reach out to the AWS team to double-check our implementation mechanisms and guidelines.

Positive

The initial setup of AWS Glue is straightforward due to its serverless architecture and fully managed nature. Specific prerequisites need to be followed, such as setting up data sources, configuring IAM permissions, creating crawlers, and running ETL jobs.

My team escalates technical questions to AWS support, ensuring our design architectures are optimal. We have a partnership with AWS, and the technical team frequently reaches out to AWS for guidance on scalability, flexibility, and integration mechanisms.

We have seen an efficient process with AWS Glue, providing the right return on investment at the right time. It ensures efficiency for our clients, giving them the desired ROI within their expected timelines.

Follow the right design principles and involve AWS at the right time to leverage the most current features and offerings from AWS Glue. Ensuring the right architecture will mitigate any issues. I'd rate the solution eight out of ten.

I had the source data, which was unstructured and non-fixable, and my responsibility was to convert it into structured data. For this task, I used PySpark as the programming language. With Python, I implemented the creation of a data frame using Glue jobs. Since Glue jobs are a serverless mechanism, I deployed my code into the Glue job, and that's how I got the job done.

We no longer had to worry much about infrastructure management because AWS Glue is serverless, and Amazon takes care of the underlying infrastructure. This allowed us to focus on the code and application logic without concerns about scaling, CPU management, or handling fluctuations in flow. The serverless nature of Glue jobs relieved us from these infrastructure-related worries.

The most valuable feature of AWS Glue for me is the ability to write PySpark code and execute it within Glue. I use Glue functions to publish parameters and interact with Glue. 

In terms of performance, if they can further optimize the execution time for serverless jobs, it would be a welcome improvement. Faster code execution would be beneficial. If AWS could enhance the serverless execution capabilities, like increasing CPU, RAM, and processing speed, that would be great.

I have been using AWS Glue for a year. 

In terms of stability, I haven't experienced any issues with AWS Glue. It has been quite reliable in my usage, and I would rate it around nine on a scale of ten.

AWS Glue is highly scalable and reliable. The serverless nature ensures that scaling is automatically managed by AWS. It's one of the strengths of Glue, and there is no doubt about its scalability.

The customer service and support were responsive and helpful in resolving the issues I had at that time. The response time was quick. So, overall, the support was okay.

Positive

Our admin team was responsible for setting up the solution. 

The pricing is a bit higher than EMR. EMR is a managed Hadoop and Spark platform. AWS Glue has a higher price. 

I found that AWS Lambda could be a possible alternative, but since AWS Lambda has a 15-minute execution limit, we opted for AWS Glue, which does not have any execution limits. I don't see any other service that can do what AWS Glue does. 

Other services that could potentially do the same job as AWS Glue would require your application to be deployed in Elastic Beanstalk, EC2, or Container Services, which is another task altogether. Since AWS Glue is already doing its job perfectly, I don't think any other service can do the same job as AWS Glue.

I would recommend that new users refer to the AWS documentation. The documentation is very well-written and easy to understand. Even new users with no prior experience with AWS should be able to get up and running quickly. I would also recommend that new users learn Python.

Overall, I would rate the solution a nine out of ten.

I use AWS Glue primarily for ETL jobs. In my organization, it's just me using it as we are a small company. The IT team consists of four people, and I am the data engineering specialist.

The best thing about AWS Glue is its scalability and how easy it is to process a large amount of data. It integrates well with Redshift, S3, and AWS Glue catalog. 

For processing extensive data, having a managed Spark service fulfills that role. If you're already working on AWS and you need to process a lot of data that can't be handled on a single node or server, AWS Glue will serve you well. While it's quite expensive, it's valuable for large data processing needs.

Setting up pipelines is challenging, especially with version control and testing requirements. While the initial setup is easy, it doesn't accommodate more complex development needs. You might feel hesitant about changing pipelines that are already running and processing business-critical data due to limited versioning and testing capabilities.

I've been using AWS Glue since 2022, so for two years.

The stability of AWS Glue is fine. I haven't had any problems with it.

The scalability of AWS Glue is commendable.

Previously, in different jobs, I have worked with Databricks for ETL processes. I've also utilized Lambda functions for handling smaller data. I didn’t switch to AWS Glue, but used it in a different context.

The initial setup of AWS Glue is easy, yet not adequate for more complex requirements. If you need to do something robust, like creating a notebook, it is straightforward. 

However, when dealing with complex pipelines handling critical business data, it's hard to set up versioning and testing.

AWS Glue receives a hesitant five out of ten from me. I recommend it if you're already on AWS and need to process large data sets. However, for smaller data volumes, I would suggest Airflow because AWS Glue can be quite expensive.

We use the solution to build tables on CSV data. We get data from some different sources, pull it in S3, and then create tables using Glue to get some metrics out of that data.

The entire use case was very easily handled or solved using AWS Glue. We had to get the files available in S3. The workflow was seamlessly integrated with the data, landing in S3, and then it detected changes made in the data. Configuring it was really easy. It gave us what we were looking for without going through a lot of hassle and that too within our budget.

The AWS Glue crawlers are valuable features. It is very versatile. It can detect the nature of the underlying data. It is quite smart, and it takes a lot of offloads rather than having to worry about configuring it or managing it.

There are output limitations and configuration of its three parts. There was a lot of trial and error that we had to go through. It is not clear how the partition discovery would have been affected by more data coming in. We've made some expensive mistakes, which, if there were any tutorials available or if there was easy documentation available with FAQs, could have been avoided. There is documentation, but it doesn't cover all.

There are three specific partition changes, and AWS Glue is tightly tied to Athena. We don't have much flexibility in managing the Athena.

AWS Glue could integrate with an AI model or a more advanced version that processes chat-based inputs rather than configuration. This would align it more closely with the functionalities of chat-based interfaces, making it easier to adopt.

I have been using AWS Glue for four to five years.

The product is stable. There are no issues, errors, or downtime. It is managed by AWS.

I rate the solution’s stability a nine out of ten.

The solution’s scalability is quite high. It is flexible and scalable. We haven't seen any challenges.

I rate the solution’s scalability an eight or nine out of ten.

The customer support is nice and very helpful without AWS premium support. There are community support, medium articles, and AWS knowledge-based articles, which provide AWS support.

Positive

I have used Spark before. We switched to AWS Glue since it is a managed solution. We were writing lines of code, managing it separately on our own, and sending the jobs on a cluster. It is taken care of by the managed service itself. We have to configure it, and then it takes care of all.

AWS Glue catalog was complex to understand. Deployment was very quick. We did some POCs. We were able to take it to production in about six to eight weeks.

We were using the console directly. There are no automated CI/CD. You can manage, create, and set up via the console using the AWS Glue service.

I rate the initial setup an eight out of ten, where one is easy, and ten is difficult.

Deployment was done in-house with the help of two data analysts.

The solution is expensive. It has a pay-as-you-go model. Whatever you are using, you are paying for that.

I rate the product’s pricing a six out of ten, where one is cheap and ten is expensive.

We are looking for Databricks. It is comparable to Spark but it was excessive for a use case. We didn't have the workload. It has a lot of additional features which we don't need. The cost is not justifiable.

Two of us are sufficient for the solution’s maintenance.

The solution is easy to set up and starts with a lot of standard data analytical use cases where we extract data. If you want something customizable, then look at other solutions because cost might be a factor for more advanced solutions.

Overall, I rate the solution a seven out of ten.

We have a lot of microservices written in Glue, which are responsible for triggering based on certain events. The solution will be responsible for another container to containerize them and run over the cloud. We use the solution for different purposes, including data computing.

You do not need many frameworks to run Glue. It's an interpreted language that does not require to be compiled at all.

It is very difficult to learn the tool and remember the syntaxes comparatively. Sometimes, I face issues integrating the solution with some third-party services or services that are not a part of Glue. Such integrations take a lot of time, and not much content is available over the internet for the same.

I have been using the solution for three to four years.

We have eight developers on our team. My team works on almost four to five Glue services. We have four team members working on Glue, including me.

I once faced an issue with Glue. There was a scenario where I wanted Glue to pick certain images, containerize them, and run over the code. That containerization integration wasn't happening successfully. I dropped a couple of messages in the community channel. I got good support from them, which helped me resolve my issue as quickly as possible. The community is very small, but the people are very helpful.

I previously worked in Java, .NET, and Python. I have extensive experience with Python and .NET. Since my organization is language-independent, we have microservices written in almost all the languages, including Glue, Python, Java, and .NET.

I'm not handling the solution's end-to-end deployment, but we have a CI/CD pipeline set up for that. The CI/CD pipeline will remain the same. It's all about how you containerize your Glue application. That is the only challenge we have faced while setting up the deployment. The rest of the configuration was pretty smooth.

Glue is not a must-have tool. You can choose Glue if you have the capability to learn Glue as quickly as possible. There are other alternatives where you will find a lot of articles, study material, and certificates over the internet apart from Glue. If you do not have any other option, go for Glue.

If Glue is not mandatory for you, go for something else because it is difficult to learn Glue and remember the syntaxes. You will need support whenever you have a bigger integration or connectivity with third-party libraries or services. You will not receive many articles or help over the internet. Although the community is available, you need to spend some time with them to make them understand the issue.

It is not easy for a beginner to learn to use the solution for the first time. There are a few videos and courses available, but it's difficult. It's not as easy as other languages in terms of content. It's hard, but you can use it once you understand the concept.

Overall, I rate the solution seven and a half out of ten.

We use AWS Glue for building ETL pipelines.

The most valuable feature of AWS Glue is that it provides a GUI format with a drag-and-drop feature. The solution provides a codeless feature or no code feature, where you can write a pipeline without adding code in AWS Glue.

If you are working on AWS services for your pipeline, AWS Glue better interacts with the AWS services than other third-party tools. The solution provides inbuilt data quality and cataloging features. AWS Glue provides a complete package, and you can do all things in one place.

AWS Glue is more costly compared to other tools like Airflow. It would be better if the solution's pricing could be reduced. The default scheduling that AWS Glue provides is not as good as Airflow. The scheduler of AWS Glue could be improved because you cannot customize it.

I have been using AWS Glue for more than three years.

AWS Glue is a stable product because it's an AWS-managed service. We can directly contact AWS for any issues we face. If there are any glitches in the version, AWS solves the issue by creating patches or version upgrades to the solution.

Around 100 to 200 people are using the solution in our organization.

As AWS Glue is a SaaS product, you don't have to set up anything. You can just create a new job, write your script, and run your job. You have to select the particular cluster or nodes you want to run and write the code. Not much admin part is required for the solution's setup.

AWS Glue is a paid service that doesn't come under the free trial of AWS. You have to pay a charge for using the solution.

If you are doing a job once or twice a day, AWS Glue will not cost you much. If you run jobs for four to five times a day or hourly jobs, it will be costlier compared to other tools. If you are required to run hourly jobs five to six times a day, then using other tools would be a better option. You can choose AWS Glue if you are running jobs only one or two times a day.

Our company decided to go with AWS Glue because the tools we were using in the pipeline were AWS services only. AWS Glue easily interacts with AWS services. The jobs we were running were also not frequent.

You can use AWS Glue for learning purposes. AWS Glue is a paid service that doesn't come under the free trial of AWS. You have to pay a charge for using the solution. You can learn the code by directly testing the basic spark code in any local system. Once you are comfortable that your code is working fine, then you can run your code in AWS Glue jobs. You should test the code in the local system first and then run it in AWS Glue. Testing on AWS Glue will be costly.

If a person is familiar with Spark jobs or Python jobs, they can easily learn AWS Glue. A new user will take the same amount of time to learn AWS Glue as he takes to get comfortable with Spark. Since it provides the GUI and no code thing, users can directly start using AWS Glue without having to learn any code. It's much easier to learn to use the solution.

Overall, I rate the solution a seven out of ten.

In my company, we use AWS Glue to build data engineering pipelines, so we ingest data from either S3 or other sources and put it back into Redshift, where we have a data lake or data warehouse.

With AWS Glue, there have been a lot of updates over the past couple of years. The feature which I like the most is that AWS Glue has now integrated Jupyter Notebook into it, causing AWS Glue to become interactive and develop new use cases. The fact that we don't have to worry about the resources required for any particular job since it gets taken care of by AWS Glue adds to the valuable feature set of the product.

AWS Glue Studio has undergone a lot of enhancements in the last couple of months. An improvement that can help the solution is if the user interface can become more user-friendly and allow for features like drag and drop, allowing it to build transformations. There can be a good improvement if the product itself supports different kinds of transformations so that the pipeline, which we want to create, can be done easily since right now, we have to write a code to do so in our company. Only people who can code, either in Java or Python, can use the product freely. Those who don't know Java or Python might find using AWS Glue difficult.

AWS has pricing for spot instances that reduces the cost substantially, but that is not available for AWS Glue AWS pricing for spot instances comes for products like EC2, and if the same gets introduced for AWS Glue, then the pricing can substantially reduce.

I have been using AWS Glue for five years. I am an end user of the product.

Stability-wise, I rate the solution an eight out of ten.

It is a very scalable solution. In our company, we don't have to bother about the latest features implemented by the solution since the solution itself allocates resources, causing the product to scale up or down as the job progresses.

The solution's technical support is good. Whenever we raise a use case where we face an issue in our company, we get a response from the solution's technical team.

AWS Glue is a platform as a service from AWS, so there is no setup required. It's very simple since you can just go into the UI and start using it.

The solution is deployed on a private cloud.

I am a part of the development team, which doesn't have access to all the different sets of features and has a different backup. There is a team that does the initial setup, user access, and all those things. Once the product is ready, my team takes it over, so the setup is done by someone else.

I rate the product's pricing a five on a scale of one to ten, where one is a high price, and ten is a low price.

AWS Glue can be compared to the database provided by Azure which has a lot of extra features or capabilities compared to AWS Glue which is only useful for processing.

I rate the overall product an eight out of ten.