AWS Lake Formation Reviews

I am an end user of AWS Lake Formation. We use AWS Lake Formation to control all our data lake at the UOL company. We use AWS Lake Formation to control privacy and build privacy controls. We split information based on user roles, and AWS Lake Formation ensures that users can access only specific data. Beyond that, we use AWS Lake Formation to control our schemas. By controlling schemas, we define the types of data we input. For instance, we have schema control that determines which field is char and which field is int. We use AWS Lake Formation to control that and input descriptions in these fields. For instance, if we have a table called sales with a field ID, our team inputs what this field means in the application. ID means the number of identification of our student or client. We use AWS Lake Formation to manage permissions and governance.

In our company, we use AWS Lake Formation just to manage data. We use other solutions to transform data.

Without doubt, the privacy control of AWS Lake Formation is most valuable. In Brazil, as in other countries with LGPD or GPDR and other types of laws that we need to follow, I can make sure we are compliant using AWS Lake Formation. This is a very important and useful feature.

These features of AWS Lake Formation currently solve all our issues and challenges to be compliant with laws. The capabilities we already have are enough for us.

In our case, all the features of AWS Lake Formation that we use in other services, for instance, Athena, are based in AWS Lake Formation. It is very useful as AWS Lake Formation controls all our information in the data lake.

To provide more context, all of our company's decisions are made based on our data, which is managed by AWS Lake Formation. We make many decisions and generate numerous insights for the company based on this data managed by AWS Lake Formation.

There is a specific issue when we list the tables and database in AWS Lake Formation regarding the UI. When we list tables, and to provide context, here and in many companies, we have numerous tables and schemas. This large amount of tables generates a long list. The UI of AWS Lake Formation is not very responsive. The UI has a lag when showing information, when clicking from one page to another, or when trying to find information. There is a search box at the top of the page, and when we type, searching for a sales table for instance, the response takes a long time. This complaint is not exclusive to AWS Lake Formation, but we observe this behavior in AWS Lake Formation as

For more than three years.

I do not use AWS Lake Formation's ability to automatically classify data and enforce security policies.

From zero to ten, I would rate the stability and reliability of AWS Lake Formation as nine.

The scalability depends on how the solution is deployed. AWS Lake Formation has been updated more than three times. We are currently on the third version. In this third version, they released many improvements that afford larger scalability. While previous versions were not as capable of scaling, based on this last version, it is very scalable.

I have interacted with their support team. Our support for AWS Lake Formation is either business or premium level, based on our contract. When we ask for help, they respond almost immediately with a first response. Based on my previous experience, I would rate the support nine out of ten.

I used a Hive solution for these use cases a long time ago, but it is outdated now. I have worked with AWS Lake Formation for more than three years.

I moved from another company that used Cloudera, which is a different type of infrastructure where we used Hive for managing the data catalog. At my current company, I use AWS Lake Formation. Comparing both solutions, AWS Lake Formation is more robust and easier to manage than the previous solutions I used.

Based on the values that AWS Lake Formation has for us, I think it is fair enough.

I would rate AWS Lake Formation nine out of ten based on everything we have discussed.

I am the Data Engineering Manager at UOL Edtech.

I will be able to manage my review, and it is acceptable for me to make my review public.

I would be happy to share the main use cases for AWS Lake Formation and what it is used for.

AWS Lake Formation has several valuable features that enhance data management. One particularly beneficial aspect is how it facilitates better collaboration within data teams. The solution provides specific features that support team coordination and data sharing capabilities.

There are areas of AWS Lake Formation that could be improved or enhanced in the future. Information about the pricing, cost, and setup cost of the AWS solutions would be beneficial.

[Full sentence answer to 'For how long have I used the solution?' from the text.]

The integration with IAM for authentication and authorization is worth mentioning. When concerns or technical questions arise, these can be escalated to the AWS team and technical support or customer service team.

Based on experience with AWS services, I can share advice and recommendations with other users who are looking to implement AWS Lake Formation for their environment.

PeerSpot is a review platform and informational platform that hosts reviews from real users on different types of solutions. There is a partnership with AWS to open new categories regarding different AWS services. The feedback is published on peerspot.com, and users have access to edit their contributions. The feedback is also provided to AWS about customer experience, what customers appreciate about the solution, and what improvements they would to see implemented.

AWS Lake Formation is used to manage access to data lakes on the AWS platform, on AWS Cloud. AWS Lake Formation uses the Glue service underneath to manage data access and metadata about the data. It grants permissions for data stored in the data lake, making data sharing with different entities, users, or groups convenient. We use AWS Redshift to access data in S3 with Lake Formation permissions.

AWS Lake Formation is a security mechanism for accessing databases and tables on S3. It is similar to relational database technologies, offering familiarity and ease of use in the data world. It centralizes access management, simplifying complex governance.

The tag-based access control option is an improvement over previous solutions such as raw Glue and S3 permissions. AWS Lake Formation also supports secure data access, which works effectively with our automated infrastructure as code processes.

AWS Lake Formation might not be easy to learn initially, especially for technical staff, due to various underlying options. There were errors and bugs, sometimes making data access challenging. AWS support often helped resolve these issues.

Extending AWS Lake Formation's scope to non-AWS services could enhance its flexibility, but this hasn't been considered yet.

The solution has been in use for three years, two years, and two years respectively.

Initial issues were reworked or redesigned. For the past two years, AWS Lake Formation runs smoothly without significant problems.

The support team is highly skilled and proactive, willing to help. While colleagues reported mixed experiences, my interactions have been consistently positive.

Positive

Initially, the data platform ran without AWS Lake Formation. However, exact details from previous projects are not recalled.

The learning curve was challenging. Migrating to tag-based access control required learning new methods. Initially, we utilized named resources before switching to tag-based access control for its advantages.

AWS Lake Formation has been on the market for about three to four years. It's mature, continuously evolving, and recommended by AWS for building data platforms. It offers centralized access management reducing complexity, allowing access sharing across AWS accounts, aiding growth.

AWS Lake Formation offers flexible access, including column and row-level restrictions. Compatibility is limited to specific AWS services.

The cost impact of Lake Formation is negligible compared to other services. It's primarily a security-focused service, with cost-effectiveness being more relevant elsewhere on AWS.

Planning a big data platform should consider AWS Lake Formation from the start as it assists in setting up efficient data management solutions.

I rated the solution a nine out of ten. However, I may need more time to finalize my decision, pending a more comprehensive review.

When it comes to AWS Lake Formation, it is about different ways AWS can help create, manage, and help with the data transformation that clients are looking for.

There are many use cases around Lake Formation. Typically, clients are reimagining their entire process from the standpoint of user experience, infrastructure duplication, and data duplication. Often, data isn’t organized properly, which affects both the user experience and infrastructure. So, when they’re reimagining their data strategy, one of the conversations revolves around Lake Formation.

Lake Formation is essentially about creating a centralized repository. It allows you to structure your back-end repository, whether the data is structured or unstructured. You can scale it and manage it effectively. It also covers securing your data and implementing various cataloging mechanisms for quick data retrieval. So, Lake Formation is crucial when clients are reimagining their data landscape.  

Another product is Amplify. Amplify is more focused on the developer experience, especially when transitioning from a traditional waterfall model to more agile methodologies like DevOps or DevSecOps. It supports full-stack development for both web and mobile applications.

AWS Amplify offers a lot of automation capabilities. It keeps the developer experience at the forefront, providing automation for CI/CD pipelines, many out-of-the-box features, and seamless API integrations. Amplify handles these with native capabilities, making it a powerful tool for developers.

There are a couple of areas for improvement with Lake Formation. One of the main challenges, especially when dealing with rich media content, like in MarTech (Marketing Technology) or ad agencies, is its versatility. Some clients feel that Lake Formation doesn’t meet their needs and they tend to prefer competitor products for those specific use cases.

The second area for improvement is in data governance. Specifically, Lake Formation could enhance its capabilities in audit logs, real-time monitoring, and advanced data governance. This includes managing the entire data lineage—where the data originated, how it moves, and where it’s currently stored. The visibility of the data as it evolves is crucial, and that’s where more advanced governance capabilities would be beneficial.

I have a pretty good understanding of the entire AWS suite. I’ve been working with AWS for many, many years, if not almost a decade.

When cloud providers like AWS started, there were a lot of skeptics, including myself. For instance, the data center AWS was never going to move. 

And then, fully and surely, we changed our own perspective and the client’s perspective. So it’s been quite a long journey, and I’ve been through the journey for over ten plus years.

As AWS customers, we typically handle the initial sale. From what I understand, Lake Formation is highly scalable.  

It's fairly easy to use for people who have been trained on it. But again, we're moving closer to the technical user and the non-technical user divide.

What my teams tell me is that when we go to sell and the client has AWS-certified staff, they say, "Hey, this is easy to use." Lake Formation is fairly intuitive. However, as AWS expands and more users start using Lake Formation, there is a need for some democratization. This would allow business users, like product managers, to access the system without needing developers or technical staff.

I think there's a need for more pre-configured templates to really democratize the usage—not for the formation of the lake itself, but for retrieving complex queries. More templates and pre-configured queries could allow more users to interact with the system.

The pricing really depends on the kind of contract you have with AWS and who is selling you the AWS services. There are many ways to consume AWS services. It depends on whether you have an enterprise license, if you're buying direct, if you're using Lake Formation or other products. It's not an easy answer to give because there are so many variables.

In general, though, AWS Lake Formation works hand in hand with other products. You're not just paying for Lake Formation; you also need S3, Glue, Athena, and other services to get the full value out of the data lake. No one buys a data lake just to store data—it’s actively used to retrieve, query, and gain insights that provide a competitive edge. So, the pricing becomes a bit complex because it involves more than just Lake Formation.

Overall, I would rate the solution a seven out of ten, with ten being the best and one being the worst, because there are a few shortcomings. Many clients are moving away from using just one hyperscaler to multiple hyperscaler, so hybrid cloud scenarios are becoming more common. Lake Formation would need to integrate with not just AWS, but also with other hyperscalers, whether it's on-premise, off-premise, or in the cloud. Scalability and integration with competitors are important factors.

Then, there is data governance and the need to support more data formats. As the use cases expand from core IT data to other business functions, like CAD/CAM drawings or marketing with rich content and videos, Lake Formation needs to support these versatile data formats. It seems limited in terms of the data formats it can support.

I use AWS Lake Formation primarily for data lakes.

Since using data lakes on AWS, I have not experienced a single breach. Additionally, I have had zero downtime for the last year and a half.

The most valuable features of AWS Lake Formation are its reliability and the ability to make a backup of the data lake. I can easily move data from cold storage to regular storage. I also like its stability, as I have not had a single downtime in a year and a half.

If I could improve AWS Lake Formation, I would add more integrations with SageMaker. I would have built-in functions that provide statistics for the data when using the GUI, such as SageMaker Insights.

I have been using AWS Lake Formation for about a year and a half.

The stability is unbeaten, as I have had zero downtime for a year and a half.

I use the CLI and find it easy to scale up or down.

I have not talked to their support team at all.

I used OpenStack for data storage previously, where I experienced expected downtimes of around one to two times per month.

From my experience, deploying and setting up AWS Lake Formation is about a medium difficulty. I program the actual interfaces using the CLI.

The pricing is expensive compared to OpenStack, but cheaper than other cloud environments. It's middle-of-the-road for regular storage yet very cost-effective when using Amazon Glacier for data.

I would not recommend AWS Lake Formation to others as it gives me an advantage.

I'd rate the solution seven out of ten.

Our data team uses the solution for ETL jobs.

AWS Lake Formation lets you see all your data and tables on one screen. It's a good product to have everything under one service. In order to be able to do ETL jobs and create some tables from the S3 buckets, you have to use a few different services. With AWS Lake Formation, you can merge everything into one single service and have all your data under that service.

Not everybody can use AWS Lake Formation. You need to have data experience to use the product.

I have been using AWS Lake Formation for five years.

We haven’t faced any issues with the solution’s stability.

The solution’s technical support is very good.

The solution's initial setup is not that hard. Depending on your setup, you need some experience to deploy the solution.

I would recommend the solution to other users.

Overall, I rate the solution a nine out of ten.

It primarily focuses on managing data permissions and entitlements and plays a crucial role in data control and governance within AWS. Once our data is stored in the cloud, we grant access to various users and applications for various purposes, such as analytics, reporting, or other data-driven activities. To manage these access patterns, we define permissions either by giving direct access to users or by creating roles via AWS IAM which can be configured within AWS Lake Formation.

It is seamlessly integrated within the AWS ecosystem, making it straightforward to manage access patterns for AWS-native services.

There are significant challenges when dealing with external applications, and vendors, or pursuing a hybrid cloud strategy because AWS Lake Formation is specific to AWS and does not integrate seamlessly with third-party services or other cloud providers. Another limitation is that Lake Formation might not offer out-of-the-box automated reporting, so you may need to create your solutions for tracking and reporting on permissions granted to users. It falls short when it comes to more granular access control, such as cell-level or row-level entitlements which is a significant drawback for organizations that require precise control over who can access specific rows of data.

I have been working with it for three and a half years.

You can reach out to AWS support through support tickets, and their response time is generally quite efficient. Typically, you can expect a response within a few hours, sometimes even on the same day, and at the most, by the next day. Also, if your organization is a significant AWS customer, you may have recurring calls with AWS account managers. These calls provide an opportunity to discuss service roadmaps and request specific features or enhancements for a service. AWS takes these requests seriously and may prioritize them on its roadmap, often opening a feature request and providing a timeline for its implementations. I would rate it seven out of ten.

Neutral

Our company has adopted both AWS and Azure, and we are looking for a solution that aligns strategically with our multi-cloud environment. While there are certain drawbacks with AWS Lake Formation, especially in terms of integration and fine-grained access control, it's essential to choose a solution that works cohesively in this hybrid cloud setup.

While AWS Lake Formation may have certain drawbacks, especially when it comes to integration with external systems and cell-level entitlements, it can still be a valuable choice if your use case is entirely AWS-native. In such cases, it's recommended to leverage the native AWS services available for managing data access and governance. However, if you're operating in a multi-cloud environment with Azure in the mix, it might be more beneficial to explore alternative solutions that offer better integration and more extensive access control capabilities. Overall, I would rate it six out of ten.

We use AWS Lake Formation typically for the data warehouse.

Customers choose to go with AWS Lake Formation because it's a cloud-based solution, and customers don't want to invest in on-premises hardware and software.

AWS Lake Formation's pricing could be cheaper.

I have been working with AWS Lake Formation for two years.

AWS Lake Formation is a stable solution.

AWS Lake Formation is a scalable solution. Our customers are mostly medium to enterprise-level businesses.

AWS Lake Formation's technical support is good and satisfactory.

Positive

AWS Lake Formation is a bit expensive.

My advice to users is that they first need to know what they want to achieve out of AWS Lake Formation. They need to have a clear-cut expectation from the solution, and they should involve a knowledgeable partner.

Overall, I rate AWS Lake Formation an eight out of ten.

We primarily use the solution as a cloud data warehouse.

The security of the solution is pretty good.

The solution has many features that are applicable to events such as audits.

The solution offers a lot of documentation online that users can use to help with troubleshooting or learning the systems. We rely on these materials when we need support.

The solution could make improvements around orchestration and doing some automation stuff on AWS front automation. It would be useful if we could use automation to build images and use hardened images which are CIS compliant. 

For AWS, currently, we are facing the challenge when previewing. On AWS, we have set up a scanner instance with a Nexus scanner. For some reason, the scanner instance has been set up on AWS so that if it's in one region or maybe if it is deployed on one subnet, the other subnet is not reachable within the same region. The scanner is not reachable to the different subnets. For example, if there is a subnet A where the scanner is deployed, and if we want to scan the subnet B within the same region, the scanner is not reachable due to the fact that it's in Subnet A. That's a problem we are currently facing and trying to troubleshoot that.

I've been using the solution for about three years at this point.

The solution is very stable. We haven't experienced issues such as bugs or glitches. AWS if very good, performance-wise. It doesn't crash or freeze.

I don't really have any insights into scaling the solution. It's not my area of expertise. I haven't attempted to scale yet, so I can't speak to its capabilities.

We have different project teams using the solution across the organization curently.

We've never dealt with technical support just yet. If there is a need, we might reach out. However, I can't speak to their level of service as we've never used them yet.

Each team in our organization has the option to work with whatever solution they like. Others might use Azure, for example. We, right now, use AWS which we find to be more agile and scalable.

Whichever solution a team chooses wold be based on their own unique requirements.

I wouldn't describe the initial setup as complex. It's pretty straightforward.

For deployment, you basically just download it and start working. It's easy.

We didn't really have a deployment strategy in place. We are at the start of all of this automation stuff. Right now, different project teams are using the cloud. It's new for them as well. We are trying to see how best we can try to get these images, hardened images, deployed in an automated fashion.

There is a dedicated team that handles maintenance on the product. Every product team is responsible for their own setup plan (subscription), and they maintain it. Maintenence might be five or ten people per 1,000 users. It depends on the team size. Eveyone is scattered all across the globe. It's pretty difficult to say which team is handling maintenance and how many are responsible for it as different teams are independantly using the solution. 

I'm not privvy to licesning information. I'm not sure how much the solution costs.

Our company does not have a business relationship with AWS. We're just a customer.

I'm not sure which version of the solution we're using.

The thing is that we are not forcing the end-user or the project team to deploy the application. They have the liberty to choose their own platform. Either they can go with AWS or they can go with Azure. Most of the projects which are deployed on non-Windows OS tend to go for AWS.

My area of expertise is more in the auditing of the security standards and maintaining the security requirements as per the industry standard. I'm more in a security architect kind of role.

Every solution has its pros and cons, however, for our purposes, this solution works quite well for us.

I'd rate it overall at eight out of ten.

We primarily use the solution in order to build infrastructure. It's basically for building network infrastructure.

The solution is quite good at handling analytics. It's done a good job at helping us centralize them.

We occasionally work with third-party libraries and they fail to integrate with this product effectively.

For the end-users, it's not as user-friendly as it could be. For example, if I have a Lake Formation return, once the code is in place, then working Lake Formation, or deploying Lake Formation code, still requires an engineer as opposed to being automated to the extent where end-users can actually deploy the whole infrastructure. 

The other issue is the debugging on the solution is not so straightforward. This should be simplified.

I've been using the solution for about a year and a half. It hasn't quite been two years. I haven't used it for that long.

The stability has come with time. At first, it wasn't so stable. We had some trouble and there was a bit of a learning curve. It gets easier and much more stable after you deploy.

While we are okay with the tooling, we're still working on the scaling process. Right now, there are no immediate challenges.

The solution isn't used by end-users. It's just used by our engineering staff who are working on the infrastructure.

Often, the people who handle the initial contact may or may not have the experience you need. It takes a while sometimes to get a person who can help. If it's a bigger issue, most of the time we have to engage the senior associates, and sometimes it takes time to find the right person. 

They also have hundreds of services there, and therefore, depending on where the problem is, finding the right person for the right service is an issue, due to the fact that, on the onset as an end-user, I don't know where the problem could be.

For example, while I may be working on the Lake at the surface, the problem may be on the Lake Formation. However, it could also be an EMR issue. Or it could be an issue in Spark, or it could be an issue within the EMR. and so on. That's why it's hard to get to the right person. You need to figure out the root cause then they need to figure out the best person to deal with the root cause.

That said, in terms of responsiveness, I don't have a problem with them. They do try to be helpful. It's just that the way the system is designed,getting to the right technical hand is complex and can take a while.

We didn't previously have another solution. Our instinct is to stick with internal solutions before moving out and looking for third-party options.

I would not describe the initial setup process as straightforward. There's definitely a learning curve involved. However, once you understand everything, it does get easier.

Once you deploy the solution, it stabilizes pretty fast and everything is a bit better from there onwards.

We're strictly customers of AWS. We're end-users.

I'm not sure which version of the solution we're using.

My advice to other organizations would be to not think of this solution as a one-time activity. It's an ongoing thing. It's a big change and there will new services and therefore it's constantly changing. You have to make sure that it's part and parcel of your everyday operations. 

The solution is pretty stable, or stable in the long run. However, every single day there is a need to grow the infrastructure, or there are new services being added externally and internally. You deploy more applications. You deploy more code. You deploy new libraries. You'll have to stay on top of everything to make sure it stays aligned with changes.

Overall, I would rate the solution at a seven out of ten.