AWS Lake Formation vs BigQuery comparison

"AWS Lake Formation has several valuable features that enhance data management, and one particularly beneficial aspect is how it facilitates better collaboration within data teams."

"AWS Lake Formation significantly improves the structure of the data mesh, making it superior to previous structures we used."

"There is no doubt that this place exceeded my expectations with its incredible ambiance, attentive service, and mouthwatering menu."

"We have observed measurable benefits in terms of cost savings, time savings, resource savings, and efficiency improvements in our workflows."

"The LF-Tag system with granular permissions was key to the project as a functionality of AWS Lake Formation."

"The most important advantage in using AWS Lake Formation is its ability to connect the data lake to the other technologies in AWS. This is what I advise my clients."

"We use this to reduce latency from minutes to seconds, as we aim for real-time visibility into patient healthcare monitoring."

"The most valuable features of AWS Lake Formation were the access model itself, as it allows implementation of filters, Blueprints, and row-level and column-level security to mask data that shouldn't be accessed by certain entities, enabling granular control without exposing PII data."

"It stands out in efficiently handling internal actions without the need for manual intervention in tasks like building cubes and defining final dimensions."

"The product’s most valuable feature is its ability to manage the database on the cloud."

"This solution is highly scalable."

"BigQuery is a powerful tool for managing and analyzing large datasets. The versatility of BigQuery extends to its compatibility with external data visualization tools like Power BI and Tableau. This means you not only get query results but can also seamlessly integrate and visualize your data for better insights."

"The solution's reporting, dashboard, and out-of-the-box capabilities match exactly our requirements."

"The query tool is scalable and allows for petabytes of data."

"The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting. I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers. Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted. For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage."

"The best features of BigQuery for me are the fact that it's low-code, no-code; you don't have to be a data scientist to really utilize the tool."

"Lake Formation could enhance its capabilities in audit logs, real-time monitoring, and advanced data governance."

"Athena can be a bit clunky when writing queries, indicating a potential enhancement point for easier user interaction with query tools such as DataGrip using provided driver JARs."

"I haven't seen any measurable benefits from using AWS Lake Formation, such as time saving, resource saving, or efficiency improvements."

"The initial onboarding process is challenging because creating a plan takes a month to a month and a half to build out."

"It falls short when it comes to more granular access control, such as cell-level or row-level entitlements which is a significant drawback for organizations that require precise control over who can access specific rows of data."

"I would appreciate online support, which I don't have access to in my corporation at the bank, so that is important."

"The solution could make improvements around orchestration and doing some automation stuff on AWS front automation. It would be useful if we could use automation to build images and use hardened images which are CIS compliant."

"The main challenge we faced with AWS Lake Formation was related to cross-account sharing. Granting access to other AWS accounts for tables or databases in a different AWS account was somewhat difficult."

"Some of the queries are complex and difficult to understand."

"So our challenge in Yemen is convincing many people to go to cloud services."

"The initial setup could be improved making it easier to deploy."

"I understand that Snowflake has made some improvements on its end to further reduce costs, so I believe BigQuery can catch up."

"BigQuery can be very expensive if not used properly. Introducing AI tools that would allow you to optimize the data extraction process is an area of serious improvement."

"An area for improvement in BigQuery is its UI because it's not working very well. Pricing for the solution is also very high."

"I would like to see version-based implementation and a fallback arrangement for data stored in BigQuery storage. These are some features I'm interested in."

"It would be helpful if they could provide some dashboards where you can easily view charts and information."

"AWS Lake Formation is a bit expensive."

"Price-wise, I think that is very reasonable."

"BigQuery is inexpensive."

"The price could be better. Usually, you need to buy the license for a year. Whenever you want more, you can subscribe to it, and you can use it. Otherwise, you can terminate the license. You can use it daily or monthly, and we use it based on a project's requirements."

"We are above the free threshold, so we are paying around 40 euros per month for BigQuery."

"1 TB is free of cost monthly. If you use more than 1 TB a month, then you need to pay 5 dollars extra for each TB."

"The product’s pricing could be more flexible for end users."

"The pricing is adaptable, ensuring that organizations can tailor their usage and costs based on their specific requirements and configurations within the Google Cloud Platform."

"Its cost structure operates on a pay-as-you-go model."