BigQuery Reviews

We are primarily using the solution to crunch data. Then, we are doing some ETL work on top of the data. 

We like that it is a serverless system. 

We can analyze terabytes of data in a very small amount of time. 

It's similar to a Hadoop cluster, except it's managed by Google.

The initial setup is simple.

We find the product to be very stable.

It scales quite well.

If they can provide any charting platform on top of this product, that would be ideal. BigQuery now only allows us to run queries. It doesn't provide us with any insights. For example, if a query took so many times, they could maybe provide any suggestions on how to optimize the queries or speed up the process. It would be helpful if they could provide some dashboards where you can easily view charts and information. That would be very useful.

I've been using the solution for two or three years. 

This is a highly stable product. There are no bugs or glitches. It doesn't crash or freeze. 

The solution is very scalable. 

Almost my entire team uses it. We have a 50-member team, and pretty much everyone is on it. They are mostly data engineers and developers. 

We have yet to reach out to technical support. We haven't had any issues. 

We chose this solution specifically since all of our services are in GCP, Google Cloud. Google Cloud has a basic internal coupling with BigQuery. That's the reason we are using BigQuery.

The initial setup is very easy. You just have to log in to the Google Cloud console, and then you can just create a few tables and start using it. 

From start to finish it takes about half an hour. It is even less than that to get the tables up and running. The deployment is quite fast.

I'm not sure about the exact cost, however, it is charged on the queries which you run, basically. For example, if you run a query, the amount of data scanned through BigQuery will dictate the costs. 

I am a customer and end-user.

I'm not sure which version of the solution we're using. 

It's a serverless platform deployed on a public cloud. 

I'd advise potential users to set up their tables accordingly. There are two sets of optimization that BigQuery provides as well. You set up whichever columns you want to do the partition and on which columns you want to do the clustering. If these columns are defined properly, then BigQuery's a breeze to use.

On a scale from one to ten, I would rate it at an eight. If they just added a few more features, it would be almost perfect.

We use Cloud SQL for our web applications. Previously, we used Microsoft Cloud, but we transitioned due to cost benefits. We find Google Cloud Platform (GCP) to be more cost-effective. For BigQuery, we store data in a message queue similar to Kafka, and when an event occurs, that data is triggered to be inserted into a BigQuery table through subscriptions.

We have seen significant improvements in data management processes, particularly with integration capabilities that allow us to easily retrieve and manipulate data through simple queries. This enhances our workflow significantly.

BigQuery allows for very fast access, and it is efficient in handling large datasets compared to other SQL databases. It integrates well with other GCP products, and creating subscriptions in the UI is straightforward. The whole ecosystem of GCP products makes BigQuery beneficial for our data-handling tasks. Additionally, it is more cost-effective compared to alternatives like AWS.

There is a limitation when copying data directly from BigQuery; it only supports up to ten MB when copying data to the clipboard. For larger data, we have to download it as JSON or Excel files. This limitation could be addressed for better usability.

I have been working with Google Cloud SQL for over one year.

I have not experienced any downtime issues with the solution; it has been stable.

There are some limitations in scalability, particularly when dealing with very large datasets. However, the cost savings we gain often balance this out.

We haven't had much interaction with technical support outside of accessing documentation available online.

Positive

Before using BigQuery, we were utilizing Azure Warehouse. We switched to BigQuery as GCP services were slightly cheaper and more cost-effective. The cost savings were a significant factor in our decision.

The initial setup process for BigQuery was straightforward. By using Terraform, we were able to manage the entire setup efficiently and keep track of who is making changes.

We have a team of developers who manage the platform. Infrastructure changes are tracked, and project owners approve updates.

I can't provide concrete documentation on ROI, but GCP's evolving services have been more cost-effective compared to AWS.

AWS has a large number of users and has built a model with high costs, whereas GCP offers cost-effective solutions.

AWS was another option considered, but due to cost considerations, we opted for GCP.

I recommend BigQuery, especially if you're already using GCP products, as the integration with other Google services is seamless.

I'd rate the solution nine out of ten.

BigQuery allows you to quickly analyze logs from your systems to identify the severity of issues. It integrates well with other Google Cloud services, such as Cloud Logging, where you can easily manipulate various data types and analyze all logs.

BigQuery excels at data analysis. It processes vast amounts of information using its advanced architecture and sophisticated querying capabilities, making it crucial for critical insights and safe for handling sensitive data. 

BigQuery should integrate with other tools, such as Cloud Logging and Local Studio, to enhance its capabilities further and enable powerful and innovative analyses.

I have been using BigQuery for two years.

I have opted for Fireye, Elasticsearch, and Alcon. One principal difference is that BigQuery starts with machine learning and WAN implementations, while you can implement VMware or other active boxes. Therefore, it is recommended that cloud VMs be used for BigQuery processes. You can execute jobs in the cloud, such as VMware.

For instance, you can compute analytics for email, apply filters, and manipulate weather data. It provides higher efficiency, though exact benchmarks are unclear. Additionally, starting the query flow login request can also be advantageous.

The initial setup is automatic. It requires one person. You need to log in to the Google Cloud platform, import the necessary package into your query, and then you can start querying your data. 

If you need a solid CRM solution integrated with Azure, you'll need knowledgeable people to support it. Three individuals can form a strong CRM team connected to Azure, leveraging BigQuery.

You can use BigQuery to generate and manage large datasets efficiently. Whether using a flexible integrated environment like Dataflow or a local studio, BigQuery provides powerful tools for querying and analyzing data.

The product is free of cost.

Setting up BigQuery on GCP is crucial. When creating a service account, you define the permissions required for project identification or access monitoring systems.

You configure policies using IAM roles to manage access permissions effectively within GCP. These roles govern the service accounts created for specific tasks such as data processing, system monitoring, or other service integrations. When you activate these policies, a JSON token is generated. This token can authenticate and authorize access to Google services like BigQuery or other third-party applications.

Moreover, by configuring VMs to match data processing requirements, you ensure that the data is securely handled by the applications associated with the service accounts. This setup enables seamless communication between your applications and Google services, facilitating efficient data acquisition and processing.

Overall, I rate the solution a seven out of ten.

We use BigQuery to store data in a table and query it. Data storage can be either an internal native table or an external table where the external source will point to Google Cloud Storage or Google Drive. 

Wherever we can have external storage, we can have a table built pointing to that external storage and query the tables. In BigQuery, we can query the table or even do DML operations, like insert, delete, etc.

The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. 

The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting.

I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers.

Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. 

It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. 

To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted.

For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage.

It would be better if BigQuery didn't have huge restrictions. For example, when we migrate from on-premises to on-premise, the data which handles all ebook characters can be handled on-premise. But in BigQuery, we have huge restrictions. If we have some symbols, like a hash or other special characters, it won't accept them. Not in all cases, but it won't accept a few special characters, and when we migrate, we get errors. 

We need to use Regexp or something similar to replace that with another character. This isn't expected from a high-range technology like BigQuery. It has to adapt all products. For instance, if we have a TV Showroom, the TV symbol will be there in the shop name. Teradata and Apache Spark accept this, but BigQuery won't. This is the primary concern that we had.

In the next release, it would be better if the query on the external table also had cache. Right now, we are using a GCS bucket, and in the native table, we have cache. For example, if we query the same table, it won't cost because it will try to fetch the records from the cached result. But when we run queries on the external table a number of times, it won't be cached. That's a major drawback of BigQuery. Only the native table has the cache option, and the external table doesn't. If there is an option to have an external table for cache purposes, it'll be a significant advantage for our organization.

I have been using BigQuery for more than three years.

BigQuery is a stable solution.

BigQuery is highly scalable. We can have unlimited storage if we do 20 records, and It's very fast. Even if we scale it to 20 trillion, it will still be fast. 

In my organization, about two in five use BigQuery. When I joined the company a year back, usage was relatively moderate. However, now usage increased because of the on-premise to GCP migration. Because of many successful projects, several people are using BigQuery now.

We have dedicated support people who help us with the framework. If there is a technical issue in BigQuery, we just get help from the technical team. But if there are any engineering issues or some data issues, our team will handle them.

I use Teradata and then Apache Spark on-premises.

The initial setup is relatively straightforward. There are some restrictions, like the project's name. It has to be unique, but once that project is created, we can simply go to an option, query, and the query control will open, and we can start creating a table, loading data, querying, and everything. So that's quite simple and straightforward.

When I joined PayPal, the setup was done in-house. When I worked at another organization, Cognizant, we had Google's help. So a Google specialist helped us set up and everything.

I have tried my own setup using my Gmail ID, and I think it had a $300 limit for free for a new user. That's what Google is offering, and we can register and create a project.

On a scale from one to ten, I would give BigQuery an eight.

I usually need to catalog. In my case, it's more related to data governance. I need to catalog information from BigQuery. I want to ensure the data quality tool is in sync with BigQuery, so I go to BigQuery and do queries to make sure it was synced with Atlan, for example, for data quality tools. I create validation rules and need to write the rule in BigQuery to create a query there, see how long it takes to run, and evaluate its performance in a data quality tool.

What I have seen is that they are using BigQuery as a central repository. They bring dispersed information to BigQuery, which facilitates exploring the data and gaining insights. Consequently, it improves operations, response time, and the business overall.

As a user, I have liked using BigQuery to create queries. They have a table explorer feature that allows you to select a table, choose fields, and generate queries easily, which significantly facilitates my workflow. I also appreciate the lineage feature, which shows how tables relate to each other and enables end-to-end usage visualization. 

Furthermore, its integration with other tools like Atlan through a Google Chrome extension is highly beneficial. Using BigQuery's central repository brings dispersed information together, which facilitates exploring the data and gaining insights. Consequently, it improves operations, response time, and the business overall.

There are integration challenges, particularly with performance when exporting data to BigQuery from other tools like Qualitics. It can be slower and more problematic compared to other platforms such as Snowflake.

I have been working with BigQuery for one year.

I have not seen a lot of problems, so I would say BigQuery is quite stable.

In my opinion, BigQuery is very scalable yet has some limitations regarding performance that are not always as required.

I don't have direct contact with BigQuery's support team. Our organization manages this through internal communication, and I contact my company’s team when issues arise.

Positive

I would recommend using BigQuery because it's a very good tool, easy to manage, and similar to other databases. Those familiar with SQL Server or Oracle can adapt to BigQuery easily. It's a scalable cloud solution. 

Overall, I would rate BigQuery as nine out of ten.

We basically used it to store server data and generate reports for enterprise architects. It was a valuable tool for our enterprise design architect.

Everything they advertised or listed worked exactly as promised. That was advantageous to us. 

In future releases, I would like to see more pre-defined aggregated forms. After using BigQuery, we need to use the data in an enterprise architecture dimensional data model. So, having pre-defined aggregated forms would be helpful. 

Additionally, I would like to see version-based implementation and a fallback arrangement for data stored in BigQuery storage. These are some features I'm interested in.

I have experience with BigQuery. 

When I joined the company, BigQuery was already implemented by our team.

It is a cheap solution. 

I would recommend getting a clear understanding of BigQuery's functionalities and what it's best suited for. If your needs align with its capabilities, then you should definitely proceed. 

BigQuery offers fantastic features, but it's important to understand its purpose beforehand. Otherwise, you might face difficulties later on.

Overall, I would rate the solution an eight out of ten.

We use BigQuery to perform data warehouse migration for clients willing to move to GCP from their on-premise solution.

The solution's pricing is really competitive compared to other peers. The most valuable features of BigQuery is that it supports standard SQL and provides good performance.

I have been using BigQuery for three years.

I rate BigQuery a nine out of ten for stability.

Around 30 to 40 users use BigQuery in our organization.

I rate BigQuery ten out of ten for scalability.

I previously worked with Microsoft SQL Server.

The solution’s initial setup is very easy. You just have to spin up a data set and start using it.

I rate BigQuery ten out of ten for the ease of its initial setup.

The solution can be deployed by one person in a few minutes.

The solution's pricing is cheaper compared to other solutions. On a scale from one to ten, where one is cheap, and ten is expensive, I rate the solution's pricing a two or three out of ten.

Potential users can trust BigQuery without any second thoughts. The solution's pricing is great compared to other solutions. The solution provides more flexibility and supports standard SQL, and anyone coming out from a different platform would not face any challenges adopting BigQuery.

Overall, I rate BigQuery a nine out of ten.

BigQuery is a PaaS solution. There's only one version available on Google Cloud. Because it's deployed on cloud, it will update automatically.

If I'm collaborating with Google Data Cloud, I can use the cache, and I don't have to pay again and again. There are some performance features like partitioning, which you can do based on an integer, and it improves the performance a lot. There's also the Array function. You can also enable Spark on BigQuery, which is actually faster than any other Spark. If you use Dataproc, Spark on BigQuery is much faster.

Spark will actually eliminate the usage of a lot of Adobe legacy things. It will act as a Spark SQL.

It is not that cost-friendly, but it is very performance-friendly. There are also machine learning features.

For example, if I have a query, and I have done everything to improve it, the query will still take 15 minutes. With other columnar databases like Snowflake, you can actually increase your VM size or increase your machine size, and you can buy more memory and it will start working faster, but that's not available in BigQuery. You have to actually open a ticket and then follow it up with Google support.

I have been using this solution for two and a half years.

BigQuery is very stable. It is getting used a lot.

It is definitely scalable. You do not have to do any configurations. It will be able to handle petabytes of data.

Technical support is excellent. It is Google, and they always provide the best. We haven't needed to contact Google for BigQuery specifically, but I have contacted Google support for other things and they were pretty responsive.

I have experience with Snowflake.

I was working on a project where we were building systems and loading the data manually. Once we moved to BigQuery, we saw ROI in terms of cost savings. We saw 27% performance improvement in most of our queries. Our total costs were reduced by about 17%. In terms of cost and time, we were able to save effort.

There was some learning and training involved, which lasted six months, so we saw the real ROI after a year.

I would rate this solution 8 out of 10.

My advice is to first identify your use case. If you have Google Cloud then you have two databases to compare, BigQuery and Snowflake. BigQuery is typically used to analyze petabytes of data. If you're looking for transitional query, then you should have a different system. BigQuery cannot handle unstructured data, so that is one thing you have to think about. 

In terms of latency, if you want single-digit millisecond latency then BigQuery is not good. It is very fast, but if you want single-digit millisecond latency, then you probably have to go to a no-SQL database solution.

My suggestion is to analyze your use case and then map it with the BigQuery features.