Spark SQL Reviews

The primary use case of this solution is to function within a distributed ecosystem. Spark is part of EMR, a Hadoop distribution, and is one of the tools in the ecosystem. You are not working with Hadoop in a vacuum—you leverage Spark, Hive, HBase—because it is just a distributed ecosystem. It has no value within itself. 

This solution can be deployed both on the cloud and on Cloudera distributions. 

This solution is useful to leverage within a distributed ecosystem. 

This solution could be improved by adding monitoring and integration for the EMR. 

We have been working with Spark SQL for a few years. We are an outsourcing and consulting company, so it's not for our use—we mostly work with clients. 

This solution is stable. 

This solution is scalable. 

The installation is straightforward because it's a cloud-based solution. 

We implement this solution for customers ourselves. 

There is no license or subscription for this solution. 

I rate this solution an eight out of ten and would recommend it to others. 

I am using this solution for data validation and writing queries.

Data validation and ease of use are the most valuable features.

There should be better integration with other solutions.

I have been using the solution for approximately two years.

The solution has been stable.

I have found the solution to be scalable. We have 20 people using the solution in my organization and we plan to increase usage.

The solution is open-sourced and free.

I rate Spark SQL a ten out of ten.

We use it to gather all the transaction data. We have Hadoop and Spark in our system, and we use some easy process flows for transport. 

It is a stable solution. 

Being a new user, I am not able to find out how to partition it correctly. I probably need more information or knowledge. In other database solutions, you can easily optimize all partitions. I haven't found a quicker way to do that in Spark SQL. It would be good if you don't need a partition here, and the system automatically partitions in the best way. They can also provide more educational resources for new users.

I have been using this solution for two months.

Its scalability is okay. We are a big organization. 

Being a new user, I would rate Spark SQL a four out of ten. 

Our primary use case is for building a data pipeline and data analytics. 

The performance is one of the most important features. It has an API to process the data in a functional manner. 

I would like to have the ability to process data without the overhead. To use the same API to process both terabytes data and be able to process one GB of data. 

I have been using Spark SQL for around four years. 

It is very stable.

It is scalable. I use it on and off. I use it mostly daily. 

From an infrastructure perspective, it was easy for us to set up because we used some cloud services. But on-premise requires more setup. There is a learning curve. If you're not a programmer there is a learning curve. It requires more effort to learn more complex steps. 

I deployed it by myself. We use cloud so we are able to do it. 

The amount of people required for deployment will depend. One person is enough for AWS but not in other places. 

If you know how to do it, the deployment can be done in minutes. 

I would rate Spark SQL a nine out of ten. 

My advice would be to read Databricks books about Spark. It's a good source of knowledge. 

In the next update, we'd like to see better performance for small points of data. It is possible but there are better tools that are faster and cheaper. 

We do have some use cases, like analysis and risk-based use cases, that we've provided and prepared for companies in order to evaluate, but not many. The business units have so many things that we don't know how to help formulate into another tool and utilize as a use case. They also have so many requirements and costs.

I work for a financial institution, so every solution that they need to consider has to be on-premise.

I'm actually just evaluating and up scaling my skill sets with this solution right now.

The speed of getting data, as our TBs are big and it's a lot of data. 

Anything to improve the GUI would be helpful.

We have experienced a lot of issues, but nothing in the production environment.

For a couple of months. However, we have not implemented in a production environment yet.

The solution has not been implemented yet. When it is implemented into the real world and production, that is when I expect to see some challenges.

We have worked with the Cloudera support for this solution. They are average.

I have an experience with other database tools for the span of more than 10 years.

The initial setup is a bit complex.

We are also planning to use Informatica since there is a way in which you can use Spark in Informatica. You can use the Spark within Informatica because there is an an option to tie in a big data addition.

We will have a lot of big data, which is why we need it. Otherwise, the solution is not needed. The solution really depends on the size of your data, its complexity, and the analysis that you are doing. Spark is good, but it is not mandatory.

Since I don't have experience in production with the solution, the best I can rate it now is a five (out of 10). 

We primarily use the solution as our data warehouse. We use it for data science.

Overall the solution is excellent.

The solution is continuing to evolve and mature over time.

The service is complex. This is due to the fact that it's a combination of a lot of technology.

The solution needs to include graphing capabilities. Including financial charts would help improve everything overall.

I've been using the solution since 2013.

The solution is relatively stable. We haven't had issues with bugs or glitches.

The solution is scalable. We've found it easy to expand as necessary.

Right now, we have about 42 users on the solution. They include IT and ETL staff as well as a business analyst.

I've never been in touch with technical support. I can't speak to any experience our company has had with them.

I've also worked with Apache SQL and SAP. This solution is a much more scalable and adventurous solution. It's also faster than the others. We used to use IQ, but at the time it couldn't scale well, so we switched to IBM Appliance. Then we switched to Spark. IBM was good, but it also had issues with scalability and it cost us a lot of money. 

The initial setup is straightforward. We found it quite easy.

We handled the implementation ourselves with our in-house team.

The pricing of Apache is much more competitive than IBM.

We use both the on-premises and cloud deployment models.

We have a relationship with Cloudera and use their distribution channels. We don't have a relationship with Apache.

Spark SQL is a good product. However, users need to have the capability of implementing the correct tools and efficiencies.

I'd rate the solution seven out of ten.

The primary use is to process big data. We were connecting into and we were applying sentiment analysis via hardware.

In the next release, maybe the visualization of some command-line features could be added.

The stability was fine. It behaved as expected.

The scalability of the solution is good.

Technical support has been fine.

We previously used Apache Hadoop.

The initial setup was fine. If somebody knows what to expect it's okay.

We've just started using this solution. We were using it until recently on a research basis, just to measure the performance, the cost, and so on and so forth. Many things could be improved, but are okay up till now, I'm happy with. I would recommend the product.

I would rate this solution eight out of ten.