Databricks Reviews

Our company uses the solution's Spark module for big data analytics as a  processing engine.  

We do not use the module as a streaming engine. The historic perception is that Spark is for batches, machine learning, analytics, and big data processing but not for streaming and that is exactly how we use it. 

The solution's features are fantastic and include interactive clusters that perform at top speed when compared to other solutions.

The ATC monitoring experience and the maturity of the APIs are very good. 

CI/CD needs additional leverage and support. Community forums are helpful for gaining knowledge but the solution should provide specific documentation.

Streaming services such as Flink should be amplified and better supported. 

There are not many connectors to MapReduce.

I have been using the solution for seven years. 

The solution is mature and stable compared to other products. 

The solution is scalable with no issues from a computer perspective.

I received support for initial challenges and it was very good. The support team was very professional and provided the answers I needed. 

I rate support an eight out of ten. 

Positive

I previously used Cloud-Bricks. 

The initial setup is easy for me because I access the solution on a web browser. 

Unilever had a specific team for implementing and managing the solution.

Walmart had a team of ten engineers for implementation and a couple of engineers for management. 

We receive an ROI for our batch constructs. 

The solution is a good value for batch processing and huge workloads. 

The price might be high for use cases that are for streaming or strictly data science. 

I have evaluated multiple options including Cloud-Brick and Dataproc for price versus performance, technical support, and CI/CD approach.

I started as a consumer and used the solution for on-premises deployment with Unilever from a data science perspective. At that time, the solution was in its beta stage but viewed as good, far ahead of its competition, and expensive. The key comparison used to be HDInsight or Adobe Cluster for cloud data and the solution was thought of as a cluster service rather than for unified analytics.

I moved along on my journey to Walmart where I was building their platform and compared it to the solution from a cloud perspective and a cluster service with notebooks. Consumers at the time were using Project Lightspeed and ATC for streaming. Spark was used as a micro-batching engine for machine learning, analytics, and big data processing. At some point, the solution became preferred and more than 100 staff members were leveraging its use.

I found that the solution had interesting features that I liked such as its notebook, interactive clusters with fast speed, and the ATC monitoring experience. I did not like the solution from a CI/CD perspective because it had a rigidity in terms of the approval process.

The solution grew from that original space and, by the time I had moved to Microsoft, was partnered with Microsoft Azure. An integration with ADF and other products solved the CI/CD issues for me.

I am now leading streaming platforms for Walmart so my interest is in the solution's streaming capabilities. I began building a streaming platform using Spark PM in Microsoft so the solution was its key competitor. Then the solution launched a vectorized machine on Photon for the Spark engine. Its performance was a key factor in moving from Microsoft because it performed much better than other products including opensource Spark, Microsoft Synapse Spark, and Dataproc.

It is important to do POCs and run tests to control the meter that also controls the price. The meter can go really high from a computing perspective if POCs and settings are not streamlined. 

I rate the solution an eight out of ten. 

We're using it to provide a unified development experience for all our data experts, including all data engineers, data scientists, and IT engineers. With the Databrick Platform we allows teams to collaborate easily towards building Data Science models for our clients. The development environment allows us to ingest data from various data sources, scale the data processing and expose them either trough API or through enriched datasets made available to web app or dashboard leveraging the serverless capacities of SQL warehouse endpoints.

Databricks allowed us to offer an homogeneous development environment accross different accounts and domains, and also across different clouds. The upskilling of our employees is far more linear and faster, while removing the complexity of infrastructure management. This lead to an increased collaboration between domain thanks to a better onboarding experience, more performant pipelines and a smoother industrialization process. Overall client satisfaction has increased and the time to first insight has been reduced.

The shared experience of collaborative notebooks is probably the most useful aspect since, as an expert, it allows me to help my juniors debug their books and their code live. I can do some live coding with them or help them find the errors very efficiently.

It has become very simple to set up thanks to its official Terraform provider and the open-source modules made available on GitHub.

I love Databricks due to the fact that we can now deploy it in 15 minutes and it's ready to use. That's very nice since we often help our clients in deploying their first Data Platform with Databricks.

The solution is stable, with LTS Runtimes that have proven to remain stable over the years. 

I would love to be able to declare my workflows as-code, in an Airflow-like way. This would help creating more robust ingestion python modules we can test, share and update within the company. 

We would also love to have access to cluster metrics in a programmatic way, so that we can analyse hardware logs and identify potential bottlenecks to optimize.

Lastly, the latest VS Code extension has proven to be useful and appreciated by the community, as it allows to develop locally and benefits from traditional software best-practices tools like pre-commits for example.

I've been using the solution for more than four years now, in the context of PoC to full end-to-end Data Platform deployment.

The product is very stable. I've been using it for three years now, and I have projects that have been running for three years without any big issues.

It's very scalable. I have a project that started as a proof of concept on connected cars. We had 100 cars to track at first - just for the proof of concept. Now we have millions of cars that are being tracked. It scales very well. We have terabytes of data every day and it doesn't even flinch.

I've had very good experiences with technical support where they answer me in a couple of hours. Sometimes it takes a bit longer. It's usually a matter of days, so it's very good overall. 

Even if it took a bit of time, I got my answer. They never left me without an answer or a solution.

Positive

The implementation is very simple to set up. That's why we choose it over many other tools. Its Terraform provider is our way-to-go for the initial setup has we are reusing templates to get a functional workspace in minutes.

Usually, we have two to five data engineers handling the maintenance and running of our solutions.

We deploy it in-house.

The solution is a bit expensive. That said, it's worth it. I see it as an Apple product. For example, the iPhone is very expensive, yet you get what you pay for.

The cost depends on the size of your data. If you have lots of data, it's going to be more expensive since your paper compute units will be more. My smallest project is around a hundred euros, and my most expensive is just under a thousand euros a week. That is based on terabytes of data processed each month.

We looked into Azure Synapse as an alternative, as well as Azure ML and Vertex on GCP. Vertex AI would be the main alternative.

Some people consider Snowflake a competitor; however, we can't deploy Snowflake ourselves just like we deploy Databricks ourselves. We use that as an advantage when we sell Databricks to our clients. We say, "If you go with us, we are going to deploy Databricks in your environment in 15 minutes," and they really like it.

Lately Fabric was released and can offer quite a similar product as Databricks. Yet, the user experience, the CI/CD capabilities and the frequent release cycle of Databricks remains a strong advantage.

We're a partner.

We use the solution on various clouds. Mostly it is Aure. However, we also have Google and AWS as well. 

One of the big advantages is that it works across domains. I'm responsible for a data engineering team. However, I work on the same platform with data scientists, and I'm very close to my IT team, who is in charge of the data access and data access control, and they can manage all the accesses from one point to all the data assets. It's very useful for me as a data engineer. I'm sure that my IT director would say it's very useful for him too. They managed to build a solution that can very easily cross responsibilities. It unifies all the challenges in one place and solves them all mostly.

I'd rate the solution nine out of ten.

We use the solution for data analytics of industrial data.

We extensively use the product’s notebooks, jobs, and triggers. We can create activities. Wherever translation is required, we use Databricks. The product fulfills our customer requirements. It is a cost-effective solution.

The product should provide more advanced features in future releases.

I have been using the solution for six months.

Our data was not too huge. It worked well. It is easily adaptable.

The tool is scalable. We can make it available for a larger audience.

The initial setup is not that difficult. I rate the ease of setup a seven out of ten. The solution is cloud-based. We use native services like Data Factory for orchestration. Sometimes, the customers require us to use Amazon as the cloud provider instead of Azure.

The pricing is average.

There are many services which are coming up. They are still in the preview stage. Overall, I rate the product an eight out of ten.

We use it for data analysis and testing of high volume web user behavioral data.

Databricks makes it really easy to use a number of technologies to do data analysis. In terms of languages, we can use Scala, Python, and SQL. Databricks enables you to run very large queries, at a massive scale, within really good timeframes.

I'm starting to build a solution using Delta Live Tables and Delta Live pipelines, and it is proving to be exceptionally easy to use. I have also been able to quickly implement a pipeline.

Databricks has added some alerts and query functionality into their SQL persona, but the whole SQL persona, which is like a role, needs a lot of development. The alerts are not very flexible, and the query interface itself is not as polished as the notebook interface that is used through the data science and machine learning persona. It is clunky at present.

I've been using Databricks for a year.

It is a stable and reliable solution. I'd rate stability at eight out of ten.

Databricks is absolutely scalable, and I'd rate scalability at eight out of ten. We probably have between 60 and 100 users in our organization, and we hope to increase usage in the future.

The technical support staff we have worked with have been amazing. They helped us initially with our Delta Live pipelines. I would give them a rating of ten out of ten.

Positive

I have previously worked with Apache Hadoop, and Databricks is definitely a better product. It's much easier to get data quickly in Databricks. As a result, a lot of the drudgery is taken away. Whereas with Hadoop, it's a bit more tricky to get data together.

We're charged on what the data throughput is and also what the compute time is.

I'd strongly recommend giving Databricks a try. We have found it to be a fantastic tool that has accelerated some of our solutions. We're an AI-heavy shop, and there are a lot of data scientists using the MLflow capabilities. I hear a lot of good things from that side as well. From a data analysis point of view, Databricks has been fantastic, and I would rate it at eight on a scale from one to ten.

We mainly use Databricks to process ingest and do the ELT processes of data to get it ready for analytics and to serve the data to ThoughtSpot, which calls queries and Databricks to get the data.

We didn't have any good tooling for ELT processing prior to Databricks. We were using Microsoft HD Insight, but it was taking too long to process the data. When we changed our data-processing ELT processes over to Databricks, the amount of time to process the data was reduced to a fraction of what HD Insight used, so we were able to run jobs much faster.

Databricks' most valuable feature is the data transformation through PySpark.

Databricks' performance when serving the data to an analytics tool isn't as good as Snowflake's. In the next release, Databricks should include a better data-sharing platform to facilitate data sharing between companies.

I've been using Databricks for three years.

Databricks' stability has been great, and I would rate it eight out of ten.

Databricks is very scalable because it's very easy to spin up multiple clusters, but the cost of doing that is tremendous. I'd rate its scalability nine out of ten, but you'll pay for it.

The technical support has been really bad, but that's because we don't have a direct agreement with Databricks.

Neutral

I previously used HD Insight from Microsoft, but it took many, many hours to process data, so we switched to Databricks.

The initial setup was pretty complex and required three people.

We used an in-house team with some consulting help.

We've had a low ROI from Databricks.

I would rate Databricks' pricing seven out of ten.

I would advise anyone thinking of implementing Databricks to know their use case. For example, if you're looking for a big data repository to query data and do ELT processing, I recommend looking at other platforms, like Snowflake. However, if you're going to do AI and machine learning, then Databricks is probably stronger in that area. Overall, I would rate Databricks seven out of ten.

Our primary use case is for data analytics. Essentially, we use it for the financial reporting for Adobe.

The way Databricks has improved my organization is definitely through giving us improved performance and the processing power. We are usually never able to achieve it using traditional data warehouses. When we have a huge volume of data that we want to process with speed, velocity, and volume, we go through Databricks.

The features I found most helpful with Databricks are the Lakehouse and SQL environments.

I believe that this product could be improved by becoming more user-friendly. 

In the next release, I would like to see more flexibility in the dashboard. It has plenty of features but it can be enhanced so that it matches with other visualization tools, like Power BI and Tableau. Also, the integrations with other tools could be better.

I have been using Databricks for the last three years.

I would rate the stability of Databricks an eight, on a scale from one to 10, with one being the worst and 10 being the best.

I would rate the scalability of this solution a nine, on a scale from one to 10, with one being the worst and 10 being the best. I would say there are around 2,000 to 3,000 users of this solution in our organization.

I've been in contact with the Databricks support team and received timely support from them. I would rate their support an eight, on a scale from one to 10, with one being the worst and 10 being the best.

Positive

Prior to Databricks, we initially used Hadoop. Afterwards, we used HANA, SAP HANA, and the Microsoft SQL Server.

The initial setup was relatively straightforward. I would rate it nine, on a scale from one to 10, with one being the easiest and 10 being the hardest.

There is no need to worry about the deployment as it can be done quickly. It is relatively automated. We used Terraform for auto-deployment, which happens in Azure. With Terraform, there are two options. As option one, you can deploy manually by creating services. For option two, you use Terraform and automate. Terraform is like infrastructure as a code where you can code the deployment part using it.

There were two or three persons involved in the deployment of this solution.

The new version of the Databricks solution requires code maintenance. This is done by the platform team.

We use this solution for advanced civilization power.

We like that this solution can handle a wide variety and velocity of data engineering, either in batch mode or real-time.

This product allows us to write the email models in a way that allows us to take the advantage of the parallel scaling computer window backend on any of the satellite services.

This solution only supports queries in SQL and Python, which is a bit limiting. 

This is a fairly expensive solution for any service outside of the basic package, and costs can add up quite quickly if there are large scaling requirements.

This is a stable solution in our experience.

We have found that part of the beauty of this platform is that it is easy to scale and expand.

The support for this product uses Microsoft as a middle man, and due to this there have been times when we experienced communication delays, as well as misunderstandings of what our issues are.

Neutral

The initial setup for this solution is very simple.

The basic version of this solution is now open-source, so there are no license costs involved. However, there is a charge for any advanced functionality and this can be quite expensive.

We looked at both Snowflake and BigQuery as a comparison with this solution. We choose this product as it offered more scalability and a higher level of security, which is extremely important in our banking environment.

We would rate this solution an eight out of ten.

We use the solution for data engineering. 

The tool helps us manage large amounts of data. 

The simplicity of development is the most valuable feature. 

Databricks has a lack of debuggers, and it would be good to see more components. 

Another issue is that the D4 data format keeps changing on our cluster. This doesn't affect me much because I use functions to define it, but it is very frustrating for some more casual users. One day the output will be in a particular format, and then it becomes an object without us changing the cluster configuration. As a small team, we don't have the capacity to dig deeply into the issue, which has been frustrating.

We have been using the solution for three years. 

The solution's stability is good. 

The product is scalable. We're a small organization with 12 users, and we don't currently have any plans to increase our usage.

We see an ROI from Databricks. 

I would rate the solution seven out of ten. 

It's a good solution and more for handling large amounts of data. Databricks is better as a batch processing system than as an interactive system. The performance is a little disappointing because the memory processing is supposed to be excellent, but it's not as competitive as some other solutions out there in this regard. Even classical databases can respond and process faster.