We were using Dremio as a data lake query engine tool. We were creating our PDSs and VDSs on top of our S3 buckets, and our data lake and the data-scientist teams were using the data for further processing. We didn't use it for any ETL jobs. We were using it as a data-lake tool.
The most valuable feature of Dremio is it can sit on top of any other data storage, such as Amazon S3, Azure Data Factory, SGFS, or Hive. The memory competition is good. If you are running any kind of materialized view, you'd be running in memory.
Dremio takes a long time to execute large queries or the executing of correlated queries or nested queries. Additionally, the solution could improve if we could read data from the streaming pipelines or if it allowed us to create the ETL pipeline directly on top of it, similar to Snowflake.
I have been using Dremio for approximately two years.
Dremio is highly stable.
Dremio is scalable which is a benefit of the solution. You can scale up to the number of instances you want in case you are feeling the load, and in case you feel your query is running low or you are receiving extra traffic.
You can set the configuration while installing and while setting it up in the Dremio. In the configuration file, you can set up a lot of settings, such as what time.
We have approximately 18 people using the solution in my organization.
We have been in touch with the support from Dremio when we had some internal issues. This happened approximately two times. The support is good.
This is the first tool in this category that I have used.
Dremio's initial setup took one or two days, one day is sufficient and typical.
There was one DevOps person used for the deployment and maintenance of the solution.
Dremio is less costly competitively to Snowflake or any other tool.
My advice to others is if they are creating a data lake for a customer, Dremio would be useful for a data engineering team. If they're willing to create a data lake and they wanted to use it with the cloud-agnostic tool, then it is a good choice. If this solution meets their requirement they should try it out.
I rate Dremio an eight out of ten.
