Azure Data Factory Reviews

Our company uses the solution to extract, transform, and load the data into the database.

Feature-wise, one of the most valuable ones is the data flows introduced recently in the solution.

Currently, our company requires a monitoring tool, and that isn't available in Azure Data Factory. Although the platform displays which pipelines are running, it doesn't offer a monitoring tool that allows for the sequential execution of pipelines and the ability to visualize end-to-end data flow. As such, this feature is currently missing from the platform.

I have been using Azure Data Factory for more than six years. Also, I am an end-user of the solution, and I initially used to work on Azure Data Factory V1. Now, I have switched to Azure Data Factory V2.

It is a stable solution. Stability-wise, I rate the solution an eight or nine out of ten.

Scalability-wise, I rate the solution a seven or eight out of ten. So, scalability can be improved. Also, there are around 150 people in my company using the solution. Moreover, we use the solution daily in our company.

I rate the technical support between eight to nine out of ten.

Positive

Previously, I was using Informatica. My company wanted to shift to a solution that could be deployed on the cloud, so we chose Azure Data Factory.

The solution's initial setup process was easy. On a scale where one is difficult and ten is easy, I rate the initial setup process an eight out of ten. The solution is deployed on the cloud.

Since multiple projects are going on in my organization, there is no uniformity in the time taken to deploy the solution in our company. However, I can say that it only takes a few days to carry out the deployment process.

Our organization has multiple project teams, so each team carries out its deployment process.

To give an average, I would consider that if there are fifty ongoing projects in our company, and if we consider a person from each project, fifty people are needed for the deployment and maintenance process.

The solution's implementation process was done with our in-house team's help.

I cannot comment on the pricing parts since our company's admin team handles it.

Those who want to move to a cloud platform can choose Azure Data Factory since it is the best tool. Since certain improvements are required in the solution, I rate the overall solution an eight out of ten.

We primarily use the solution in a data engineering context for bringing data from source to sink.

The solution is very comfortable to use. I'm happy with the user interface and dashboards. I'm pretty happy with everything about the solution. 

We haven't had any issues connecting it to other products.

It's a stable product. 

I have not found any real shortcomings within the product.

I've been using the solution for the past year. 

The product has been very stable and reliable. I'd rate the stability nine out of ten. There are no bugs or glitches. It doesn't crash or freeze. 

There is a team of 30 people working on the solution. 

I've connected with technical support a few times. 

They sent a support engineer or a field engineer to us, and he helped us out. 

Positive

I'm not sure about the exact cost of the solution. 

I'm a customer and end-user.

Our company chose to use this solution based on the fact that it is a Microsoft product. We're using a lot of solutions, including Outlook and Teams. We also use Power BI. We try to use Microsoft so that everything is under one umbrella. That way, there is no difficulty with connecting anything. 

It's a good solution to use. There are lots of videos available on YouTube, and it is very easy to learn. It's very easy to perform things on it as well, which is one thing that a product like ThoughtSpot lacks. There is no training needed like Power BI. 

I'd rate the solution nine out of ten.

I can't go into specifics about the use case for Azure Data Factory, but it's for analytics related to an assessment.

What I like best about Azure Data Factory is that it allows you to create pipelines, specifically ETL pipelines.

I also like that Azure Data Factory has connectors and solves most of my company's problems. I can't recall a case where I couldn't use the solution for solving problems.

I'm also happy about the Azure Data Factory dashboard.

A room for improvement in Azure Data Factory is its speed. Parallelization also needs improvement. As for the rest of the features of Azure Data Factory, I'm happy.

I cannot suggest an additional feature I'd like to see in Azure Data Factory in the future because some of the features aren't available internally because the features undergo security evaluation first, and my organization controls which features would become available to users.

I've been using Azure Data Factory for the last two years.

We're happy with the stability of Azure Data Factory.

Azure Data Factory is scalable, with clusters available on demand. There isn't any issue with scaling the solution.

We have an internal support team and the Azure Data Factory support team. We raise tickets and follow up on those tickets, and on a scale of one to five, we'd rate support as four because sometimes there are delays. Otherwise, we are satisfied with Azure Data Factory support.

My company didn't set up Azure Data Factory as the Azure team did it.

We outsourced the implementation of Azure Data Factory directly to the Azure team.

I have no idea how much Azure Data Factory costs.

We're using AWS apart from Azure Data Factory. We're trying out Palantir Foundry as well. They are the leading service providers in the data analytics and ETL world.

I'm familiar with Palantir Foundry, but my company just recently got the Palantir Foundry license, so I'm still not using it, but checking it for shortcomings.

I have experience with Azure Data Factory, too.

I'm unsure of the exact version of Azure Data Factory, but I'm using the latest version or whatever's available on Azure.

I have a vague figure of users of Azure Data Factory, but it's more than one thousand to one thousand five hundred people.

I'd tell people who want to use Azure Data Factory that Microsoft offers excellent courses, ESI (Enterprise Skill Initiatives). You should register and take the courses. Azure Data Factory is a solution I'd recommend to others.

I'd rate Azure Data Factory as nine out of ten because it has a lot of connectors, even custom connectors, for data onboarding. It can also integrate with Spark notebooks and allows my organization to parallelize code. Azure Data Factory also has provisions for Spark and SQL scripts or any scripts, plus the infrastructure is highly scalable, so it's a nine for me.

My organization is a customer of Azure Data Factory.

Our primary use case for the solution is data integration and we deploy it only on Azure.

When we were integrating the Ports product with our internal data warehouse, we had to update all the reports to our internal data warehouse on the Ports system database. However, they were not given access to the database company, and they dump some files or provide you with them. In one case, they were providing files. In another case, they provided some APIs where you need to call in a batch of thousands of records multiple times. It works very well with Azure Data Factory to pull the records, parse them quickly and post them in the database and data warehouse.

We have found the bulk load feature very valuable.

The only challenge with Azure Data Factory is its exception-handling mechanism. When the record fails, it's tough to identify and log.

We have been using the solution for a year and a half and are currently using the latest version.

The solution is scalable and we intend to further increase its usage in the future.

I rate customer service and support an eight out of ten.

Positive

We previously used different solutions.

The initial setup is straightforward.

The implementation was done in-house.

I cannot comment on licensing costs because I was not involved.

I rate the solution a six out of ten. The solution is good but its exception handling and logging mechanisms can be improved. I advice users considering this solution to go for it especially if their integrations are heavy on the data side.

We use this solution to ingest data from one of the source systems from SAP. From the SAP HANA view, we push data to our data pond and ingest it into our data warehouse.

Azure Data Factory didn't bring a lot of good when we were also using Alteryx. Alteryx is user-friendly, while Azure Data Factory uses many resources and has issues with parallel workflows. Alteryx helps you diagnose issues quicker than Azure Data Factory because it's on the cloud and has a cold start debugger.

Azure Data Factory has to wake up whenever you are trying to do testing, and it takes about four to five minutes. It's not always online to do a quick test. For example, if we want to test an Excel file to see if the formatting is correct or why the data-flow or pipeline is failing, we need to wait four to five minutes to get the cold start debugger to run. Compared to Alteryx, Azure Data Factory could be better. Nevertheless, we are using it because we have to.

Initially, when we started using it, we didn't like it because it needed to be more mature and had data-flows, so we used the traditional pipeline. After that, Azure Data Factory introduced the concept of data-flows, and it started to become more mature and look more like Alteryx. Azure Data Factory became more user-friendly when data-flows were introduced.

They introduced the concept of Flowlets, but it has bugs. Flowlets are a reusable component that allows you to create data-flows. We can configure a Flowlet as a reusable pipeline and plug it inside different data-flows, so we don't have to rewrite our code or visual transformation.

If we make any changes in our data-flow, it reverts all our changes to the original state of the Flowlet. It does not retain changes, and we must reconfigure the Flowlets repeatedly. We had these issues three months ago so things might have changed. It works fine whenever we plug it in and configure it in our data-flow, but if we make minor changes to it, the Flowlet needs to be reconfigured again and loses the configuration.

We have used this solution for about a month and a half. It is a cloud-based tool, so there are no versions. It is all deployed on Azure Cloud.

Everything is computed inside the SQL server if we're working with pipelines, so we have to be very careful when designing our solution in Azure Data Factory. Alteryx spoiled us because we never cared how it looked in the backend because all the operations were happening on the Alteryx server. But in Azure Data Factory, they run on the capacity of our data warehouse. So Azure Data Factory cannot run your queries, and it directly sends the query to the instance in the SQL server or data warehouse. So we have to be very careful about how we perform certain operations.

We need to have knowledge of SQL and how to optimize our queries. If we are calling a stored procedure, it joins one table in Alteryx. It is pretty easy, and we just put a joint tool. Suppose we want to do it with a stored procedure in the Azure Data Factory. In that case, we have to be very careful about how we write our code. So that is a challenge for our team because we were not looking into how to optimize their SQL queries when fighting queries from Azure Data Factory to the data warehouse.

In addition, the workflows were running very slow, the performance was bad, and some queries were getting timed out because we have a threshold. So we faced many challenges and had to reeducate ourselves on SQL and query optimization.

In regards to scaling, when Azure Data Factory was introduced as your Databricks, it worked similarly to Hadoop or Spark, and it had some Spark clusters in the back end that could scale it as much as it could, and speed up the performance. So it is scalable, especially with Databricks, because a lot of data-related transformations can be performed.

On my team, there are approximately 20 people who work with Azure Data Factory.

We do not have experience with customer service and support.

It does not require any installation and is more like software as a service. You need to create an instance of Azure Data Factory in Azure and configure some of the connections to your databases. You can connect to your block storages and some authentication is necessary for Azure Data Factory.

The setup is straightforward. It doesn't take much time, and it's on cloud. It requires a few clicks, and you can quickly set it up and grant access to the developer. Then the developer can go to the link and start developing within their browser.

We have a team that takes care of the cloud infrastructure, so we raise a ticket and request infrastructure, and they just exceed it based on the naming convention with the project name.

We have an entire team that takes care of the cloud infrastructure. So we raise a ticket when we need infrastructure, which is executed based on the naming convention for the project name.

The nature of our solution is not based on ROI because we are building solutions for other functions within the same organization. In addition, due to the large size of our organization and the services we provide, the ROI is not something we consistently track. It's something discussed with the management, so I can't comment on it.

The cost is based on usage and the computing resources consumed. However, since Azure Data Factory connects with so many different functionalities that Azure provides, such as Azure functions, Logic apps and others in the Azure Data Factory pipelines, additional costs can be acquired by using other tools.

We did not evaluate other options because this solution was aligned with out current work environment. 

I rate the solution a seven out of ten. The solution is good and constantly improving, but the concept of Flowlets can be reconfigured to retain the changes we make. I advise users considering this solution to thoroughly understand what Azure Data Factory is and evaluate what's available in the market. Secondly, to assess the nature of the use cases and the kind of products they will be building before deciding to choose a solution.

The primary use case of this solution is to extract ETLS, transform and load data, and organize database synchronization.

The most valuable feature of this solution is the data flow, which is the same SQL server in important service, integration services, which is a very robust and powerful tool to transform data.

The solution can be improved by decreasing the warmup time which currently can take up to five minutes.

I have been using the solution for two years.

The solution is extremely stable.

The solution is scalable.

Previously I used AWS Glue and SSIS.

The initial setup is straightforward.

The implementation was completed in-house and is immediate because it is a native cloud tool.

I have seen an ROI with the time saved migrating data for reports.

The solution's fees are based on a pay-per-minute use plus the amount of data required to process. The more data you process the more CPUs and time is required which increases the cost of using this solution.

I give the solution ten out of ten.

The only thing you need to deploy the solution is to click on publish.

The solution includes a feature that increases the number of processors used which makes it very powerful and adds to the scalability.

We have three people using the solution in our organization and one engineer that maintains it.

I recommend to any potential user to factor in the five-minute warm-up time that is required for each execution.

We use Azure Data Factory for data transformation, normalization, bulk uploads, data stores, and other ETL-related tasks.

Azure Data Factory allows us to create data analytic stores in a secure manner, run machine learning on our data, and easily adapt to changing schema.

The most valuable feature of Azure Data Factory is that it has a good combination of flexibility, fine-tuning, automation, and good monitoring.

The documentation could be improved. They require more detailed error reporting, data normalization tools, easier connectivity to other services, more data services, and greater compatibility with other commonly used schemas.

I would like to see a better understanding of other common schemas, as well as a simplification of some of the more complex data normalization and standardization issues.

It would be helpful to have visibility, or better debugging, and see parts of the process as they cycle through, to get a better sense of what is and isn't working.

It's essentially just a black box. There is some monitoring that can be done, but when something goes wrong, even simple fixes are difficult to troubleshoot.

I have been working with Azure Data Factory for a couple of years.

There is only one version.

Overall, I believe the stability has been good, but there have been a couple of occasions when Microsoft's resources needed to be allocated were overburdened, and we had to wait for unacceptable amounts of time to get our slot. It has now happened twice which is not ideal.

There is no limit to scalability.

We only have a few users. One is a data scientist, and the other is a data analyst.

We use it to push up various dashboards and reports, it's a transitional product for transferring, transforming, and transitioning data.

It is extensively used, and we intend to expand our use.

You don't really get that kind of support; it's more about documentation and the community support that is available. I would rate it a three out of five compared to others.

You could call them, and pay for their consulting hours directly, but for the most part, we try to figure it out or look through documentation. 

I think their documentation is lagging because it's not as popular of a tool, there's just not a lot, or as much to fall back on.

Neutral

We had only our own tools, and we switched because you get to leverage all of the work done in a SaaS or platform as a service, or however they classify it. As a result, you get more functionality, faster, for less money.

The initial setup is straightforward.

It is a working tool. You can start using it within an hour and then make changes as needed.

We only need one person to maintain the solution; it doesn't take much to keep it running.

It's not a problem; it's a platform.

We completed the deployment ourselves.

We have seen a return on investment. I can't really share many details, but for us, this becomes something that we sell back to our clients.

You pay based on your workload. Depending on how much data you process through it, the cost could range from a few hundred dollars to tens of thousands of dollars.

Pricing is comparable, it's somewhere in the middle.

There are no additional fees to the standard licensing fee.

We looked at some other tools, such as Databricks, AmazonGlue, and MuleSoft.

We already had most of our infrastructure connected to Azure in some way. So the integration of where our data resided appeared to be simpler and safer.

I believe it would be beneficial if they could find someone experienced in some of the tools that are a part of this, such as Spark, not necessarily Data Factory specifically, but some of those other tools that will be very familiar and have a very quick time for productivity. If you're used to doing things in a different way, it may take some time because there isn't as much documentation and community support as there is for some more popular tools.

I would rate Azure Data Factory a seven out of ten.

For developers that are very accustomed to the Microsoft development studio, it's very easy for them to complete end-to-end data integration.

One of the features that still is in development is data privacy to the cloud side of the SAP integration.

I have been using Azure Data Factory for 3 years. 

I rate the stability a 9 out of 10. 

6 developers are using the solution at present. 

The initial setup is not very straightforward. I rate it a seven out of ten. 

The pricing is a bit on the higher end. 

Overall, I rate the solution an 8 out of 10.