Azure Data Factory Reviews

I mainly use Data Factory to load data for ETL processes or to Azure Storage and for testing purposes in our business unit.

Data Factory's most valuable feature is Copy Activity.

I've been using Data Factory for around two years.

Data Factory is stable.

We've had no problems with Data Factory's scalability.

Microsoft's technical support is responsive and quick to help.

We used consultants to implement Data Factory.

Data Factory's cost is too high.

I would advise anybody thinking of implementing Data Factory to calculate their costs at the initial stage in order to have some knowledge about future costs for the whole project. I would rate Data Factory as eight out of ten.

We created data ingestion solutions. We have built interpreters, and we have data factories that pull data from our clients. They submit data via Excel spreadsheets, and we process them into a common homogeneous format.

It has helped with some automation. Instead of individual people reviewing these files, we were able to automate the ingestion process, which saved a bunch of time. It saved hours of repeated manual work.

The data mapping and the ability to systematically derive data are nice features. It worked really well for the solution we had. It is visual, and it did the transformation as we wanted.

I couldn't quite grasp it at first because it has a Microsoft footprint on it. Some of the nomenclature around sync and other things is based on how SSRS or SSIS works, which works fine if you know these products. I didn't know them. So, some of the language and some of the settings were obtuse for me to use. It could be a little difficult if you're coming from the Java or AWS platform, but if you are coming from a Microsoft background, it would be very familiar.

For some of the data, there were some issues with data mapping. Some of the error messages were a little bit foggy. There could be more of a quick start guide or some inline examples. The documentation could be better.

There were some latency and performance issues. The processing time took slightly longer than I was hoping for. I wasn't sure if that was a licensing issue or construction of how we did the product. It wasn't super clear to me why and how those occurred. There was think time between steps. I am not sure if they can reduce the latency there. 

I have been using this solution for a year and a half.

It is very stable.

It is very scalable. It is a cloud product. It is being used by business analysts, business managers, and Azure cloud architects. We have just one developer/integrator for deployment and maintenance purposes.

We have plans to increase its usage. We'll be rolling it out for other clients.

Microsoft has these things well-documented. There were videos. I was able to find answers when I needed them. To the uninitiated, it was a little difficult, but we got there.

It was of medium complexity. Because it goes to the cloud, the duration was short. The deployment was minutes and hours.

We are a consultant and integrator. You can use our company for its implementation.

I would rate this solution a nine out of ten.

It's a PaaS service. It's a hybrid solution. The cloud provider is Microsoft.

We are not using Azure Data Factory as for users. Rather, we're using it as a process base. We're just using it for orchestration, not for any kind of ETL stuff.

We have plans to increase usage. It's going to take a major role in any kind of traditional data warehousing. It has big potential, especially as a PaaS offering.

There has been improvement in data resilience, in the way that we're moving the data from on-prem to cloud and vice versa.

The most important feature is that it can help you do the multi-threading concepts. It's in Informatica, but the resourcing is quite robust. You can scale up and scale down as per your needs.

There should be a way that it can do switches, so if at any point in time I want to do some hybrid mode of making any data collections or ingestions, I can just click on a button. I can change a switch and make sure a batch can be a streaming process.

I've been using Azure Data Factory for more than two years.

The stability of Azure as a PaaS could be improved.

It's scalable.

I would rate their technical support 3 out of 5. It's not great, but it isn't bad.

The setup is complex. It has nothing to do with the technology but with the design. We were wondering how to leverage the orchestration layer where we are having the Azure Data Factory and how to integrate with the Databricks. That's where we had some challenges in terms of choosing the right product.

You can do deployment in-house. 

I would rate this solution 8 out of 10.

For someone who is looking to use this solution, my advice is to do proper due diligence of your current application, know where your application is fitting, and look for the requirements. It all depends upon the current use case that you have currently in your system.

I am a manager of a team that uses this solution.

Azure Data Factory is primarily used for data integration, which involves moving data from sources into a data lake house called Delta Lake.

It's fairly simple to use. The most valuable feature of this solution would be ease of use.

It does not appear to be as rich as other ETL tools. It has very limited capabilities. It simply moves data around. It's not very good after that because it's taking the data to the next level and modeling it.

I have been working with Azure Data Factory for less than a year.

I would say that we are working with the latest version.

The stability of Azure Data Factory is good. The performance is good.

I haven't had to scale this solution as of yet.

We have six people in our company who use this solution.

Increasing the usage is not on our strategy pathway.

I have not contacted technical support. I have not required any yet.

I have had very little contact with Microsoft support, but it's been good.

I have also worked with Talend. I didn't switch products, but rather companies.

Talend is a more robust enterprise-wide solution that can handle everything from start to finish, whereas Azure Data Factory is more of an ingestion tool.

I was not involved with the initial setup.

We are an enterprise that uses an integrator.

It does not require any maintenance, it's simple.

I don't see a cost; it appears to be included in general support. I have been told that you have to be very careful because it can blow out. I have not experienced it yet, but I've been warned that as Azure ingestion increases, the costs can rise.

In my opinion, the price is competitive.

It's a good tool, a good product that does what it's supposed to do well, which is ingesting data from a source to your target, to another cloud, to another source. Just be conscious to monitor your costs.

I would rate Azure Data Factory an eight out of ten.

We are currently using it as an ETL (Extract, Transform, and Load) tool. We are using it to connect to various information providers or, in general, to various sources, to extract data, and then to insert it to our storage devices, databases, or data warehouses.

The two most valuable features of Azure Data Factory are that it's very scalable and that it's also highly reliable.

One area for improvement is documentation. At present, there isn't enough documentation on how to use Azure Data Factory in certain conditions. It would be good to have documentation on the various use cases.

Sometimes, it's really difficult to find the answers to very technical questions regarding certain conditions.

I've been using Azure Data Factory since 2019.

It has been stable so far.  

Azure Data Factory is a very scalable solution. Including internal developers and external consultants working for us, we have about 10-15 people using this solution.

We had been using various ETL tools during the years before moving to the cloud. We picked Azure Data Factory because we were moving towards the Azure cloud.

The initial setup is very easy.

We used a consultant as it was a big project. We had five to six specialists, including both internal and external employees, working on it. It took about about three to six months to complete.

Azure Data Factory is a very easy to use tool. If you want to extract, manipulate, and load data to any type of Azure repository, I recommend this solution. However, I would not recommend it if the manipulation of data is very deep and complicated.

I would rate this solution at eight on a scale from one to ten.

We generally implement this product for data transformation for our clients. We create the pipelines and provide training before handing it over to them. We generally deal with large-scale organizations. I'm a senior solutions architect. 

I think the main benefit of this solution is the ease of use, especially for companies that have come from an SSIS type of background where they are used to Microsoft tools. 

If you have a very simple pipeline you can use Data Factory for transformations, but it's really for serious analytics. This is an excellent tool for pipeline orchestration; connecting the different components and activities as well as gathering data. It's an orchestration tool, not a transformation tool. The monitoring feature has drastically improved.

Data Factory is embedded in the new Synapse Analytics. The problem is if you're using the core Data Factory, you can't call a notebook within Synapse. It's possible to call Databricks from Data Factory, but not the Spark notebook and I don't understand the reason for that restriction. To my mind, the solution needs to be more connectable to its own services.

There is a list of features I'd like to see in the next release, most of them related to oversight and security. AWS has a lake builder, which basically enforces the whole oversight concept from the start of your pipeline but unfortunately Microsoft hasn't yet implemented a similar feature.

I've been using this solution for five years. 

From what I've seen this is a stable solution. 

The solution is easy to scale keeping in mind that Data Factory doesn't do any computations. We use it mainly to push the computations to Databricks or Synapse. Projects with our clients generally last a few months and only until they go into production. I believe the ability to increase is always there.

We typically do not use customer support, but there were a few cases several years ago as the product was moving to the cloud that things were not so stable and we contacted support services - they were very good. 

When I first started in this field, everything was basically Hadoop on-premise and Hadoop infrastructure. With the increase in cloud integrations, things have changed. Once the big data services got introduced, we were probably one of the few companies in North America that were actually into analytics and big data and we were the first to implement related Microsoft products in Canada.

The initial setup is straightforward. I'm a huge fan and user of CI/CD pipelines and never do deployments manually. It's all automated and deployment takes a few minutes.

Licensing costs of Data Factory are reasonable. The cost is mainly on the Synapse and Databricks side of things because they are the tools where the computations are done and where you need more nodes and servers.

It's important to study the solution before purchasing it. The problem in this market is that because most users are generally not very knowledgeable, they typically fall for services that are not compatible with their use case. Data Factory comes with all the transformations but that doesn't work for serious analytics customers who generally need to resort to Databricks or Synapse which involves training and education. Since it's a new field and everything has just blasted off, it's very hard for people to catch on.

In my opinion, Airflow still ranks as number one but I would rate Data Factory an eight out of 10. 

Our customers use it for data analytics on a large volume of data. So, they're basically bringing data in from multiple sources, and they are doing ETL extraction, transformation, and loading. Then they do initial analytics, populate a data lake, and after that, they take the data from the data lake into more on-premise complex analytics.

Its version depends on a customer's environment. Sometimes, we use the latest version, and sometimes, we use the previous versions.

It is very modular. It works well. We've used Data Factory and then made calls to libraries outside of Data Factory to do things that it wasn't optimized to do, and it worked really well. It is obviously proprietary in regards to Microsoft created it, but it is pretty easy and direct to bring in outside capabilities into Data Factory.

It is very flexible. You can build any features you want.

There is always room to improve. There should be good examples of use that, of course, customers aren't always willing to share. It is Catch-22. It would help the user base if everybody had really good examples of deployments that worked, but when you ask people to put out their good deployments, which also includes me, you usually got, "No, I'm not going to do that." They don't have enough good examples. Microsoft probably just needs to pay one of their partners to build 20 or 30 examples of functional Data Factories and then share them as a user base.

I have been using this solution for the last five years, but probably, the last three years have been significant.

It has been stable. I have not experienced any issues.

It is decent for most things. I'm not sure if it is necessarily intended for large volume and high-speed streams of data. By large, I mean really big, but for pretty much anything that most users would want to do, including ourselves, it is fine. Our clients are large government organizations.

It scales fine within its environment. You can literally throw another Data Factory in or replicate one and do things pretty quickly. So, it is not at all hard to increase your processing footprint, but you have to pay for it. It doesn't end up being quite expensive. Although I haven't really done it, I would suspect that if I did the equivalent in AWS, Azure would be more expensive than AWS because of the way they price data.

They're all right. I would rate them a seven out of 10. They do fine, but there is a lot that they don't do.

I'm not sure if even Microsoft has enough SMEs from a user point of view. They are helpful for getting it set up, making it work, and helping you figure out why it doesn't work. If you want to ask them about something that you are trying to do, they'll try to direct you to a partner, which is fine, but the partners also don't necessarily have an experience. It is Catch-22. There aren't a lot of people out there with Azure experience because Azure started to be in demand only over the last two years.

The customer used a lot of homebrew stuff. They were doing a lot of internal stuff and some Oracle stuff. They were doing things, and they made a workaround and said, "Okay, we'll bring it into Oracle Database, and then we'll do all these things to it." We're like, "Okay, that works, but then you're taking it out of that database and putting it over into the data lake. I don't understand why are you doing that?" That's what they were doing.

It is pretty straightforward. Devil is in the details, but you can easily get up and running in a day with Data Factory. Anybody who is comfortable in Azure can set up Data Factory, but it takes experience to know what it can and can't do or should and shouldn't do.

It is proven, and it works. Make sure you have a well-defined use case and build a quick prototype to ensure that it, in fact, does what you need. Give yourself some benchmarks. That's exactly what we did. We defined the use case, and then we set up Data Factory. We found a couple of things that it didn't do. We figured out a way to work around those things and have it do those things. After that, we confirmed it. It is operational, and it is doing its job. It has been pretty much error-free since then.

It would become easier to use as more people become Azure-capable. If I want to find an AWS SME, I can get tons. They're expensive, but I have them. If I want to find an Azure SME, I usually have to create them. Azure was later to market than AWS. So, there are fewer people who are experts in Azure, and they are in high demand.

I would rate Azure Data Factory a nine out of 10. They just don't have enough good examples out there of things.

I use Azure Data Factory in my company because we are implementing a lot of different projects for a big company based in the USA. We're getting certain information from different sources—for example, some files in the Azure Blob Storage. We're migrating that information to other databases. We are validating and transforming the data. After that, we put that data in some databases in Azure Synapse and SQL databases.

I think it makes it very easy to understand what data flow is and so on. You can leverage the user interface to do the different data flows, and it's great. I like it a lot.

It would be better if it had machine learning capabilities. For example, at the moment, we're working with Databricks and Azure Data Factory. But Databricks is very complex to do the different data flows. It could be great to have more functionalities to do that in Azure Data Factory.

I have been using Azure Data Factory for about one year.

Azure Data Factory is a stable solution.

It's scalable. We're doing a lot of different integrations with a lot of data, and scalability is great.

The initial setup is straightforward. I think that it's so easy to start a project using that technology.

We have a team that's in charge of doing the deployments in Azure in different environments.

I would tell potential users that there are many technologies to do this. For example, if you like to manage big data and do something with it, it would be better to use Databricks.

On a scale from one to ten, I would give Azure Data Factory a nine.