Azure Data Factory Reviews

There was a need to bring a lot of CRM and marketing data for some PNL analysis. We are connecting to the Salesforce cloud. In it, there's a specific solution in Salesforce Core CRM for the pharmaceutical industry. We are using the solution to connect to that and we are bringing in the various dimensions and transactions from that data source.

Data Factory itself is great. It's pretty straightforward. You can easily add sources, join and lookup information, etc. The ease of use is pretty good. 

They have a lot of other components like a newer monitor, which helps track and generate alerts for any failed jobs and things of that nature, which is helpful.

At this point in time, they should work on somehow integrating the big data capabilities within it. I have not explored it, but it would be good if somehow we could call a Spark job or something to do with the Spark SQL within ADS so that we wouldn't need a Spark tested outside.

On the UI side, they could make it a little more intuitive in terms of how to add the radius components. Somebody who has been working with tools like Informatica or DataStage gets very used to how the UI looks and feels. 

In ADS, adding a new table or joining a new table and overriding that with an override SQL that I could customize would be helpful.

Being able to debug from the design mode itself would be helpful.

I've been using the solution for one year.

In the latest version, the v2 version, the solution is pretty stable. It does not give unknown letters or things like that.

The solution allows you to create reusable components, so it can be scaled pretty easily.

Being an IT services company, we have a gold or a platinum partnership with Microsoft. For us, getting the technical support we need is not a big issue. Their community is also pretty active in responding to any issues. It's quite good. We've been satisfied with the level of support that is offered.

We were not actually involved in the initial setup. That was all with the client, so I won't be able to comment on it.

In terms of licensing costs, we pay somewhere around S14,000 USD per month. There are some additional costs. For example, we would have to subscribe to some additional computing and for elasticity, but they are minimal. 

For disaster recovery and readability setups, we did that on Data Lake.

We use the public cloud deployment model.

I'd warn others to ensure that the design should be frozen before you start building because overriding each other's code and managing code takes effort. To avoid or to reduce that effort, ensure that the design is frozen. You can build some configurable code rather than hard-coding everything into the jobs. That's the biggest recommendation.

I'd rate the solution seven out of ten. It's a pretty good solution, but over the past year, I've been limited on the number of cases I have on it. If it had a better user interface and was more intuitive I would have rated it higher.

The support for SAP services and databases, specifically SAP HANA, has been a game-changer for us.

It would be helpful if they could adjust the data capture feature so that when there are source-side changes ADF could automatically figure it out.  

The solution needs to integrate more with other providers and should have a closer integration with Oracle BI.

I've been working with the solution for just over a year.

From what we have seen so far, the solution seems very stable.

The solution is scalable. Right now, we have only three or four people on it. We may increase usage int eh future.

We have direct technical support in Israel. In India, we haven't used it yet. However, for other services, the help provided is fine. 

The initial setup was pretty fast and simple. I wouldn't say it was too complex. For deployment, it depends on your environment. In our POC, it didn't take more than two to three hours. In the production environment, however, it takes a bit more time for operations.

In general, we've been satisfied with the pricing. However, since it's a service based on usage, it's really hard to estimate the cost in advance for this product. This makes it hard to quote. It's not so easy to estimate the costs in advance when giving a quote. That makes it difficult to convince some customers because most would like to see a number in advance. 

We primarily use the solution in a hybrid environment. We're Azure partners.

I would suggest others start with a small POC and with a smaller dataset or simple pipeline to test the product before fully implementing it. Users shouldn't hesitate to try it, because there are not so many risks. Practice is the best way to start with this service, as opposed to planning. 

I'd rate the solution eight out of ten.

The user interface is very good. It makes me feel very comfortable when I am using the tool.

The solution could use some merge statements.

The solution should offer better integration with Azure machine learning. We should be able to embed the cognitive services from Microsoft, for example as a web API. It should allow us to embed Azure machine learning in a more user-friendly way. 

The solution is scalable.

We've been satisfied with the level of technical support we've received.

The initial setup is very straightforward. The only thing which we had to consider at the beginning was the gateway for the on-premises environment. Otherwise, it's very easy.

We use both the on-premises and cloud deployment models. We typically work with enterprise-level companies.

Azure Data Factory is pretty good but should be considered as an orchestrator, not as an integrated tool. We can use some building components in that tool to orchestrate the entire workflow but if we are thinking about more details, processing, or data modification during the flow, we'd have to consider Azure Databricks or Data Flow for making those calculations or changes. Users will need Azure Data Factory plus third party tools to reach that level of functionally.

I would recommend using Data Factory. I don't have a lot of experience with integration or with integration services, for example, SQL server integration services. However, there are points that should be considered if you are already using SQL server integration services already. You can implement the packages already prepared in Azure Data Factory. It's something that needs to be considered when deciding which technology you are going to use.

I'd rate the solution eight out of ten.

We are working on a data warehouse integration which means that I am working on some big data projects. I'm preparing data for the licensing. One of the projects is preparing data in Azure Data Lake, to run some transformation scripts, perform some ETL processing, and to fulfill the stage layer of the data warehouse. It means that I help with ETL use cases.

I like the feature that allows you to connect to different sources. You can go through forms and click on them and you don't have to provide the scripts. The solution has a good interface and the integration with GitHub is very useful.

There is also a self-service integration that allows you to run the BTS pages on the SQL server from an SIS. Users have an option to configure this self-service integration engine to run items as a part of a pipeline of the Azure data factory.

I think more integration with existing Azure platform services would be extremely beneficial.

In the next release, it's important that some sort of scheduler for running tasks is added.  A built-in scheduling mechanism for running the treasury will be a very helpful improvement.

I've never experienced any issues with the solution's stability.

The scalability of the connected engines makes this solution very scalable.

Although I've had to contact support for other Azure products, I haven't had to for this solution, so I can't speak to any experience with technical support directly.

The initial setup is straightforward. It's got a very nice welcome page where everything explained. Depending on the complexity, deployment takes only a few days. You only need one person for the deployment of the solution. This is dependent, however, on how many pipelines you are implementing. Afterward, you would probably only need one person to maintain it.

I'm an integrator, so I work with clients to help them implement the solution.

I'm not sure of the specific cost, but it is monthly and is somewhere in the ballpark of a few hundred Euros. You don't have to pay for extras. Everything is included in the cost.

I don't use Azure Data Factory for my own company; I help clients implement the solution for their companies.

In terms of advice that I would give to others looking to implement the solution, I would say you have to learn. You have to really understand the overall concept. It does not allow you to just click and go.

I would rate this solution ten out of ten.

Used Azure Data Factory, Data Flow (private preview) and Databricks to develop data integration processes from multiple and varied external software sources to an OLTP application Azure SQL database. The tools are impressively well-integrated, allowing quick development of ETL, big data, data warehousing and machine learning solutions with the flexibility to grow and adapt to changing or enhanced requirements. I can't recommend it highly enough.

This solution will allow the organisation to improve its existing data offerings over time by adding predictive analytics, data sharing via APIs and other enhancements readily. It's as simple as extending the data pipelines with new modules and components. The solution is improving the organisation by offering something the organisation can grow with.

Data Flow and Databricks are going to be extremely valuable services, allowing data solutions to scale as the business grows and new data sources are added.

Data Flow is in the early stages — currently public preview — and it is growing into a tool that will offer everything other ETL tools offer. There are a few features still to come. The thing we missed most was data update, but this is now available as of two weeks ago. A feature that is confirmed as coming soon is the ability to pass in a parameter and filter, etc.