What is our primary use case?
We are pulling data from an upstream tool, which is D365 ERP, and pushing it to a particular DB or lakehouse for BI purposes, to be consumed by the BI users. We are using Fabric Data notebooks to integrate systems using Microsoft Graph APIs, pulling data from D365 ERP and pushing it to a compliance tool where the business users will make decisions.
Our main use cases for Fabric Data are building ETL and integrating systems. We are using data pipelines as well as Fabric Data notebooks majorly. We have also built an interim solution using SharePoint list and Fabric Data as an MDM solution, though it is not a true MDM.
What is most valuable?
We were using small-scale ETL tools called Sinch. We are now moving away from such tools as Sinch and Rally to Fabric Data, which provides a unique, unified way of working. It brings teams together and makes collaboration much easier. In that collaborative environment, we can include a BI user as part of a workspace, and end users can also be part of a workspace.
Collaboration has improved significantly. Fabric Data has data flow activities in the ETL data pipeline where a user without extensive data engineering knowledge can easily build an ETL data pipeline using those data flows, as long as they are familiar with DAX queries. This reduces the workload for data engineers.
What needs improvement?
I have observed some limitations with Fabric Data, especially when it comes to bringing data from private networks. Being a SaaS product, there is limited control, so having the option to create data gateways more easily within Fabric Data itself would be great. It would help to have more clarity about licensing capacities, as there are various capacities such as F16 and F64. A more detailed knowledge section within Fabric Data would be beneficial, while Microsoft Fabric Learn has extensive details, I would prefer to see this information integrated directly within Fabric Data itself.
The user interface of Fabric Data is quite standard, with regular changes published by Microsoft that come in handy when I have multiple tabs open. As for scalability, it is based on use cases. I do not think scalability is the biggest advantage of Fabric Data compared to any other Azure resources. We have auto-scaling in other facilities available, but it is not a major advantage compared to usage.
For how long have I used the solution?
I have been using Fabric Data for two years.
What do I think about the stability of the solution?
I have not experienced any downtime or reliability issues with Fabric Data as of now.
I have not faced any issues with performance since the data we are processing currently is comparatively less. However, we have various systems to connect to, and since Fabric Data provides most of the functionalities available in ADF pipelines, I have nothing to complain about.
What do I think about the scalability of the solution?
I have not encountered any challenges in scaling Fabric Data as of now, since the data volume is relatively low. I think it is very easy to scale it up and down.
The scalability of Fabric Data meets our needs as our data grows, as we have F16, F64, and other licensing capacities that fulfill our organization's requirements.
How are customer service and support?
I have not reached out to customer support for Fabric Data at this point.
Which solution did I use previously and why did I switch?
We were using small-scale ETL tools called Sinch. We are now moving away from such tools as Sinch and Rally to Fabric Data, which provides a unique, unified way of working. It brings teams together and makes collaboration much easier.
We were using small-scale ETL tools that did not have features such as drag and drop. Each data flow had to be configured using C#, and while they had some connection facilities to sources such as SQL Server and Amazon Marketplace, utilizing C# to orchestrate data workflows was challenging and time-consuming. The accuracy and visibility were comparatively low, which led us to switch to Fabric Data.
How was the initial setup?
Compared to ADF, Fabric Data is much easier to build an ETL data pipeline. In ADF, I would have to create datasets and linked services, but with Fabric Data, I do not have to worry about it much and can directly go through and create a copy activity. As it is a kind of SaaS product, it is always better, and even a minor task such as building or adding an activity to send an email alert for an ETL data pipeline is much easier compared to ADF. In ADF, I have to create a web activity and call the URL to send the email alerts. There are many of these kinds of minor changes made on top of ADF, making it handy. The workspaces give better security control, and overall, the architecture built has security advantages compared to ADF. It is also very easy to set up.
What about the implementation team?
I am not in a position to train the team, but personally, I find it very easy to learn. The user interface is nice, and there are plenty of resources available in Microsoft Learning and other various open sources.
What was our ROI?
We have certainly saved a significant amount of time by switching from small-scale tools to Fabric Data, which has improved visibility and increased accuracy.
What's my experience with pricing, setup cost, and licensing?
I do not have much knowledge regarding pricing, setup costs, or licensing for Fabric Data because our Azure admin handles it.
Which other solutions did I evaluate?
We primarily operate within Azure, so our options were mainly ADF and Microsoft Fabric. Since we were already utilizing Power BI and ADF for data migration activities, it was a clear choice to adopt Fabric Data.
What other advice do I have?
Security is a major challenge for our organization, as there are many users who work from different locations and have different roles. With Fabric Data, having workspaces and control over lakehouses and other resources allows us to maintain granularity and manage user access effectively, which has been a great advantage.
Within Fabric Data, I am using notebooks to integrate with other platforms and compliance tools. I need to be familiar with REST APIs and Microsoft Graph APIs. There are some inbuilt connections to Fabric Data, such as with Profisee, which provide an easier way to integrate.
Fabric Data provides a nice UI to track data lineage. For data governance, we are using Purview. For auditing and access management, we have the capability to assign the right access to each user based on their roles, which is a good advantage.
My advice for others considering using Fabric Data is to have prior knowledge about building ETL data pipelines. If they have worked with ADF, it makes their process much easier compared to someone who is coming directly to Fabric Data. Developers will have a significant advantage in creating their own data pipeline using DAX queries. I would rate Fabric Data an eight overall.
Which deployment model are you using for this solution?
Private Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Microsoft Azure