Enterprise Data Architect at a manufacturing company with 201-500 employees
Real User
2023-08-22T12:03:30Z
Aug 22, 2023
Good morning, Ornit.
We are using the enterprise edition of Hitachi Vantara's Pentaho Data Integration (PDI) (it also has a community edition that is free to use but lacks some of the automation features that the enterprise edition has) for over a decade now and it will do just about anything we throw at it. We use if not only for ETL but also to generate reports that are burst out to the business. I've used Informatica in the past and the thing that impressed me about Pentaho was it was much easier to build transformations. That may have improved since then (this was back in the early 2000s) but having to link each and every field between each step was super tedious. PDI does it by name with the option to change the mapping manually if needed.
Search for a product comparison in Data Integration
I recommend Talend if you're looking for an on-premises solution. For cloud-based options, ADF or Azure Fabric would work well. These tools are suggested based on the diverse data sources and the large data volume involved.
I recommend Maiora's - ZARUS Data Suite for your ETL requirement. With Zarus you can process your data without any codes. It is designed keeping in mind the non-tech persons using an ETL tool.
Kindly share your available time slots to discuss this further.
For those with a
greater focus on the cloud, there is the AWS Athena service, which is
also very interesting and worth studying:
https://aws.amazon.com/athena/?nc1=h_ls
Data Strategist, Cloud Solutions Architect at BiTQ
Real User
Top 5
2023-08-22T01:20:09Z
Aug 22, 2023
Hi Ornit, My preffered would be Informatica, SSIS, Wherescape. Informatica because it's a mature product that has been outfor a while with minimal development effort required. SSIS for SQL Server based solutions. Wherescape for ETL automation. Wherescape enables SQL users to write etl code in SQL using templates built into the product
Data Integration solutions harmonize data from different sources, ensuring smooth data flow throughout an organization. They are essential in enabling consistent data analysis, fostering better decision-making, and driving efficiency.Data Integration empowers organizations by connecting disparate data systems, reducing duplication, and enhancing data quality. This process involves combining data from various sources and providing users with a unified view. Users benefit from reduced...
Good morning, Ornit.
We are using the enterprise edition of Hitachi Vantara's Pentaho Data Integration (PDI) (it also has a community edition that is free to use but lacks some of the automation features that the enterprise edition has) for over a decade now and it will do just about anything we throw at it. We use if not only for ETL but also to generate reports that are burst out to the business. I've used Informatica in the past and the thing that impressed me about Pentaho was it was much easier to build transformations. That may have improved since then (this was back in the early 2000s) but having to link each and every field between each step was super tedious. PDI does it by name with the option to change the mapping manually if needed.
I recommend Talend if you're looking for an on-premises solution. For cloud-based options, ADF or Azure Fabric would work well. These tools are suggested based on the diverse data sources and the large data volume involved.
IBM DataStage. Run all over the world - the only solution that can scale to meet massive data needs with its parallel processing engine.
Hi Ornit,
I recommend Maiora's - ZARUS Data Suite for your ETL requirement. With Zarus you can process your data without any codes. It is designed keeping in mind the non-tech persons using an ETL tool.
Kindly share your available time slots to discuss this further.
Write back to vijayraj.amin@maiora.co
Regards,
Vijayraj Amin
Maiora's ZARUS supports BYOL (Bring your Own License) to connect to your existing systems and applications.
In WSO2 Enterprise
Integrator 7 there is a very interesting and performant ETL
capability that is worth studying:
https://ei.docs.wso2.com/en/latest/streaming-integrator/guides/performing-etl-tasks/
For those with a
greater focus on the cloud, there is the AWS Athena service, which is
also very interesting and worth studying:
https://aws.amazon.com/athena/?nc1=h_ls
Qlik
Hi Ornit, My preffered would be Informatica, SSIS, Wherescape. Informatica because it's a mature product that has been outfor a while with minimal development effort required. SSIS for SQL Server based solutions. Wherescape for ETL automation. Wherescape enables SQL users to write etl code in SQL using templates built into the product
I recommend SSIS especially if you have already the license for MS SQL Enterprise edition