The first benefit is the time saved by moving to Apache NiFi. Generally, writing a Python program and creating different connectors using a Python program may take three weeks to one week for a developer to create, debug, and deploy. When using Apache NiFi, the time was reduced by almost half. Flows can be created in two days a week, whereas with Python it took almost one to one and a half weeks to complete a single flow. In terms of debugging, Apache NiFi is the only platform where errors can be easily debugged. A bulletin board is available where the exact issue can be viewed and traced back to see where it went wrong. These tasks can be completed quickly, possibly in 30 to 50 minutes to debug an issue. Whereas with Python, if working with a single program level and creating multiple files and multiple jobs, it is very difficult to go back and see exactly where the issue is because tracing from the start to find the issue is necessary, and then every line of code must be debugged. With Apache NiFi, the issue location is immediately identified, and focus can be placed on that particular processor. SLAs have been improved because if an SLA requires completing a job within 30 minutes, using a native programming language or normal jobs requires refinement and optimization for very large transformations, such as ingesting 5 billion records every day, which is not feasible with a normal Python program. Apache NiFi provides the infrastructure where an unlimited number of records, such as billions of records, can be processed in minutes, which ensures that SLAs have never been missed or crossed. SLAs are continuously met, and if additional processing is needed, threads can be increased and little configurations in Apache NiFi can be adjusted, which ensures SLA compliance. These are the metrics that have been seen and benefited from. If an organization is looking for a unified tool to ingest data at very large scale and wants both batch and real-time streaming with one unified data tool, Apache NiFi is recommended. If a code-free approach is preferred and spending much time writing code or debugging code should be avoided, Apache NiFi is an excellent choice where data flows can be created simply through clicks instead of using other particular tools. This review has been given a rating of 8 out of 10.
The customizability of Apache NiFi helps even with unique use cases, as I mentioned before, given that Apache NiFi can be used in this capacity. While there are better applications or software options available, when you are trying to keep it simple and finding ways to utilize a couple of processors for a unique solution, you can do that in Apache NiFi. For example, we have several notification-type pipelines we have built in Apache NiFi, such as reading from a SQL database to identify users who have not completed training and then sending them an email reminder to complete that training. We have that running regularly, week by week. Another instance involves a processing data flow that scans for specific data found in logs, which triggers an email notification to the relevant team letting them know that a unique identifier has appeared, allowing them to handle the situation. I encountered some odd cases such as increasing concurrent threads on a processor, which should work similarly to copying several processors, yet functional throughput varies. It seems that using a distributed processor yields better throughput than just increasing the concurrent threads on one processor, which has been odd but is a workaround we had to adopt to boost throughput. Resolving such quirks could elevate the rating further. I rate Apache NiFi an eight out of ten. I choose eight because, as open-source software, there is always room for improvement, but the tradeoff between learning how to use the software and the savings it provides, along with its customizability, ranks it pretty high. It is effective for what it does and continues to improve, so it could score higher if there are significant enhancements in custom-built processors and ongoing improvements in functionality.
On a scale of one to ten, I would rate Apache NiFi an eight. I chose eight out of ten because there is scope for improvement; in this age, folks want more automation, which is why I kept it at eight. My advice for others considering using Apache NiFi is that it is a good tool, but there are also other options to consider before making a decision. I gave this review an overall rating of eight out of ten.
Apache NiFi should be considered if a scalable and flexible tool is needed for building ETL pipelines and reducing time to production. This review has a rating of 8.
Apache NiFi receives a rating of 9 out of 10. This rating of 9 out of 10 for Apache NiFi was chosen because of the documentation and the support of the product. The advice for others looking into using Apache NiFi is to test the solution with a POC and then go to production in a quick way.
I recommend the product for its data privacy features. It allows secure data handling because the data is stored on my nodes. However, a skilled technician is necessary due to the reliance on Java, especially for back-end operations and error debugging. Enterprise versions may offer easier troubleshooting. As an open-source solution, good support is crucial. I rate the overall product as eight out of ten.
Engineering Lead- Cloud and Platform Architecture at a financial services firm with 1,001-5,000 employees
Real User
Oct 25, 2023
If I were to advise someone, I would ask the user what endpoints they want to touch. If I want to read something from Kafka and I want to put this thing on the S3 bucket, what is the alternative I have? I have Kafka Connect, where I can connect Kafka with one Kafka, and I can put it into an S3 bucket. Is this scalable? No. Is this monitoring No. We can't monitor it. We can't scale it. It's going to be a complete black box. The person who knows Kafka Connect, or Kafka, can understand what is happening there while using Kafka Connect. But if I compare it, I literally don't need to understand what Kafka is. I know, "Okay, this is Kafka. These are the endpoints, and this is the URL I have to point to." That's it. My job is done. I will create a complete flow pipeline within, let's say, thirty minutes or something without having any current knowledge. I can read, I can Google it, and I can just implement it. For people who are new to big data technologies like Kafka and BigQuery, I would give this solution an eight out of ten. Let's say you need to build a solution to read from Kafka and write to an S3 bucket. You could use Kafka Connect, but if your requirements change and you need to start reading from a database instead, Kafka Connect will not work. With Apache NiFi, you can easily modify your flow pipeline to start reading from the database instead.
The solution must be improved to compete with Kafka. As it is an open-source tool, it will take time to get all the functions. I would recommend the product to others. Overall, I rate the product a seven out of ten.
The architect needs to evaluate the entire architecture with this platform so eventually, we are left with our architects and we need to get approval from them to do that. I would rate this solution an eight out of ten.
Apache NiFi offers a flexible platform for data orchestration, transformation, and ingestion, catering to both low and high-code customization needs. It streamlines data movement with a powerful visual interface and robust scalability, facilitating seamless integration with diverse data sources.With Apache NiFi's drag-and-drop capabilities and extensive built-in processors, users can easily simplify complex workflows. Its open-source framework promises cost savings and increased productivity,...
The first benefit is the time saved by moving to Apache NiFi. Generally, writing a Python program and creating different connectors using a Python program may take three weeks to one week for a developer to create, debug, and deploy. When using Apache NiFi, the time was reduced by almost half. Flows can be created in two days a week, whereas with Python it took almost one to one and a half weeks to complete a single flow. In terms of debugging, Apache NiFi is the only platform where errors can be easily debugged. A bulletin board is available where the exact issue can be viewed and traced back to see where it went wrong. These tasks can be completed quickly, possibly in 30 to 50 minutes to debug an issue. Whereas with Python, if working with a single program level and creating multiple files and multiple jobs, it is very difficult to go back and see exactly where the issue is because tracing from the start to find the issue is necessary, and then every line of code must be debugged. With Apache NiFi, the issue location is immediately identified, and focus can be placed on that particular processor. SLAs have been improved because if an SLA requires completing a job within 30 minutes, using a native programming language or normal jobs requires refinement and optimization for very large transformations, such as ingesting 5 billion records every day, which is not feasible with a normal Python program. Apache NiFi provides the infrastructure where an unlimited number of records, such as billions of records, can be processed in minutes, which ensures that SLAs have never been missed or crossed. SLAs are continuously met, and if additional processing is needed, threads can be increased and little configurations in Apache NiFi can be adjusted, which ensures SLA compliance. These are the metrics that have been seen and benefited from. If an organization is looking for a unified tool to ingest data at very large scale and wants both batch and real-time streaming with one unified data tool, Apache NiFi is recommended. If a code-free approach is preferred and spending much time writing code or debugging code should be avoided, Apache NiFi is an excellent choice where data flows can be created simply through clicks instead of using other particular tools. This review has been given a rating of 8 out of 10.
The customizability of Apache NiFi helps even with unique use cases, as I mentioned before, given that Apache NiFi can be used in this capacity. While there are better applications or software options available, when you are trying to keep it simple and finding ways to utilize a couple of processors for a unique solution, you can do that in Apache NiFi. For example, we have several notification-type pipelines we have built in Apache NiFi, such as reading from a SQL database to identify users who have not completed training and then sending them an email reminder to complete that training. We have that running regularly, week by week. Another instance involves a processing data flow that scans for specific data found in logs, which triggers an email notification to the relevant team letting them know that a unique identifier has appeared, allowing them to handle the situation. I encountered some odd cases such as increasing concurrent threads on a processor, which should work similarly to copying several processors, yet functional throughput varies. It seems that using a distributed processor yields better throughput than just increasing the concurrent threads on one processor, which has been odd but is a workaround we had to adopt to boost throughput. Resolving such quirks could elevate the rating further. I rate Apache NiFi an eight out of ten. I choose eight because, as open-source software, there is always room for improvement, but the tradeoff between learning how to use the software and the savings it provides, along with its customizability, ranks it pretty high. It is effective for what it does and continues to improve, so it could score higher if there are significant enhancements in custom-built processors and ongoing improvements in functionality.
On a scale of one to ten, I would rate Apache NiFi an eight. I chose eight out of ten because there is scope for improvement; in this age, folks want more automation, which is why I kept it at eight. My advice for others considering using Apache NiFi is that it is a good tool, but there are also other options to consider before making a decision. I gave this review an overall rating of eight out of ten.
Apache NiFi should be considered if a scalable and flexible tool is needed for building ETL pipelines and reducing time to production. This review has a rating of 8.
Apache NiFi receives a rating of 9 out of 10. This rating of 9 out of 10 for Apache NiFi was chosen because of the documentation and the support of the product. The advice for others looking into using Apache NiFi is to test the solution with a POC and then go to production in a quick way.
Overall, I rate Apache NiFi an eight out of ten. I am quite happy with it.
I recommend the product for its data privacy features. It allows secure data handling because the data is stored on my nodes. However, a skilled technician is necessary due to the reliance on Java, especially for back-end operations and error debugging. Enterprise versions may offer easier troubleshooting. As an open-source solution, good support is crucial. I rate the overall product as eight out of ten.
I rate Apache NiFi an eight out of ten.
If I were to advise someone, I would ask the user what endpoints they want to touch. If I want to read something from Kafka and I want to put this thing on the S3 bucket, what is the alternative I have? I have Kafka Connect, where I can connect Kafka with one Kafka, and I can put it into an S3 bucket. Is this scalable? No. Is this monitoring No. We can't monitor it. We can't scale it. It's going to be a complete black box. The person who knows Kafka Connect, or Kafka, can understand what is happening there while using Kafka Connect. But if I compare it, I literally don't need to understand what Kafka is. I know, "Okay, this is Kafka. These are the endpoints, and this is the URL I have to point to." That's it. My job is done. I will create a complete flow pipeline within, let's say, thirty minutes or something without having any current knowledge. I can read, I can Google it, and I can just implement it. For people who are new to big data technologies like Kafka and BigQuery, I would give this solution an eight out of ten. Let's say you need to build a solution to read from Kafka and write to an S3 bucket. You could use Kafka Connect, but if your requirements change and you need to start reading from a database instead, Kafka Connect will not work. With Apache NiFi, you can easily modify your flow pipeline to start reading from the database instead.
If the volume is manageable, I would recommend it. Overall, I would rate the solution a six out of ten.
The solution must be improved to compete with Kafka. As it is an open-source tool, it will take time to get all the functions. I would recommend the product to others. Overall, I rate the product a seven out of ten.
I rate the solution an eight out of ten.
I would rate this solution an eight out of ten.
There are some claims that NiFi is cloud-native but we have tested it, and it's not. I would rate this solution a seven out of ten.
The architect needs to evaluate the entire architecture with this platform so eventually, we are left with our architects and we need to get approval from them to do that. I would rate this solution an eight out of ten.