Apache Kafka Reviews

We are using Apache Kafka to extract data from a Portuguese data source, utilizing an open-source project for data capture. The connector for this project is linked to both Kafka and Confluence platforms. We then transform the extracted data and store it in Elasticsearch.

We appreciate the ability to persistently and quickly write data, as well as the flexibility to customize it for multiple customers. Additionally, we like the ability to retain data within Apache Kafka and use features, such as time travel to access past customer data. The connection with other systems, such as Apache Kafka and IBM DB2. 

Apache Kafka can improve by adding a feature out of the box which allows it to deliver only one message.

I have used Apache Kafka within the last 12 months.

Apache Kafka is a stable solution.

The scalability of Apache Kafka is good. It can process many requests simultaneously.

We have approximately 600 people using this solution in my organization.

I have not contacted the support from Apache Kafka.

The initial setup is relatively easy as I am using Docker and the files provided by Confluent. However, setting up Apache Kafka in a production environment is not as straightforward. I prefer to use solutions, such as Confluence that already have everything preconfigured. As a developer, creating an environment for it is not a problem for me, but I think it can be challenging for those responsible for the production environment. There have been issues with data loss and other problems in the past. Configuring it for production is not easy.

My deployment was very quick because I am using it locally. We have someone else that does the cloud deployment.

I did our local implementation and we have someone else that does the cloud deployment.

The price of Apache Kafka is good.

I rate the price of Apache Kafka an eight out of ten.

I don't see any major issues with using Apache Kafka. Many companies use it and it's a good solution. My advice would be to use it as a software-as-a-service rather than setting up your own cluster. This way, you can benefit from a preconfigured and maintained platform. It's better to opt for a software-as-a-service solution.

I rate Apache Kafka an eight out of ten.

Our primary use cases allow software developers, and application developers, the option to not have to code in their own logic for the retry mechanism. A lot of software, and applications, have this feature of retry built in some way or the other, but they all have some kind of a pre-alpha version of Kafka, up to a certain extent.

So it is a good backbone for microservices. So basically you want to write microservices, which you can shut down and bring it up whenever you want. You want to be able to shut it down to actually replace it with a newer version and bring it up. The bottom line is you can kill the microservice and bring it back up and do all the things that you want to do with it. But whenever it comes back up, it should pick up and run from where it had left off. That is what everybody tries to do. And in order to build such a system, they have to write several logical pieces of code, and most of that code has already been built for in Kafka so that you don't have to do it yourself.

One of the most valuable features I have found is Kafka Connect.

Basically, the bootup time, if you have large messages, sometimes takes up more time than I would really like it to. So that is the area that Kafka can actually improve upon. But that is okay, the way we get around it is to make sure that Kafka has started up first and warmed up before anything else starts up. I would also like to see monitoring service tools.

I have been using Apache Kafka for the past three years.

The stability is good as long as you have a short retention period.

Confluent is the cloud version of Apache Kafka and it is scalable.

We do the implementation in-house.

If you are managing your own implementation the return on investment is pretty good. What you need is good developers.

Apache Kafka has open-source pricing.

Apache Kafka is a good choice, so I would recommend people not have a real-time application if they do not have to. It is better to have a very fast batch operation than a real-time operation. I would rate Apache Kafka a nine on a scale of one to ten.

We used Kafka as a central message bus, transporting data from SNMP through to a database. Some of the processing in between was handled by other components.

We built a solution for a client and the client was happy with the solution.

The message bus capabilities, basically sending messages to it, and the way it handles events or messages is pretty good. The throughput was good. Generally, it was a good component.

We struggled a bit with the built-in data transformations because it was a challenge to get them up and running the way we wanted. There was a bit of a learning curve. It may be that we didn't fully grasp the information.

Also, the documentation covering certain aspects was a bit poor. We had to trawl around different locations to try to find what we needed. When we were able to find documentation on transformation, for example, there wasn't a good set of documentation examples we could use, and the examples we had weren't quite meeting the need. Better examples would've helped us.

I used this solution for about a year and a half. 

It seemed pretty stable and didn't have any issues at all.

I don't know how many people were using it on the client's side, but we had a four-person team doing the development work. 

Our team handled the deployment in-house.

Kafka is an open-source solution, so there are no licensing costs. There are third-party companies who support and provide add-ons to Kafka, but we didn't need to use any of those. Confluence, for example, provides plug-ins for Kafka. 

There were other solutions, like Apache MQ, but there were a number of components we looked at that were based around being a message bus, and Kafka was the winner from that review work.

The documentation can be a challenge. There are quite advanced capabilities of Kafka, like the transformations that you can build to modify the data as needed. We found that the biggest challenge was documentation and being able to gain the knowledge of exactly how to do stuff. We also struggled on the transformation, but other components were fine, so some parts are good, and some parts are bad.

I would rate this solution as an eight out of ten. 

We use it extensively in our data pushing, for analytics and all of this type of data that is pushed, rather than on a real-time and payment basis. However, we are using it for offline messages, pushing it for processing, and for heavy, heavy usage, rather than extensively using it for financial data.

The main advantage is increased reliability, particularly with regard to data and the speed with which messages are published to the other side. 

The connectivity from the application is straightforward, as is the API integration.

These are some of the most valuable features of this solution. 

In terms of partition tolerance, message reliability is also present, which is a very good feature from the customer's perspective.

The area for improvement in Kafka is difficult to say because it's a solid product that works well in its intended applications. And, we are looking for something that can be used as part of financial implementations, because we don't want too many messages to be delivered to the other side, which is one of the areas I am looking at as well.

One of the things I am mostly looking for is that once the message is picked up from Kafka, it should not be visible or able to be consumed by other applications, or something along those lines. That feature is not present, but it is not a limitation or anything of the sort; rather, it is a desirable feature.

The next release should include a feature that prevents messages from being consumed by other applications once they are picked up by Kafka.

Then there is message dependability because a message is of no use if cannot be consumed. Alternatively, if the message is consumed but not committed, it should not be recorded in the Kafka queues. It should be because that is one of the features that is existing in MQs consistently provide: if the message is not committed, it will be committed back to the queues.

I have not seen that in Kafka.

We have been using Apache Kafka for approximately three years in the organization.

I believe we are working with version 10. Confluent Kafka is what we are using.

It's a stable solution. Once completed, it is a very stable solution.

The scalability is very good. It is scalable horizontally rather than vertically. 

It can scale up to any level horizontally. However, if the message, once used horizontally scalable, cannot be shrunk once the requirement is reduced, some process is actually taking place. That is one thing that is lacking.

I believe there are approximately 10 to 15 people who use it.

This is being used by the data migration, data team, data analytical team, and data engineer. It's being used by all application architects who are just looking into it, as well as middleware integrators and middleware application integrators.

We have big plans to increase the use of various other innovations and stuff like that. We are using it in relation to data activities. 

Also, we are only planning to use the financial part for publishing it, subscribing, and publishing a pop-up model for various use cases.

Apache usually has a community deployment. If you use Apache or any other software, you will usually receive community support. Otherwise, some companies are taking it and beginning to process it. For example, in Kafka, there is a version of Confluent that they use and support. Or, as we call it, the Oracle Big Data platform.

It will be included with Hadoop, Spark, and other similar technologies. That is coming as, one of the back software packages that are part of that offering, and it is supported by Oracle. Depending on the type of open source, there are various types of support available. Other than the community, we will not receive assistance. Otherwise, it's free enterprise, and we can take it from Confluent or other vendors who offer similar products.

Prior to implementing this solution, we were not using another solution. We have been using, Kafka from the beginning with regard to these use cases. However, we are using other queuing solutions, such as MQ, ActiveMQ, IBM IQ, and Q, but the use cases are different. This is primarily due to the large volume, faster processing, and other benefits of using Kafka.

It is not deployed on-premises. 

We use Kafka as part of the OCI Oracle Cloud platform and the Oracle Big Data platform because Kafka is included.

The Apache Kafka setup will take some time because it is not simple, and we have a lot of other components to install. It's fine because we needed all the plugins and other things for the simple implementations, but the containers' implementation is simple. The only difference is that when it comes to Zookeeper, there are a lot of supporting applications running on top of it, such as Zookeeper. As part of their area, Apache Kafka is running on top of Zookeeper. What do they think? As part of their... manageability, the Kafka area, and Apache Zookeeper. As a result, everything must be removed. And it will be preferable if the implementation is simple.  I believe Confluent is doing this, but we have not yet begun.

The deployment, and configuration, will take one hour to complete. However, it is also dependent on the fact that you require a large number of configurations, which we have.

The deployment was completed in-house.

Currently, there is a team of three to maintain this solution. There are application support personnel in charge of access control.

It will be included in the Oracle-specific platform. It is approximately $600,000 USD.

When it comes to Apache Kafka, they must understand how it works and what its internals are. There could be numerous challenges associated with the product and its entire life cycle. You will have to have a good understanding and knowledge of the configuration. You will need a technical person who is knowledgeable in Kafka which will be an advantage and on an ongoing life partner.

It's a very good solution, I would rate Apache Kafka a nine out of ten.

We use Apache Kafka for patching real-time data that we receive over a data transport layer and for putting the data into Apache Kafka. From Apache Kafka, we use several applications to subscribe to topics from different applications that we serve directly to browsers. Additionally, we use these applications inside our solution and have Apache Kafka Stream, which is connected to MongoDB.

Since we receive data in real-time consisting of IoT devices, running vehicles, their locations, their states, and their VNs, the solution is helpful.

The product could be improved with proper documentation. Proper documentation should be the SSE. We have a challenge with configuration, so it isn't easy to configure a standalone Apache Kafka on the premises. It needs to be set up on-premises and surveys being provided in the market want to be excluded. Hence, being a developer and configuring Apache Kafka is very hard. It is user-friendly, but initially, we found it challenging. Improving the documentation in this solution would be much better if documents were provided on GitHub for different things. As the market is growing, Spring solution is working hard to get products in the market so when Python, React JS, and Node.Js came, they were lacking. But today, Spring Boot has a solid framework. So the support on Apache Kafka could be improved, but finding some configurations with Spring Boot isn't easy.

We have been using this solution for over three years and are currently using the latest version.

The solution is stable, and the most fantastic thing about it is its throughput. For example, I have tried MQs, which also have Apache Kafka Streams. So the throughput of Apache Kafka Stream is impeccable and impressive.

The solution is very scalable, and its availability is brilliant. We have approximately 32,000 people on our customer base.

We do not have any experience with customer service and support.

We have tried different MQs, but the subscription and charting available on this solution are better. We have used Queues previously, but this solution is more stable, so we chose it.

The initial setup is dependent on the individual. For example, it would be straightforward if a person practices these things a lot and understands the documentation correctly. However, since most people prefer examples instead of reviewing documentation, it would be easy to set up if they find steps on the internet but difficult if they do not have examples.

I rate the pricing for this solution an eight out of ten. It could be a bit cheaper.

I rate this solution an eight out of ten. It is good, but the documentation could be improved.

Apache Kafka is a messaging solution where you have topics to pass on your information. You can send messages to multiple topics.

We need to manage limited resources. Additionally, we can send or push messages through a specified port. This is a significant feature because, unlike traditional queues, Kafka uses a cluster of nodes, making it easy to integrate with various algorithms. This clustering is an advantage and a key feature of Kafka, providing good interaction and scalability.

For example, when you want to send a message to inform all your clients about a new feature, you can publish that message to a single topic in Apache Kafka. This allows all clients subscribed to that topic to receive the message. On the other hand, if you need to send billing information to a specific customer, you can publish that message on a topic dedicated to that customer. This message can then be sent as an SMS to the customer, allowing them to view it on their mobile device.

Apache Kafka is different in its design. If you have topics around the front end of clusters in the facility, it is scalable. The software is scalable to handle and process data. However, it might not be suitable for handling specific types of images or media files. Other than that, it should handle the rest of the data processing needs.

There are no multiple versions, which simplifies the process of granting access with Kaspersky. Every message is accurately delivered. However, Kafka does not support sending messages directly. You need to publish messages finalization. If you want to resend a message, you must resend it manually. Kafka does not automatically handle this. Another thing is the need for a redo option if an issue occurs. If a message is not sent properly, it can be retransmitted within the core system. You should enable the gateway in your program for it to function correctly. Messages will not be delivered or refreshed unless you enable the direct replay option in the product settings.

I have been using Apache Kafka since 2020-21

The initial setup of Apache Kafka is challenging and requires experience. Each message should always receive a response, so prioritizing traffic is essential. Furthermore, the client or consumer must always be in sync, or the message will not be processed.

One pair of nodes is sufficient for the system. If our other system requires more than five nodes, it might not be feasible. Currently, other components are functioning as expected. The Kafka setup won't take much time.

When using Apache Kafka, it’s important to manage different environments carefully to avoid confusion. For instance, you can configure different client applications for producing and consuming messages. Ensure that the configurations for each environment (development, testing, production, etc.) are separated. This includes managing source code and data appropriately to maintain security and efficiency. Proper management of Kafka assets and operations phases is crucial for a smooth workflow.

I recommend Apache Kafka since it is extremely fast, stable and has been used for a very long time. We haven't encountered any major issues or concerns regarding its performance and customer service.

Overall, I rate the solution a nine out of ten.

I have previous professional experience using Kafka to implement a system related to gathering software events in one centralized location. 

One example of how Kafka has been beneficial is in the context of financial trading. When a trade is executed, it generates an event. I used Kafka to create an application that captures these events and stores them in a topic, allowing for efficient processing in real time.

Regarding the most valuable feature in Kafka, I would say it's scalability. The publisher-subscriber pattern and low latency are also essential features that greatly piqued my interest.

Maintaining and configuring Apache Kafka can be challenging, especially when you want to fine-tune its behavior. It involves configuring traffic partitioning, understanding retention times, and dealing with various variables. Monitoring and optimizing its behavior can also be difficult.

Perhaps a more straightforward approach could be using messaging queues instead of the publish-subscribe pattern. Some solutions may not require the complex features of Apache Kafka, and a messaging queue with Kafka's capabilities might provide a more complete messaging solution for events and messages.

I have been using Apache Kafka for the past 10 years. 

The stability may improve if the configuration and management aspects become less challenging.

It depends on the configuration., but scalability is one of the best features of Kafka. I would rate it nine out of ten.

Support can vary depending on whether you're using the open source version or a paid one. Our version, the paid console version, offers highly available support, and you can find a wealth of information and assistance from various providers online. However, when I used MSA on AWS, I encountered limited support for it.

Neutral

Despite the challenges we faced with configuration and management, I believe the return on investment is safeguarded.

The cost can vary depending on the provider and the specific flavor or version you use. I'm not very knowledgeable about the pricing details.

I believe that when working with Kafka Apache, it's essential to have a specialist who thoroughly understands and can optimize all the available variables within the solution to achieve the desired behavior.

I would rate it an eight out of ten.

We use Kafka for Elastic Stack and Kafka SCRAM login.

I have many users of Apache Kafka. It's like a subject to study in enterprises. However, we have not decided if the systems should generalize Apache Kafka for every application and every IT system.

We use Kafka for mapping and ThoughtSpot data from one IT system source to the destination. We also prefer it to exchange data from our internal IT systems.

Kafka is a new method we opted to apply to our need for data exchange. Also, we use the solution's integration capabilities.

Irovement-wise, I would like the solution to have more integration capabilities. Also, the solution's setup, which is currently complex, should be made easier.

I have experience with Apache Kafka.

It is a stable solution.

A lot of my experience indicates that Apache Kafka is scalable. We can have ten or even fifty hundred users on the solution. So, it's possible because we are a big enterprise.

I have experience with Apache Kafka's technical support.

The solution's initial setup process was complex. The deployment process took three or four years.

Right now, I can't deliver the planning process required for deployment.

For deployment and maintenance, we have a manager and an operational person. However, I can't give an exact count of the people required for deployment and maintenance.

To be able to recommend Kafka to others, especially considering every context, we will have to set a benchmark and compare Kafka with other tools.

I rate the overall solution a seven out of ten.