Apache Kafka Reviews

Our primary use cases allow software developers, and application developers, the option to not have to code in their own logic for the retry mechanism. A lot of software, and applications, have this feature of retry built in some way or the other, but they all have some kind of a pre-alpha version of Kafka, up to a certain extent.

So it is a good backbone for microservices. So basically you want to write microservices, which you can shut down and bring it up whenever you want. You want to be able to shut it down to actually replace it with a newer version and bring it up. The bottom line is you can kill the microservice and bring it back up and do all the things that you want to do with it. But whenever it comes back up, it should pick up and run from where it had left off. That is what everybody tries to do. And in order to build such a system, they have to write several logical pieces of code, and most of that code has already been built for in Kafka so that you don't have to do it yourself.

One of the most valuable features I have found is Kafka Connect.

Basically, the bootup time, if you have large messages, sometimes takes up more time than I would really like it to. So that is the area that Kafka can actually improve upon. But that is okay, the way we get around it is to make sure that Kafka has started up first and warmed up before anything else starts up. I would also like to see monitoring service tools.

I have been using Apache Kafka for the past three years.

The stability is good as long as you have a short retention period.

Confluent is the cloud version of Apache Kafka and it is scalable.

We do the implementation in-house.

If you are managing your own implementation the return on investment is pretty good. What you need is good developers.

Apache Kafka has open-source pricing.

Apache Kafka is a good choice, so I would recommend people not have a real-time application if they do not have to. It is better to have a very fast batch operation than a real-time operation. I would rate Apache Kafka a nine on a scale of one to ten.

My primary use case for Apache Kafka is replacing ETL and doing data transformations.

Kafka allows you to handle huge amounts of data and classify it into different categories. If you have huge amounts of data, Kafka is a very good solution for data classification. When you need to route it in different directions, you have to take a look at the messages that you get, interfile them, and then send them to the correct place. Kafka is a good product to use in the backend.

The feature I find most valuable is the classification feature. Kafka enables you to tag content with a category.

Kafka contains two components. The component that does the synchronization between the rest of the components, that's an older version of the software and it causes all kinds of configuration problems. The Confluent, which is the company that sells a commercial version of Kafka is getting away from that component precisely because of that. Kafka is a nightmare to administer.

In the next release, I would like to see that one troublesome component that causes configuration issues removed.

I have been using Apache Kafka for a couple of years.

The stability of this solution depends on whether it is properly configured. Having said that, Kafka is incredibly complex to configure, set up, administer, and maintain.

My opinion is that Apache Kafka is a scalable solution. In our organization, there are hundreds of thousands of users using Kafka.

The initial setup was extremely complex. In our case, it took a team of 12 two months to deploy.

These systems were installed by somebody else, not me.

I would advise others to schedule a month or two to just set it up and have it up and running.

There are other options. For example, Databricks is a Kafka alternative. We decided to go with Kafka because one of our clients already chose Kafka.

While evaluating, we found out Databricks is more expensive, for the level of activity that Kafka handles (in this case, millions of requests per day). Databricks could do it, but it would be overly expensive.

I would rate Apache Kafka's pricing a seven out of ten, with one being cheap and 10 being very expensive.

Since it has become so popular, large enterprises especially want to do it. For smaller enterprises, Kafka would probably be too expensive because they would have to hire people to maintain it.

I would rate the Apache Kafka solution a seven out of ten.

We use it extensively in our data pushing, for analytics and all of this type of data that is pushed, rather than on a real-time and payment basis. However, we are using it for offline messages, pushing it for processing, and for heavy, heavy usage, rather than extensively using it for financial data.

The main advantage is increased reliability, particularly with regard to data and the speed with which messages are published to the other side. 

The connectivity from the application is straightforward, as is the API integration.

These are some of the most valuable features of this solution. 

In terms of partition tolerance, message reliability is also present, which is a very good feature from the customer's perspective.

The area for improvement in Kafka is difficult to say because it's a solid product that works well in its intended applications. And, we are looking for something that can be used as part of financial implementations, because we don't want too many messages to be delivered to the other side, which is one of the areas I am looking at as well.

One of the things I am mostly looking for is that once the message is picked up from Kafka, it should not be visible or able to be consumed by other applications, or something along those lines. That feature is not present, but it is not a limitation or anything of the sort; rather, it is a desirable feature.

The next release should include a feature that prevents messages from being consumed by other applications once they are picked up by Kafka.

Then there is message dependability because a message is of no use if cannot be consumed. Alternatively, if the message is consumed but not committed, it should not be recorded in the Kafka queues. It should be because that is one of the features that is existing in MQs consistently provide: if the message is not committed, it will be committed back to the queues.

I have not seen that in Kafka.

We have been using Apache Kafka for approximately three years in the organization.

I believe we are working with version 10. Confluent Kafka is what we are using.

It's a stable solution. Once completed, it is a very stable solution.

The scalability is very good. It is scalable horizontally rather than vertically. 

It can scale up to any level horizontally. However, if the message, once used horizontally scalable, cannot be shrunk once the requirement is reduced, some process is actually taking place. That is one thing that is lacking.

I believe there are approximately 10 to 15 people who use it.

This is being used by the data migration, data team, data analytical team, and data engineer. It's being used by all application architects who are just looking into it, as well as middleware integrators and middleware application integrators.

We have big plans to increase the use of various other innovations and stuff like that. We are using it in relation to data activities. 

Also, we are only planning to use the financial part for publishing it, subscribing, and publishing a pop-up model for various use cases.

Apache usually has a community deployment. If you use Apache or any other software, you will usually receive community support. Otherwise, some companies are taking it and beginning to process it. For example, in Kafka, there is a version of Confluent that they use and support. Or, as we call it, the Oracle Big Data platform.

It will be included with Hadoop, Spark, and other similar technologies. That is coming as, one of the back software packages that are part of that offering, and it is supported by Oracle. Depending on the type of open source, there are various types of support available. Other than the community, we will not receive assistance. Otherwise, it's free enterprise, and we can take it from Confluent or other vendors who offer similar products.

Prior to implementing this solution, we were not using another solution. We have been using, Kafka from the beginning with regard to these use cases. However, we are using other queuing solutions, such as MQ, ActiveMQ, IBM IQ, and Q, but the use cases are different. This is primarily due to the large volume, faster processing, and other benefits of using Kafka.

It is not deployed on-premises. 

We use Kafka as part of the OCI Oracle Cloud platform and the Oracle Big Data platform because Kafka is included.

The Apache Kafka setup will take some time because it is not simple, and we have a lot of other components to install. It's fine because we needed all the plugins and other things for the simple implementations, but the containers' implementation is simple. The only difference is that when it comes to Zookeeper, there are a lot of supporting applications running on top of it, such as Zookeeper. As part of their area, Apache Kafka is running on top of Zookeeper. What do they think? As part of their... manageability, the Kafka area, and Apache Zookeeper. As a result, everything must be removed. And it will be preferable if the implementation is simple.  I believe Confluent is doing this, but we have not yet begun.

The deployment, and configuration, will take one hour to complete. However, it is also dependent on the fact that you require a large number of configurations, which we have.

The deployment was completed in-house.

Currently, there is a team of three to maintain this solution. There are application support personnel in charge of access control.

It will be included in the Oracle-specific platform. It is approximately $600,000 USD.

When it comes to Apache Kafka, they must understand how it works and what its internals are. There could be numerous challenges associated with the product and its entire life cycle. You will have to have a good understanding and knowledge of the configuration. You will need a technical person who is knowledgeable in Kafka which will be an advantage and on an ongoing life partner.

It's a very good solution, I would rate Apache Kafka a nine out of ten.

We have a scalable architecture where we need multiple workers to handle some processing. To make it possible, the backend catches the request and puts it in a common medium, which is the queue of Apache Kafka. The workers then can share and process it.

The most valuable feature of Apache Kafka is the clustering which is very easy to scale and we have multiple servers all over our platforms. It has been useful for stability and performance.

Apache Kafka can improve by providing a UI for monitoring. There are third-party tools that can do it, but it would be nice if it was already embedded within Apache Kafka.

I have been using Apache Kafka for approximately two years.

Apache Kafka is stable. We have not had any issues.

the scalability of Apache Kafka is good. We have parts of the information we use in different geographical sites and it doesn't pose any problem.

I have not used technical support.

I previously used RabbitMQ. We switched because Apache Kafka was more stable and had better performance.

The initial setup of Apache Kafka was easy because it is Dockerized. However, if you were to install it yourself it would be difficult. Having it Dockerized makes it worth it. 

The first deployment took approximately two hours. The updates of the solution can be done in a matter of minutes.

Our DevOps team in our IT department did the deployment of the solution. It was mostly virtual work. The maintenance of the solution does not take a lot of time.

We are using the free version of Apache Kafka.

We had a good experience with the solutions, the maintainability and scalability are good. I would recommend the solution to others.

I rate Apache Kafka a nine out of ten.

We use Apache Kafka for patching real-time data that we receive over a data transport layer and for putting the data into Apache Kafka. From Apache Kafka, we use several applications to subscribe to topics from different applications that we serve directly to browsers. Additionally, we use these applications inside our solution and have Apache Kafka Stream, which is connected to MongoDB.

Since we receive data in real-time consisting of IoT devices, running vehicles, their locations, their states, and their VNs, the solution is helpful.

The product could be improved with proper documentation. Proper documentation should be the SSE. We have a challenge with configuration, so it isn't easy to configure a standalone Apache Kafka on the premises. It needs to be set up on-premises and surveys being provided in the market want to be excluded. Hence, being a developer and configuring Apache Kafka is very hard. It is user-friendly, but initially, we found it challenging. Improving the documentation in this solution would be much better if documents were provided on GitHub for different things. As the market is growing, Spring solution is working hard to get products in the market so when Python, React JS, and Node.Js came, they were lacking. But today, Spring Boot has a solid framework. So the support on Apache Kafka could be improved, but finding some configurations with Spring Boot isn't easy.

We have been using this solution for over three years and are currently using the latest version.

The solution is stable, and the most fantastic thing about it is its throughput. For example, I have tried MQs, which also have Apache Kafka Streams. So the throughput of Apache Kafka Stream is impeccable and impressive.

The solution is very scalable, and its availability is brilliant. We have approximately 32,000 people on our customer base.

We do not have any experience with customer service and support.

We have tried different MQs, but the subscription and charting available on this solution are better. We have used Queues previously, but this solution is more stable, so we chose it.

The initial setup is dependent on the individual. For example, it would be straightforward if a person practices these things a lot and understands the documentation correctly. However, since most people prefer examples instead of reviewing documentation, it would be easy to set up if they find steps on the internet but difficult if they do not have examples.

I rate the pricing for this solution an eight out of ten. It could be a bit cheaper.

I rate this solution an eight out of ten. It is good, but the documentation could be improved.

We utilize Apache Kafka in several areas, including financials, logistics, and client management to name a few.

We used to lose some of our messages when we integrated them in bulk, this solution has stopped that happening.

It is a useful way to maintain messages and to manage offset from our consumers. 

I would like to see an improvement in authentication management.

We have been using the solution for around four years.

The stability is good; the solution operates on our clusters without a big impact.

It is easy to scale.

We used to use a different solution, but our increased throughput meant we needed a product that would allow for a larger queue.

The initial setup was complex for us because we built it internally. This meant that full deployment took around a month.

The implementation was carried out in-house.

I would recommend that other businesses do the deployment themselves, but manage the tool with the aid of a service provider, rather than in-house.

I would rate this product seven out of ten.

Our primary use case of this solution is for data integration and for real-time data consumption. I'm a senior staff engineer for data and infrastructure and we are customers of Apache. 

I love the simplicity of the library and the design as well as the architectural concept which is like horizontal scaling.

When compared to other commercial competitors, Kafka doesn't have the ability to scale down, the elasticity is lacking in the product. The other issue for us is the delayed queue, which was available to us in the commercial software but not in Kafka. It's something we use in most of our applications for deferred processing and I know it's available in other solutions. I'd like to see some tooling support and language support in the open source version. 

I've been using this solution for four years.

The stability is good. 

The solution scales horizontally and scales better than its competitors. We have around 400 to 500 microservices consuming this cluster and the company has around 600 employees. We have four different verticals, each with around 100 engineers with 100 to 150 microservices. 90% of the microservices have a touchpoint with Kafka.

I think the community is very good and will respond if you raise a ticket. We also use external third-party libraries that were built in GitHub. It would be good to have some direct support from Apache.

Four years ago we were using Rabbit MQ but we switched to Kafka because Rabbit was designed for a very narrow use case. It became difficult for us to run and maintain that server and our client libraries. We had a huge outage, so we shifted to Kafka because of the simplicity in the architecture.

The initial setup was simple although we had a couple of hiccups. It took around a week but that was several years ago and we haven't had any problems since. Our team carried out the deployment and we currently have a few engineers who deal with maintenance. 

We are currently using the open-source version. 

There is room for improvement with this solution so I rate it eight out of 10. 

We use Apache Kafka to ingest a lot of data in real time that Apache Spark processes, and the result is used for a tech decision in real time – in the IT environment, infrastructure environment, and IOT environment, like for a  manufacturing plant.

This is an open-source framework. We also sell professional services on this solution and specifically create a business application for customers. 

The application is called Sherlogic. We have two kinds of customers. We have end-user customers that use the Sherlogic solution, and maybe customers don't know that there is Spark and Kafka in Sherlogic. But we have another kind of customer that uses professional services by Xautomata to create tailor-made applications in analytics and the automation process.

We use Apache Kafka for our digital cloud.

To store a large set of analytical data we are using SQL repository. This type of repository works very well but we need specific and high maintenance. The user experience is friendly.

We are looking for alternative solutions, we tried with noSQL solutions and Confluent specific features but the results were not satisfactory both in terms of performance and usability.

We are working on automated SQL repository management and maintenance tools in order to increase the democratization of our platform.

We've been using this solution for a year and a half.

The stability is very nice. We currently manage 50 million events daily.

It's scalable.

Support is good. It's typical for an open source application. You can have all the information in a public portal. If you want specific consulting, there is a company that promotes this consulting worldwide called Conduent. Their consulting is quick and they have a lot of know-how.

It's very complex, like Spark. 

Deployment took 50 minutes for all the Kubernetes ports, Spark, Kafka, and other components based on Sherlogic. In 30 minutes, we created an environment using this program to make installation easier.

Deployment was done in-house, but we're starting a collaboration with another company and we introduced this company to running this solution. Specifically, we started a collaboration with AWS to promote our platform in a Western marketplace. In this way, it's very easy to use our solution because it is a part of an AWS service, certificated by an engineer.

The return on investment has been having people dedicated to this solution because it's open source so it hasn't been necessary to invest in licensing or pay a fee. So, internal know-how has been the ROI.

It's a bit cheaper compared to other Q applications.

I would rate this solution 7 out of 10.

I would recommend this solution because the queue manager is very fast and stable.