Apache Kafka on Confluent Cloud Reviews

Whenever you need to handle a huge load of real-time data processing, Kafka is useful. We currently use it for an output management system for insurance, where the system receives data in a fixed amount and has to process it in several steps. 

We manage these steps with Kafka because the load can be quite big, with millions of XMLs coming into the system that need to be processed in near real-time.

The most valuable feature to me is it scales really well when there is a data influx, irrespective of the peak time. It can handle both large and small amounts of data. 

Also, the state-saving feature is very much appreciated. It allows me to rewind a certain process if I see an error and then reprocess it.

The administration port could be more extensive. Additionally, managing the states of certain events could be made easier, perhaps with automatic rollback instead of having to program it manually.

I have been using it for a few months. 

For us, the stability has been great. No complaints here.

Scalability is great. We haven't gone into production yet, but so far, it scales very well.

I would rate the scalability a nine out of ten. There are maybe a dozen or so people who really have to interact with Kafka.

The setup was straightforward. Confluent Cloud takes a lot of the work out of the user's hands, and it's easy to set up. We're quite happy with it.

We didn't need any help. We have one developer, and that's it.

The benefit is that we can offer clients cloud streaming for data processing, which we host for them. They benefit from the speed and reliability.

Since we are in development, we have no license. We will need one eventually. The price isn't low, but it's not too high.

I evaluated other options for workflow solutions like Camunda, and other messaging services, like NQ series.

Overall, I would rate the product an eight out of ten. So far, we haven't found any obstacles, and we're happy that it's been straightforward and problem-free.

The product is very good, but if you need to set it up on your own, there's quite a bit of work. That's why we chose Confluent as a partner who has made it tailored for us. That's surely a good way to start. When you have more experience, you can try setting it up on your own.

We use Apache Kafka with Confluent Cloud for specific real-time transaction use cases, both on-premise and in the cloud. We have been using Confluent Cloud for about five years.

We initially used it for data reputation, then expanded to microservices integration and Kubernetes, focusing on improving data quality and enabling real-time location tracking.

We configure it for data transactions across various topics and partitions, depending on the specific use case and required throughput.

From an IT perspective, I've used this product across all domains: system development, operations, data management, and system quality.

We have experience using Kafka on Confluent Cloud for data pipelines. We've implemented several techniques to optimize topic usage, integrated it with microservices, and even utilized change data capture (CDC) components.

We leverage topic configurations and partitions extensively. We simulate various use cases with different partition numbers, like high throughput scenarios with 45 partitions or high transaction environments with other configurations.

In our microservices architecture running on Kubernetes, Confluent Cloud helps us manage transactions effectively. Additionally, it integrates seamlessly with our data analysis tools like DataStage, Big Data, and Teradata, providing a smooth flow for large data volumes.

The overall integration with other tools and efficient transaction management are the key benefits I experience with Confluent Cloud for large-scale data streams.

I saw an interesting improvement related to the analytics environment.

We have been using this solution since 2018.

We have a well-defined process and platform for scaling big data solutions. When multiple providers propose their options, we configure a custom platform based on our current use cases. 

However, we're planning to migrate to a new big data platform within the next fifteen months. This timeframe is due to our internal process for evaluating and deploying new platforms.

In terms of configuring the product, specifically Confluent, understanding the design and configuring values for various parameters is something only I am familiar with. The initial setup, including the initial Non-Disclosure Agreement (NDA) and progress in implementation, is quite difficult.

We primarily use on-premises Kafka for high-transaction scenarios. If something crashes there, we handle data processing manually. It might not be the most efficient, but we haven't considered it a major concern. 

For other use cases, we also prefer on-premises. 

The implementation took us one year. It involved configuring the platform over a year. The time required for configuring or implementing use cases varies; some take longer, while others might also take up to a year.

I attempted the deployment myself. However, there were three of us involved in these tasks within this analytical environment.

My role revolves around deploying use cases in analytics. I also operate within Architect areas, focusing on data architecture.

For maintenance, the same three people take care of it. We might need two more, but for now, three is sufficient.

The platform and container operations themselves provided significant value.

Our use case is for real-time data integration. It was a preferred tool for this purpose. Additionally, we employed Azure EventHub, another service, as an indicator for real-time data in a couple of larger programs focused on integrating real-time data and visualization.

Kafka and Confluent Cloud have proven to be cost-effective, especially when compared to other tools. In a recent BI integration program over the past year, we assessed multiple use cases spanning ship-to-shore and various Azure integrations. Our findings revealed that Confluent Kafka performed exceptionally well, standing out alongside Genesys and Azure Event Hubs. While these three are top contenders, the choice among other tools depends on the specific use case and project requirements.

The customer initially used tools like SMQs, FITRA, and Stream for real-time data processing. However, after our recommendation, Confluent Cloud proved to be a superior choice, capable of replacing these three tools and simplifying their data infrastructure. This shift to a single tool, Confluent Cloud, streamlined their operations, making maintenance and management more efficient for their internal projects.

Regarding real-time data usage, there were challenges with CDC (Change Data Capture) integrations. Specifically, with PyTRAN, we encountered difficulties. We recommended using our on-premises Kaspersky as an alternative to PyTRAN for that specific use case due to issues with CDC store configuration and log reading challenges with the iton components.

We implemented this solution approximately a year and a half ago for significant projects.

I would rate the stability 8 out of 10. 

I would rate the scalability 8 out of 10. 

I haven't personally implemented it. The recommendation was made during a consulting program based on a thorough assessment of the tool's capabilities, connectors, and services. However, there hasn't been direct implementation on-site.

While most of the capabilities meet current organizational needs, the pricing is slightly higher. Companies need to consider this aspect in their decision-making process, given the managed and comparatively higher price point. I would rate it an eight due to its capabilities and CDC integrations. It stands out as a top tool that can replace others for real-time data integration processes. The CIBC capability has the potential to handle EDCs effectively. While I assume the support is good for out-of-the-box challenges, I'm not certain about the cost. Overall, I believe it's a strong choice with a rating of eight.

We had a legacy website collecting user data as they logged into the portal. We wanted to capture that information in Snowflake and store it in a mobile app. We used Apache Kafka on Confluent Cloud for real-time data streaming.

It is a single platform to publish any information on any given topic. In case of huge transactions on the web or mobile apps, it helps you capture real-time data and analyze it.

There could be an in-built feature for data analysis.

We have been using Apache Kafka on Confluent Cloud for a year.

I rate the product’s stability a nine out of ten. There is room for improvement as we need to put some effort into setting it up to develop use cases. 

The product is highly scalable.

The technical support services are good.

The initial setup requires effort to deploy in the cloud or local environments. Once the servers and other prerequisites are ready, it is simple. It took us one month to complete the process. However, the approximate deployment time depends on particular use cases.

We implemented the product in-house.

I recommend Apache Kafka on Confluent Cloud and rate it a ten out of ten.

Overall, I think it's a good experience. Apache Kafka can be quite complex and difficult to maintain on your own, so using Apache Kafka on Confluent Cloud makes it much easier to use it without worrying about setup and maintenance.

The solution is expensive. 

I have been using Apache Kafka on Confluent Cloud, the platform's main product, for about three months.

I haven't experienced any major stability problems or bugs.

The tool is scalable. I've found Apache Kafka on Confluent Cloud to be very scalable. We've been able to scale up the volume of data we're handling without any issues with performance.

Setting up a new cluster on Apache Kafka on Confluent Cloud is pretty easy. You can click a few options, and there's also integration with other tools to create the necessary resources. However, there are some caveats to be aware of. The type of networking setup you choose can impact your functionality and visibility in the platform. For example, if you have a public cluster, you'll see more metadata information in the console than a more restricted network deployment.

Regarding pricing, Apache Kafka on Confluent Cloud is not a cheap tool. The right use case would justify the cost. It might make sense if you have a high volume of data that you can leverage to generate value for the business. But if you don't have those requirements, there are likely cheaper solutions you could use instead.

I rate the solution an eight out of ten.