What needs improvement with Confluent?

I recommend that Confluent should improve its solution to keep up with competitors in the market, such as Solace and other upcoming tools such as NATS. Recently, there has been a lot of buzz about Confluent charging high fees while not offering features that match those of other tools. They need to improve in that direction by not only reducing costs but also providing better solutions for the problems customers face to avoid frustrations, whether through future enhancement requests or ensuring product stability. The cost should be worked on, and they should provide better solutions for customers. Solutions should focus on hierarchical topics; if a customer has different types of data and sources, they should be able to send them to the same place for analytics. Currently, Confluent requires everything to send to the same topic, which becomes very large and makes running analytics difficult. The hierarchy of topics should be improved. This part is available in MQ and other products such as Solace, but it is missing in Confluent, leading many in capital markets and trading to switch to Solace. In terms of stability, it is not the stability itself that needs improvement but rather the delivery semantics. Other products offer exactly-once delivery out of the box, whereas Confluent states it will offer this but lacks the knobs or levers for tuning configurations effectively. Confluent has hundreds of configurations that application teams must understand, which creates a gap. Users are often unaware of what values to set for better performance or to achieve exactly-once semantics, making it difficult to navigate through them. Delivery semantics also need to be worked on.

People do not appreciate that Confluent is pushing us more towards Teams because they want to use a true Microsoft Word-type format where we can format our sentences better, instead of just saying 'paragraph' or just having headers. They want to be able to change fonts, change the size of the wording to make it easier to format, and have better color coding of the documents. We need a better way of creating the hierarchies within the pages. Sometimes it's very difficult to find information in the tree-structured setup. Word documents and dealing with SharePoint are easier from a project management point of view because that's where many of us are moving towards now. Confluent has potential if it would be easier to cut and paste things into and to export information into industry-standard documents. We run into formatting problems when we cut and paste things in or when we're pulling things out. Tables are difficult to cut and paste or bring back in. Confluent has fallen behind in being the tool of the industry. It's taking second place to things such as Word and SharePoint and other office tools that are more dynamic and flexible than Confluent. We still see its place, but it's not as significant as it was 10 years ago.

I am not very impressed by Confluent. We continuously face issues, such as Kafka being down and slow responses from the support team. The lack of easy access to the Confluent support team is also a problem. If it were a homegrown solution, it might have been easier to manage.

One area we've identified that could be improved is the governance and access control to the Kafka topics. We've found some limitations, like a threshold of 10,000 rules per cluster, that make it challenging to manage access at scale if we have many different data sources.

Confluence is easy to use and modify. However, sometimes there are too many pages. We have to reorganize the folder or parent account. Since everyone can create a page, the same knowledge might be created in multiple places by different people. This leads to redundancy and makes it difficult to find information. It's not centralized. So it could be more user-friendly and centralized. A way to reduce redundancy would be helpful. It's very easy to use, so everyone can create knowledge. But it would be good to synchronize and organize that information a bit better. Another improvement would be in Confluence search. You can search for keywords, but it's not like AI, not even ChatGPT or OpenAI. It would be nice to get more relevant or organized answers. If you're outside the company, you just get some titles containing the keyword you input. But if Confluence were like a database, you could input something and get a well-organized search offering from multiple pages.

Areas for improvement include implementing multi-storage support to differentiate between database stores based on data age and optimizing storage costs, as well as enhancing the offset management tool for easier management of offset topics, reprocessing, and message filtering based on predicates.

Confluent's price needs improvement.

More instructions should be included. It should guide the people to build the site in an easy way because when you ramp up conference spaces, people start making an effort with, domain providers. Easy project management tools could be added.

One of the challenges lies in Access Management, particularly when you need to share documents with an external audience, such as customers. Currently, it is more suited for in-house usage, and it doesn't readily support collaborative document sharing with external parties. As a workaround, many resort to using other platforms like Slack to create user IDs and facilitate interactions with external stakeholders, even integrating tools like Canvas for this purpose. Its integration with Microsoft documentation is not as robust as desired. There is a limitation when it comes to seamlessly importing Microsoft documents into Confluent pages, which can be inconvenient for users who frequently work with Microsoft Office tools and need to transition their content to Confluent.

The formatting aspect within the page can be improved and more powerful.

The Schema Registry service could be improved. I would like a bigger knowledge base of other use cases and more technical forums. It would be good to have more flexible monitoring features added to the next release as well.

The product should integrate tools for incorporating diagrams like Lucidchart. It also needs to improve its formatting features. We also faced issues while granting permissions.

This solution is different from other products we have used. It could be improved by including a feature that automatically creates a new topic and puts failed messages. When we moved to this solution, we noticed that failed messages do not automatically go into Dead Letter Queue (DLQ), which we had when we used Azure. We want something similar to be included in Confluent Kafka. The Azure Service Bus background should be improved where the message processing and messages fail and automatically go into a DLQ. In Confluent Kafka, there is no concept of DLQ, so if any messages fail, you have to handle it on your end and push it to another topic. But in Azure Service Bus, it happens automatically if any message delivery fails. It puts it in a DLQ. We didn't bother about anything on the code front, but with Confluent Kafka, there is no concept of DLQ, and you have to manage when a message fails to deliver. We have to push it to another topic manually. This should have been on the Confluent Kafka, and if anything fails, it should automatically move the message to another topic that has been created. Still, we have to do this using our code explicitly.

The solution could have an extra plugin or upgrading feature. In addition, it could have more integration with different platforms and be more compatible.

Currently, in the early stages, I see a gap on the security side. If you are using the SaaS version, we would like to get a fuller, more secure solution that can be adopted right out of the box. Confluence could do a better job sharing best practices or a reusable pattern that others have used, especially for companies that can not afford to hire professional services from Confluent. Another area of improvement is on the metric side. The metrics are integrated within the documentation of the metrics and implementation of that metric. This ties back to the requirement of best practices. How to use metrics in the best possible way out of the box. If you are using Confluent on-prem or cloud-based where you are packaging it within your own cloud infrastructure, there is a lot of documentation for that making it easier. However, if you are using only the SaaS platform, there is less support from Confluent Cloud. We are still understanding the additional features but we have seen a gap in the features of the fully managed connector. They are lesser fully managed connectors that are relevant to our business requirements so we have requested those fully managed connectors be developed and released within the Confluent Hub portfolio of connectors. Confluence requires documentation regarding how to manage connectors that are not fully managed, including what parameters are required for each processing connector. The documentation that is provided is not easy to understand.

They should remove Zookeeper because of security issues.

From the control center perspective, there is a lot of room for improvement in the visualization. They're now aggressively pushing the Cloud version which is good, but there are still a lot of features missing there.

The beginner who doesn't know how to work on HTML will struggle as when you create spaces in the Confluent, if you want to have some meeting notes or anything else, you need to know HTML and which HTML tags to include. In my earlier organization, people used to struggle in adding the HTML tags to the Confluent. However, if a person is from an IT background, and he or she knows how to add the tags and browse product documentation, for them it is easy. A new user must go through the few training sessions for Confluent. In Confluent, there could be a few more VPN options. I remember in my previous company, I had an issue in adding a query where I could pull down information from Jira. The product, the team productivity, the team timesheets, et cetera. A few of those plugins must be more advanced in Confluent. The plugin options need to be improved. I don't know whether we have something in Confluent where we can have a plugin for support teams. If not, that would be ideal.

It could have more themes. The themes in the version I'm using are very limited; they offer two to three themes. They should have more reporting-oriented plugins as well. It would be great to have free custom reports that can be dispatched directly from Jira.