What needs improvement with Honeycomb.io?

The only complaint I have is that even though we are on a paid tier where we are paying one hundred thirty dollars per month, we are still lacking the amount of ingestion we have to do. It counts each and every event. We cannot blindly send the performance metrics and other things directly to Honeycomb Enterprise as it will raise our bills. They have some buffer protection, but that only happens twice or three times a month. After that, the number of events will get billed accordingly. On the negative side, dashboarding is not that great. I have used solutions like Signoz which are open-source and Grafana. In those platforms, where we have open-source solutions where we are sending in the metrics, we are getting a complete dashboard and a complete solution around it where we can analyze everything in one shot. That is not something which is really great with Honeycomb Enterprise. These kinds of solutions in Honeycomb Enterprise feel very difficult to build because I think it is an over-engineered product in terms of metrics. For traces, I have to give it to them. They have done an excellent job. The only thing is the UI is not that great for Honeycomb Enterprise. That is something where it is not very appealing when I am looking into it. It is more of an engineered thing. I am an engineer and I understand this. You cannot make the UI very beautiful. However, if you use its peers, such as DataDog or Signoz, there are very interactive dashboards and other things. The dashboard is not that great, honestly, in Honeycomb Enterprise. There is a cold start problem because we were using something called Lambda in AWS. The cold start issues basically occurred because we had not provided proper RAM to our Lambda function. Because of that, the Boto3 boot up was taking a lot of time. That was caught during these tracing sessions when we added the traces. We were able to identify that the Boto3 startup, the init part of Boto3, was taking more than twelve seconds or ten seconds to boot up. That is basically a very huge latency when it comes to a SaaS product. Those are the major things. One thing they should do is discontinue Honeybee CLI. Right now, no one is using Honeycomb Enterprise like Honeybee CLI as per my understanding. Very few people are using it, and only POC kind of things are happening on this. People are shifting to OpenTelemetry. They should actually discontinue that product. It does not make much sense because they are trying to bring a dependency on people who are using that and they want to lock them into Honeycomb Enterprise because once the code is telemetered, it is very difficult to change it again. However, now with cloud and everything we have, it is very easy to make those changes. There is no mode there. If there are engineers around it who are managing those things, they can actually shift to a better place where they can provide more value around it. Because people will eventually move to OpenTelemetry where they will set up their OpenTelemetry system, one OTel collector, and then OTel collector will send everything to Honeycomb Enterprise. People are moving out of these enterprise solutions mostly and they are using OpenTelemetry day in and day out. Most of the architects I talk to in terms of these things always talk about this. Personally, I have also not used Honeycomb Enterprise CLI. I have read a lot of things and articles around it. They are actually maintaining it very well it seems, but I think they should discontinue it. That is just my opinion based on my observations. I may be wrong, I am not sure. This is what I have observed during communication channels, followed by the usage I have done, and followed by people I talk around. All have similar opinions.

I have used better tools, I would say. I would not say that I prefer Honeycomb Enterprise as much. I have used Dynatrace, and I found it more comprehensive, and AppDynamics and other tools. These tools can also provide good information, but I find other tools better. Most of the products, I would say, such as Dynatrace or AppDynamics or New Relic, are targeting this microservices market. I think Honeycomb Enterprise can have something very dedicated for microservices because there is an explosion in the migration from monolithic to microservices. If Honeycomb Enterprise can create a stable solution which is easy to use and which gives additional value and helps for faster debugging with microservices, they can certainly gain market share from others. Tracing is already there. I just wish that these tools are a bit less cryptic. These tools sometimes get quite cryptic for new users. The less cryptic they can be made, that can help these tools. Another thing is that for microservices, when you have multiple microservices installed, that is also required. There are tools where you install on a single microservice, but then these microservices interact with multiple microservices. That kind of picture, I have seen that in AppDynamics; they do give a picture showing that a particular request which arrived here had interaction with these other third-party services or microservices and databases. That is what we need. That is what performance engineers and SREs need to see for each request, where it spent the entire time; how many other services or databases it interacted with and what took more or less time, and if there is a sequence, it should highlight that also. Was it parallel or if, for instance, a call to service A and then a call was made to a database, or a call to service A and a database were in parallel, that kind of information.

The major thing that's missing from Honeycomb Enterprise is AI compatibility. As far as I know, it's not really a text-based or code-based tool. It's more of a UI right now, which before this paradigm shift where everyone is using AI agents to work, was pretty useful. However, after this paradigm shift, I think it's really important for a tool like this to be AI-friendly. Honeycomb Enterprise is really lagging behind in this area, and I don't know how they could manage that. For us it was sometimes pretty slow using Honeycomb Enterprise. I don't know if that can be improved. Although we might use the paid solution or self-host it somewhere because of privacy concerns. Maybe that's not Honeycomb Enterprise's fault. However, the main thing is that I think everything should very hard aim for the direction of being AI compatible because every engineer, or most engineers now use AI to code. If something is not easy to work with AI agents, that will stay in the past.

I asked very specific questions to Mr. Pell about consideration of code security scenarios in pattern design and rules, specifically that tuned with OWASP Top 10. I believe addition of code security focus can be a value-add, though the way Grit architecture is designed and how it works, it is and may not become an alternative choice of code security solutions. Rather, it must be treated as a powerful supplementary tool that augments the existing code security solutions (such as Snyk or Checkmarx) in a DevSecOps or Secure DevOps environment. Anyone interested in learning more on this front or have queries, can get in touch with me for a consulting.

We faced some OpenTelemetry metrics lost between the communication from the service and the Honeycomb.io. I can't say if this is a Honeycomb.io issue or if there are some limitations in OpenTelemetry. Alerts are very helpful in Honeycomb.io, but we don't usually merge because we can compare queries with queries for making alerts. We can make alerts based on static numbers, which may block us from building alerts that could be generic enough or could be serviced.

The process of log scraping gets delayed on Honeycomb.io. At times, it gives false alerts to the application team. It would be good if Honeycomb.io could integrate with third-party tools or paid services, which can make it easy for us to alert our teams more frequently.