What is our primary use case?
We're in the middle of building out a persistent cache layer using Couchbase. That's across multiple international regions, but we have other instances where we're using it for data stores and some for its analytics features.
How has it helped my organization?
There are a couple of benefits. It's cheaper than, for instance, Redis for us to implement a caching solution.
The other benefit that I've noticed is the ability built into the tool to do data replication across the enterprise. If I've something in AsiaPac, something in NAM, and something in EMEA, I don't have to set up the infrastructure, buy the software, and buy the hardware outside of my database to be able to replicate between the three regions. That also applies to disaster recovery. It's built-in, and I just specify the bucket that I want to copy, and it works. So, price, extensibility, and scalability are the benefits of this solution.
The other benefit that we've noticed is speed-to-market. I began a PoC in October of last year, and there were four sprints in the PoC. By the end of the year, we were only expecting to be done with the first sprint, but we were already through the first sprint and about to complete the second sprint. The implementation time is very less. The biggest challenge we had was with indexes.
What is most valuable?
The whole stack is valuable, but the portion of the stack that we're finding really handy is the analytics engine because that allows us to take and pre-build views. One of the use cases that we're doing currently is for reporting, and in some cases, the users are having to come up with complex joins and complex. We call them formulas. They are not just joins. There are filters, transformations, and those sorts of things, which take a considerable period of time to perform. So, we use the analytics engine to pre-build that data for them so that when they do the reports, they're just selecting the fields of the data that they need as opposed to having to build the data that they need. We can present it in a single view, and they can then just select A, B, C, D, E, F, and G.
What needs improvement?
Their security model leaves something to be desired. Their indexes could use some work, but once we found out what was going on with them, it became second. So, the biggest one so far is integrating with an external security model and having a security model that you can use. I know there's a performance hit with it, but if possible, I'd like to be able to go down to potentially a column-level instance. I'd like to be able to set up entitlements based on that. I can do that currently, but it's a little bit convoluted in the way I have to do it. It can become, from a large enterprise standpoint, a bit unwieldy in the way I have to manage it to get it to work.
The other thing that they could really improve is their documentation. Their online documentation or just documentation in general is pretty bad. The online documentation is very confusing. It's not very intuitive to find information. There are no white papers on best practices for doing things. As much as everybody rags on IBM, considering the various different software products that IBM has, their documentation, for the most part, is very good. You can find many white papers, and it's very easy to find things, whereas, with Couchbase, the whole documentation setup is very confusing. It's not usable, for the most part.
They should support native S3 ingestion into the default bucket or into the system. Currently, when I want to look at S3 stuff, I use the analytics engine, and that works fine. However, when I want to use S3 as a landing point to be able to load into the default Couchbase—not use in the analytics engine but just to import into Couchbase—there's no native way of having Couchbase talk to S3 to get JSON documents out and loaded into the data store. So, I've to use a third-party product called Data Sync to be able to pull stuff out of S3. The integration options aren't as robust as they probably need to be. They only have about two.
It's easy to deploy. Where the challenge comes in is when you start putting data in, doing the indexes, and doing the integration with systems. Integration is one of their weakest points. Natively, there should be a wide range of integration options to be able to get data in. Once you get the data in, it's really easy to get the data out. It's very simple. It's a JDBC, etc. It's just that the integration to get the data in can be a challenge.
For how long have I used the solution?
In my portion of the company, we've been using it for about six months, but we've had it for a couple of years in other portions.
What do I think about the stability of the solution?
It's very stable for the most part. There were some edge cases that we did, which were more our fault than the product's. There are some bugs that we found that Couchbase is addressing in version 7.11, but for the most part, it's stable. We haven't had any major issues with stability.
What do I think about the scalability of the solution?
It's very scalable. When I go to production with this PoC, ultimately, I will have 150 terabytes worth of active data that will be accessed by the solution. Couchbase has told us we're one of the larger or largest instances of Couchbase out there. So, it's very scalable. There are some challenges with how you do that scalability, but once you figure out the way you have to structure your data to do that, it's very scalable. You can't just throw 150 terabytes worth of data into a single Couchbase instance and expect it to perform. It just won't. So, you have to look at how to structure your data. You have to look at how you implement your clusters. You don't want every single one of your stacks—the analytics engine, the query engine, the index engine, and the data engine—running in the same nodes. You want to look at your performance and maybe put your analytics engine on one node, your real-time query or your natural language query on another node, and your index service on another. You need to look at what performance you want and how your data is structured, but once you get down to how to do that, it's very scalable. With the PoC, I got a total of three people who are administering to see that it's going to lead to 150 terabytes. So, we don't need a huge admin staff.
Across the enterprise, there are probably 150,000 people, and most of them are just application users who are querying it to get reporting data or getting data for their dashboards. Ultimately, they're just users of the application that we put in front of Couchbase to get the data out. There are a couple of hundred admins. Most of them are administrators. There are some people who are the equivalent of DBAs who onboard new data sets, set up indexes, etc. We have a couple of hundred across the entire enterprise.
It's being used throughout the enterprise, and our usage of it's scaling up considerably. Most of it has been a line of business here, a line of business there, etc. With this PoC, we are moving towards having this across the whole enterprise for our data caching as well as data reporting. So, we are ramping this up. We are, so far, very happy with what we're seeing and what we're getting.
How are customer service and support?
So far, they've been okay. They haven't been the best. We've had to play what I call the 800-pound gorilla a couple of times to get focus on our problems, but once I played the 800-pound gorilla initially, they've been very responsive to our issues and helped us resolve issues. We found bugs, and they're working on solving those bugs. Of course, we're a big bank implementing a big solution. So, we're going to get a little bit of a different experience than a mom-and-pop credit union, but for the most part, they're about as typical as any support organization.
Which solution did I use previously and why did I switch?
We also use Redis, but we are getting rid of it. The reason why we're getting rid of Redis is that it's not scalable. It's not a true data cache and does not handle large volumes of data for reporting very well. It doesn't handle complex data very well. It's literally built as a data cache only, and for programming, and it acts as thus. So, if we went in and did a PoC where we loaded 900 megabytes into the Redis cache, and name, address, city, state, and zip were the tags, that data took nine hours to load into the cache. After we literally changed the tag names for the data to A, B, C, D, E, F, and G, that nine hours of run took nine minutes to run, but we can't expose that data for the enterprise and the business to use by having somebody look at a site and say, "oh, an A is a name, and a B is a city." So, we're phasing out Redis because of that and because of the cost.
We're still using Hadoop. We are going to phase out Hadoop, but that's more of a long-term project. That's simply because it doesn't deliver what it promised. It's very easy to get data into Hadoop, but it's almost impossible to get data out of Hadoop for reporting in a real-time fashion. It's more along the lines of a batch-level system, and there are multiple limitations to that.
We are still using Mongo, though not as much. Mongo is a special thing. We use it, but we want to limit it simply because there are a lot of problems. From the SQL to get stuff out of Mongo to the persistence layer to the replication isn't easy. It's very good for dedicated tasks, but when it comes to an enterprise stack, it's not very good at what we need it to do.
How was the initial setup?
Installing the software and getting it up to a base level and running takes about a day. It's very easy to figure out how to deploy it.
What was our ROI?
We've seen an ROI. We're able to reduce our costs. We've been able to improve our speed-to-market. Barring the integration work, we've been able to get rid of Redis and the cost of that. So, we've seen an ROI.
What's my experience with pricing, setup cost, and licensing?
It seems very reasonable. It's a lot cheaper than Redis, but we've got an enterprise license. So, it's about normal. It's not outrageous in price as far as we've seen.
From Couchbase, there's no additional fee as far as I'm aware, but when you're integrating, there's an additional fee because a lot of times, they don't have an integration stack. They literally have two ways of getting data loaded in. It can be done either via the API or via a bulk load, and there's a way for them to read from Kafka, but instead of having to stream to Kafka, if you want to get it in or set up a batch file interface, your choices are limited as to how to integrate. So, you end up going to vendors like CData and buying their Data Sync application to be able to get data out of, for instance, an Oracle database, or natively out of Hadoop. That's what we ended up having to do because of the S3 issue. So, there are integration costs, which are not part of the core, that you might have to face.
What other advice do I have?
Engage Couchbase-based technical staff soon and often. They're your best source of information. Don't struggle with trying to find it in the documentation because the online documentation is not intuitive. The other thing is to look at your data and see how your data is structured. Understand your data, and understand what you're trying to do from the get-go. Bring somebody on board who has experience implementing Couchbase. That has been a real help for us. We've got an on-staff SME with extensive background, but we also have a lot of communication with Couchbase itself. Couchbase presales have been good at helping us through things. So, use the resources that are there and engage them, and know your data—how it's structured and how you want to use it.
I'd rate it an eight out of ten. I don't like giving tens because nothing is ever perfect. There is also the difficulty of having to engage third parties outside of Couchbase to do the integration stuff, but overall, I'm very pleased and very happy with them.
Which deployment model are you using for this solution?
On-premises
Disclosure: My company does not have a business relationship with this vendor other than being a customer.