Red Hat Ceph Storage Reviews

We use this as an OpenStack storage back-end of Nova/Glance/Cinder.

• High reliability with commodity hardware

• There is no cost for software

I would like to see better performance and stability when Ceph is in recovery.

Latency increases abruptly when conducting recovery. This impacts the upper application.

It is OK to add a node or a disk, but it may impact the latency of read/write of the application which is running.

The level of technical support is acceptable.

The initial setup was straightforward without much effort.

People can try the vanilla Ceph, if they are confident with their technical skills.

We evaluated VMware vSAN.

It is easy to set up.

Ceph has helped our organization to provide a Software Defined Storage solution in our private cloud.

The ability to provide block storage and object storage from the same storage cluster is very valuable for us.

We are using Ceph as back-end storage for our OpenStack cloud. Ceph provides:

• Block storage for storing the OpenStack images or VM templates
• Block storage for OpenStack Cinder volume service
• Block storage for OpenStack Nova VM compute service boot volumes
• Object storage for the OpenStack Swift service

Without Ceph, we would have ended up with at least two storage systems: One for block storage and another for providing Swift Objectstore.

The other big advantage is that Ceph is free software. Compared to traditional SAN based storage, it is very economical.

Ceph does not deal very well with, or takes a long time to recover from, certain kinds of network failures and individual storage node failures.

I believe the community that supports Ceph is working on this. They will be providing solutions to improve these issues in the newer versions, like Jewel and in the future with technologies like BLU Store and RDMA.

Stability in a normal operating environment is satisfactory. Improvements would come from providing better data re-balancing algorithms when the storage cluster is expanded. Currently, cluster expansion is a user impacting process.

We have not noticed any issues with scalability. In fact, when more nodes/disks were added to the cluster, it improved performance due to its nature of being a native object store.

We are using the open source version. However, there seem to be many vendors, in addition to RedHat, who sell or provide support for Ceph.

We used traditional fiber based SAN storages before we started using Ceph. The main reasons for switching to Ceph were:

• Ability to provide block as well as object storage
• Open source system
• Scalability: Performance actually improves as we scale the cluster bigger

The initial setup required a lot of research and learning to understand Ceph storage's underlying technology. Once we had the right understanding and configurations, it was pretty straightforward.

However, this is not a traditional storage solution. It may not be straightforward for storage administrators, but easier for cloud administrators with good Unix/Linux knowledge.

The key things to consider while deploying Ceph, especially for block storage (also known as RBD) are:

• Use a higher number of disks to get more IOPS. (Ceph is a copy-on-write storage, so usage is less of a worry than providing the right number of IOPS.)
• Use SSD journal disks to improve write performance. (In fact, with the price of SSD drives coming down, use all SSD or NVME+SSD configurations - more IOPS makes a better solution.)
• Use SSD for Ceph MONITOR nodes
• Use networking speeds of at least 20 Gbits/sec or more since this is a network based storage on all clients as well as Ceph nodes. As you move to full SSD or NVME disks, the networking needs to match up.
• Select the right CRUSH map and Placement Group numbers based on your storage pool size and node distribution in the data center.

Pricing/licensing depends on what kind of internal knowledge or expertise exists in your organization about Ceph.

If you don't have the expertise, choose the right partner or vendor based on proven expertise by the vendor in large production environments.

We did not evaluate other storage solutions. We spent the time understanding Ceph better to provide a stable solution.

Ceph is open source and there are large organizations running huge Ceph clusters which have published blogs on how they deployed Ceph.

Do your research based on the lessons learned from these users of Ceph to decide on which configuration and architecture to use for Ceph.

As organizations move to Linux container based technologies and container orchestration frameworks (especially Kubernetes), Ceph is still relevant as it provides integration into these future technologies to provide block storage for them as well.

It's ultimately all about IOPS. When a failure occurs CEPH tries to 'rebalance' data on the surviving nodes which can consume a lot of IOPS affecting client IO. If there's not enough IOPS or fast data rebalancing, it can take a lot of time to rebalance data. Some of this can be improved with faster networks and faster drives like SSD or flash drives (which people can implement right now in older versions of CEPH), some of the improvements will come from how CEPH writes data using BlueStore and replicate or rebalance data between OSD nodes using RDMA (which may become stable for users in newer versions).

The most valuable feature is the stability of the product. 

The storage capacity of the solution can be improved. 

I have been using this solution for ten years. 

I would rate stability a nine out of ten. 

We use this solution in our internal app. Around fifty people in our organization use this solution. I would rate the scalability a six out of ten. 

We solve a problem by searching on Google instead of contacting technical support. We never communicated with the support team. 

Previously we used a product that we developed in-house.

The initial setup was easy. It takes less than an hour to install this product. 

To perform maintenance, security experts and developers are needed. About three to four people are required for maintenance.

I would recommend others to use this product if they get the chance. I would rate the product a nine out of ten.