Red Hat Ceph Storage Reviews

We’re using Ceph storage to provide allocation space to instances (VMs) in an OpenStack environment.

The product spawned a new vision of storage deployment, as well as a strong interest in reusing equipment and increasing ROI. We have some legacy servers that can be associated with this structure. With Ceph, we can rearrange these machines and reuse our investment.

• We can reuse servers of any type and include them in the deploy of the solution.
• radosgw and librados provide a simple integration with clone, snapshots, and other functions that aid in data integrity.

In the deployment step, we need to create some config files to add Ceph functions in OpenStack modules (Nova, Cinder, Glance). It would be useful to have a tool that validates the format of the data in those files, before generating a deploy with failures.

No current issues. Almost all our difficulties were related to implementation. After that, everything ran well.

Most times that I have implemented Ceph, I have used some type of deployment tool, like RDO (Red Hat Director). With these tools, I can make the environment scale in or out without issues. An attention point is looking for a journal and disk separation on the YAML file.

It is possible that you only have support if you partner with a vendor like Red Hat. However, you can find many articles in forums or GitHub.

We had no previous solution. My first contact with ephemeral storage is through Ceph.

My first deployment was complex, connecting Ceph with all OpenStack modules, but that was  before I was testing Ceph and doing installations manually and with hard coding.

It is not a complex implementation, but you need to look into all the structure requirements and OSDS division.

Most of time, you can get Ceph with the OpenStack solution in a subscription as a bundle.

No.

I rate it at nine out of 10. It is a product which is constantly undergoing improvements.

We use it as cloud storage, connected to OpenNebula cloud system for one of our customers. System includes 56 Supermicro nodes that are specially configured to be used as a hyperconverged system. Same nodes used for storage and virtualization.

It opens doors for completely open-source cloud. No monthly charges, no revenue share policy. It just works.

When going open-source, there is actually not much of a choice. Ceph was chosen to maintain exact performance and capacity characteristics for customer cloud.

Ceph is not a mature product at this time. Guides are misleading and incomplete. You will meet all kind of bugs and errors trying to install the system for the first time. It requires very experienced personnel to support and keep the system in working condition, and install all necessary packets.

The system needs some polishing to be stable enough for a production environment.

CEPH is scalable to almost infinity.

We didn’t use Red Hat services, due to bad experience with them in the past. They usually play email ping pong, while you solve the problems yourself.

None. This was a pilot project and it has worked out.

It is complex. Before the system is built, all your technical staff must research as much about CEPH technology as they can.

We tried Scality. It is the perfect solution, but was out of budget.

Think twice.

Cloud. 

• Speed of storage
• Flexible

• RADOS
• Swift
• S3
• Replication
• Compression

Please create a failback solution for OpenStack replication and maybe QoS to allow guaranteed IOPS.

Everything is perfect.

Ceph has allowed us to deploy block storage for our users in a reliable, performance, and easy to manage fashion without vendor lock-in.

By being open source, Ceph is not tied to the whim or fortunes of any one vendor. The community of Ceph code contributors and admins is large and active. Ceph’s ability to adapt to varying types of commodity hardware affords us substantial flexibility and future-proofing.

Routing around slow hardware.

Ceph is more stable than many proprietary solutions.

Ceph is a scale-out solution designed to be free of scale limitations common when using proprietary storage solutions. Ceph will continue to scale meeting our needs for years to come.

Red Hat’s technical support is valuable, but with Ceph, one can often do well with community resources.

I have used traditional storage on much smaller scales: ZFS, SVM, VxVM, and NetApp. They do not fit the use case and demands of a growing Cloud infrastructure.

Initial setup is straightforward with automation tools.

Not applicable.

Ceph is de facto for mixed modality of cloud storage.

Pick up a copy of this excellent book: Learning Ceph - Second Edition.

We use this as an OpenStack storage back-end of Nova/Glance/Cinder.

• High reliability with commodity hardware

• There is no cost for software

I would like to see better performance and stability when Ceph is in recovery.

Latency increases abruptly when conducting recovery. This impacts the upper application.

It is OK to add a node or a disk, but it may impact the latency of read/write of the application which is running.

The level of technical support is acceptable.

The initial setup was straightforward without much effort.

People can try the vanilla Ceph, if they are confident with their technical skills.

We evaluated VMware vSAN.

It is easy to set up.

Ceph has helped our organization to provide a Software Defined Storage solution in our private cloud.

The ability to provide block storage and object storage from the same storage cluster is very valuable for us.

We are using Ceph as back-end storage for our OpenStack cloud. Ceph provides:

• Block storage for storing the OpenStack images or VM templates
• Block storage for OpenStack Cinder volume service
• Block storage for OpenStack Nova VM compute service boot volumes
• Object storage for the OpenStack Swift service

Without Ceph, we would have ended up with at least two storage systems: One for block storage and another for providing Swift Objectstore.

The other big advantage is that Ceph is free software. Compared to traditional SAN based storage, it is very economical.

Ceph does not deal very well with, or takes a long time to recover from, certain kinds of network failures and individual storage node failures.

I believe the community that supports Ceph is working on this. They will be providing solutions to improve these issues in the newer versions, like Jewel and in the future with technologies like BLU Store and RDMA.

Stability in a normal operating environment is satisfactory. Improvements would come from providing better data re-balancing algorithms when the storage cluster is expanded. Currently, cluster expansion is a user impacting process.

We have not noticed any issues with scalability. In fact, when more nodes/disks were added to the cluster, it improved performance due to its nature of being a native object store.

We are using the open source version. However, there seem to be many vendors, in addition to RedHat, who sell or provide support for Ceph.

We used traditional fiber based SAN storages before we started using Ceph. The main reasons for switching to Ceph were:

• Ability to provide block as well as object storage
• Open source system
• Scalability: Performance actually improves as we scale the cluster bigger

The initial setup required a lot of research and learning to understand Ceph storage's underlying technology. Once we had the right understanding and configurations, it was pretty straightforward.

However, this is not a traditional storage solution. It may not be straightforward for storage administrators, but easier for cloud administrators with good Unix/Linux knowledge.

The key things to consider while deploying Ceph, especially for block storage (also known as RBD) are:

• Use a higher number of disks to get more IOPS. (Ceph is a copy-on-write storage, so usage is less of a worry than providing the right number of IOPS.)
• Use SSD journal disks to improve write performance. (In fact, with the price of SSD drives coming down, use all SSD or NVME+SSD configurations - more IOPS makes a better solution.)
• Use SSD for Ceph MONITOR nodes
• Use networking speeds of at least 20 Gbits/sec or more since this is a network based storage on all clients as well as Ceph nodes. As you move to full SSD or NVME disks, the networking needs to match up.
• Select the right CRUSH map and Placement Group numbers based on your storage pool size and node distribution in the data center.

Pricing/licensing depends on what kind of internal knowledge or expertise exists in your organization about Ceph.

If you don't have the expertise, choose the right partner or vendor based on proven expertise by the vendor in large production environments.

We did not evaluate other storage solutions. We spent the time understanding Ceph better to provide a stable solution.

Ceph is open source and there are large organizations running huge Ceph clusters which have published blogs on how they deployed Ceph.

Do your research based on the lessons learned from these users of Ceph to decide on which configuration and architecture to use for Ceph.

As organizations move to Linux container based technologies and container orchestration frameworks (especially Kubernetes), Ceph is still relevant as it provides integration into these future technologies to provide block storage for them as well.

It's ultimately all about IOPS. When a failure occurs CEPH tries to 'rebalance' data on the surviving nodes which can consume a lot of IOPS affecting client IO. If there's not enough IOPS or fast data rebalancing, it can take a lot of time to rebalance data. Some of this can be improved with faster networks and faster drives like SSD or flash drives (which people can implement right now in older versions of CEPH), some of the improvements will come from how CEPH writes data using BlueStore and replicate or rebalance data between OSD nodes using RDMA (which may become stable for users in newer versions).

The most valuable feature is the stability of the product. 

The storage capacity of the solution can be improved. 

I have been using this solution for ten years. 

I would rate stability a nine out of ten. 

We use this solution in our internal app. Around fifty people in our organization use this solution. I would rate the scalability a six out of ten. 

We solve a problem by searching on Google instead of contacting technical support. We never communicated with the support team. 

Previously we used a product that we developed in-house.

The initial setup was easy. It takes less than an hour to install this product. 

To perform maintenance, security experts and developers are needed. About three to four people are required for maintenance.

I would recommend others to use this product if they get the chance. I would rate the product a nine out of ten.