Over the past decade VMware has changed the way IT is provisioned through the use of Virtual Machines, but if we want a truly Software-Defined Data Centre we also need to virtualise the storage and the network.
For storage virtualisation VMware has introduced Virtual SAN and Virtual Volumes (expected to be available in 2015), and for network virtualisation NSX. In this, the first of a three part series, we will take a look at Virtual SAN (VSAN).
So why VSAN?
Large Data Centres, built by the likes of Amazon, Google and Facebook, utilise commodity compute, storage and networking hardware (that scale-out rather than scale-up) and a proprietary software layer to massively drive down costs. The economics of IT hardware tend to be the inverse of economies of scale (i.e. the smaller the box you buy the less it costs per unit).
Most organisations, no matter their size, do not have the resources to build their own software layer like Amazon, so this is where VSAN (and vSphere and NSX) come in – VMware provides the software and you bring your hardware of choice.
There are a number of hyper-converged solutions on the market today that can combine compute and storage into a single host that can scale-out as required. None of these are Software-Defined (see What are the pros and cons of Software-Defined Storage?) and typically they use Linux Virtual Machines to provision the storage. VSAN is embedded into ESXi, so you now have the choice of having your hyper-converged storage provisioned from a Virtual Machine or integrated into the hypervisor – I know which I would prefer.
Typical use cases are VDI, Tier 2 and 3 applications, Test, Development and Staging environments, DMZ, Management Clusters, Backup and DR targets and Remote Offices.
VSAN Components
To create a VSAN you need:
- From 3 to 32 vSphere 5.5 certified hosts
- For each host a VSAN certified:
- I/O controller
- SSD drive or PCIe card
- Hard disk drive
- 4 GB to 8GB USB or SD card for ESXi boot
- VSAN network – GbE or 10 GbE (preferred) for inter-host traffic
- Layer 2 Multicast must be enabled on physical switches
- A per socket license for VSAN (also includes licenses for Virtual Distributed Switch and Storage Policies) and vSphere
The host is configured as follows:
- The controller should use pass-through mode (i.e. no RAID or caching)
- Disk Groups are created which include one SSD and from 1 to 7 HDDs
- Five Disk Groups can be configured per host (maximum of 40 drives)
- The SSD is used as a read/write flash accelerator
- The HDDs are used for persistent storage
- The VSAN shared datastore is accessible to all hosts in the cluster
The solution is simple to manage as it is tightly integrated into vSphere, highly resilient as there is zero data loss in the event of hardware failures and highly performant through the use of Read/Write flash acceleration.
VSAN Configuration
The VSAN cluster can grow or shrink non-disruptively with linear performance and capacity scaling – up to 32 hosts, 3,200 VMs, 2M IOPS and 4.4 PBs. Scaling is very granular as single nodes or disks can be added, and there is no dedicated hot-spare disks instead the free space across the cluster acts as a “hot-spare”.
Per-Virtual Machine policies for Availability, Performance and Capacity can be configured as follows:
-
Number of failures to tolerate – How many replicas (0 to 3 – Default 1 equivalent to a Distributed RAID 1 Mirror)
-
Number of disk stripes per object – The higher the number the better the performance (1-12 – Default 1)
-
Object space reservation – How Thickly provisioned the disk is (0-100% – Default 0)
-
Flash read cache reservation – Flash capacity reserved as read cache for the storage object (0-100% – Default 0)
The Read/Write process
Typically a VMDK will exist on two hosts, but the Virtual Machine may or may not be running on one of these. VSAN takes advantage of the fact that 10 GbE latency is an order of magnitude lower than even SSDs therefore there is no real world difference between local and remote IO – the net result is a simplified architecture (which is always a good thing) that does not have the complexity and IO overhead of trying to keep compute and storage on the same host.
All writes are first written to the SSD and to maintain redundancy also immediately written to an SSD in another host. A background process sequentially de-stages the data to the HDDs as efficiently as possible. 70% of the SSD cache is used for Reads and 30% for Writes, so where possible reads are delivered from the SSD cache.
So what improvements would we like to see in the future?
VSAN was released early this year after many years of development, the focus of the initial version is to get the core platform right and deliver a reliable high performance product. I am sure there is an aggressive road-map of product enhancements coming from VMware, but what we would like to see?
The top priorities have to be efficiency technologies like redirect-on-write snapshots, de-duplication and compression along with the ability to have an all-flash datastore with even higher-performance flash used for the cache – all of these would lower the cost of VDI storage even further.
Next up would be a two-node cluster, multiple flash drives per disk group, Parity RAID, and kernel modules for synchronous and asynchronous replication (today vSphere Replication is required which supports asynchronous replication only).
So are we about to see the death of the storage array? I doubt it very much, but there are going to be certain use cases (i.e. VDI) whereby VSAN is clearly the better option. For the foreseeable future I would expect many organisations to adopt a hybrid approach mixing a combination of VSAN with conventional storage arrays – in 5 years time who knows how that mix will be, but one thing is for sure the percentage of storage delivered from the host is only likely to be going up.
Some final thoughts on EVO:RAIL
EVO:RAIL is very similar in concept to the other hyper-converged appliances available today (i.e. it is not a Software-Defined solution). It is built on top of vSphere and VSAN so in essence it cannot do anything that you cannot do with VSAN. Its advantage is simplicity – you order an appliance, plug it in, power it on and you are then ready to start provisioning Virtual Machines.
The downside … it goes against VMware’s and the industries move towards more Software-Defined solutions and all the benefits they provide.
Disclosure: My company has a business relationship with this vendor other than being a customer. We are Partners with VMware.
