Nutanix Cloud Infrastructure (NCI) Reviews

Originally posted at http://vcdx133.com/

If you have just heard about Hyper-Converged infrastructure and Nutanix, read my Tech101 – Nutanix Virtual Computing Platform article.

First point, Nutanix has just announced the name change of their Virtual Computing Platform (VCP) to Xtreme Computing Platform (XCP) with the release of Acropolis and their new “Invisible Infrastructure” paradigm during the Miami .Next 2015 conference.

If you are not a Nutanix Customer, Partner or Employee, it is going to be very challenging getting your hands on their technology, KB articles and documentation. They have released the Nutanix Community Edition to get you started, but it will provide limited value for their Enterprise platform.

Nutanix need to update their policies on who can access their binaries, KB articles and documentation. Particularly if they want non-customers to investigate and become familiar with their technology. I have been informed they are trying to fix this. For example:

• If you take the Nutanix Virtual Computing Platform Administration 4.1.1 course for NPP, some of the training material links to Nutanix documents that require a valid support contract.
• You must be a Nutanix Partner or Employee to download the Foundation and Phoenix binaries.

The Nutanix documentation (HTML, EPUB or PDF) you want to get your hands on for NOS 4.1:

• Setup Guide (publicly available)
• Advanced Setup Guide
• Field Installation Guide
• Hardware Administration and Reference
• Upgrade Guide
• Web Console Guide (publicly available)
• vSphere Administration Guide (publicly available)
• Acropolis Virtualisation Administration Guide (publicly available – previously “KVM Administration Guide”)
• Hyper-V Administration Guide (publicly available)
• NOS Advanced Administration Guide
• Nutanix Cluster Check (NCC) Guide
• Command Reference (publicly available)
• API Reference (HTML only – publicly available)

General observation, Nutanix documentation is very well written with an elegant font and format; very easy to read and understand.

The Nutanix Portal areas (HTML, EPUB or PDF) you want to checkout (if you have access):

• Knowledge Base Articles
• End of Life Bulletins (& Policy)
• Field Advisories
• Security Advisories
• Support (for Support Request creation and management)

General observation, Nutanix support is very prompt and helpful, even for low priority requests.

The Nutanix hardware, software, binaries and licences you will need to start your journey:

• Nutanix XCP block (any model), even as a PoC for a few months
• Foundation (2.1 at time of writing) with the embedded Phoenix images for ESXi, KVM and Hyper-V
• Nutanix Installer Package (2.1.3 at time of writing)
• Prism Central (4.1.3 at time of writing – separate Hyper-V and ESXi versions)
• Professional/Ultimate Edition Licences for your Cluster (for advanced features)
• Nutanix Cluster Check (NCC – 1.4.1 at time of writing)
• Nutanix Community Edition (with whitebox hardware or nested virtual)
• Nutanix supported version of ESXi ISO image from my.vmware.com
• Nutanix supported version of Hyper-V ISO image from microsoft.com
• Nutanix KVM Hypervisor upgrade bundle
• Various Tools and Firmware (eg. SCVMM Clone AddIn, SCOM Management Pack etc.)

What can you do to get started:

• Register for Nutanix Training and get certified (free and paid options)
• Register for the Nutanix Next Community to access Blogs, Activities and Forums
• Register for the Nutanix Community Edition and deploy it in your lab (nested or physical)
• Contact your local Nutanix office and request a PoC for your company
• Work with a Nutanix SE to witness and participate in the deployment process in your PoC
• Become a Nutanix Customer
• Join a Nutanix Partner
• Join Nutanix

What is most valuable?

How has it helped my organization?

What needs improvement?

For how long have I used the solution?

What do I think about the scalability of the solution?

How are customer service and technical support?

Which solution did I use previously and why did I switch?

How was the initial setup?

What about the implementation team?

What's my experience with pricing, setup cost, and licensing?

What other advice do I have?

Originally posted at http://vcdx133.com

In my opinion, Hyper-Converged Infrastructure (HCI) is the future of Private and Hybrid Cloud infrastructure. If you are designing a greenfield data center and HCI is not on your list for serious consideration, then it should be.

With that being said, every technology has its Pros and Cons; nobody rides for free. The advantages of HCI are many and outweigh the disadvantages, here is what you need to watch out for:

• People – If your server virtualization, network and storage teams operate in silos and are constantly at war with each other, then HCI is probably not for you at this time. Successful HCI projects are built upon a very close collaboration between these teams. In fact, it makes more sense to merge these three teams into one “Enterprise Infrastructure” team. It is also very important to cross-skill these team members and let them evolve into “Enterprise Architects”, “Enterprise Administrators” and “Enterprise Operators”. However, make sure you keep your Backup/Recovery/Archive responsibilities separate (see RBAC point below).
• Data Center Facilities – A data center full of legacy, 3-tier infrastructure is not the same as one packed with HCI. The resource density ratio is around 4-8 to 1 depending upon your current legacy infrastructure. You need to design for 25+kW racks with a matching cooling system. If you use a traditional, legacy data center (designed for 5-8kW per rack), then you will have problems down the road (hot spots and power exhaustion).
• Switch Fabric – By moving to HCI, you need a scalable LAN fabric that provides non-blocking throughput for East-West traffic. Legacy network switch design (Core – Distribution – Aggregation – Access layers) is not going to cut it for large scale HCI, which is optimised for North-South traffic. You may get away with it initially, however you will need plans to migrate to a non-blocking leaf and spine switched LAN. HCI has made Fiber Channel infrastructure obsolete, but the same principles that drove SAN design now apply to your LAN with the move to IP storage.
• Controller VM – The storage processor of legacy storage arrays has now become a virtual appliance running on the host itself. Make sure your administration/operations staff, Standard Operating Procedures and monitoring systems understand the importance and give it the respect it deserves. The current version of NOS with ESXi still allows vSphere administrators to modify the CVM (Nutanix Acropolis does not allow this for CVMs with Nutanix KVM). For example, an untrained vSphere Administrator powers off all Nutanix Controller VMs and reduces the RAM from 24GB to 8GB to provide additional resources for adding new VMs across the entire cluster.
• Role Based Access Control – When I consider failure scenarios for Business Continuity and Disaster Recovery, my nightmare risk is not a natural disaster, but the disgruntled rogue administrator, who has all of the keys to the kingdom, taking out every system. With HCI and the “Enterprise Administrator”, this risk is compounded. So it is very important to separate the administration/operations responsibilities for operational data and backup/recovery/archive. This way if either one is wiped out across all data centers, you still have the other to recover from. Apply this concept to physical data center security as well.
• Data Locality and the Working Set – “Data Locality” is the amount of local storage resources (capacity and performance) presented via the Controller VM to the Hypervisor for serving your virtual workloads. The “Working Set” is the active footprint (capacity and performance) of those virtual workloads. As an organization (architects, administrators and operators), you need to make sure that the “Working Set” of your virtual machines have the optimum fit with respect to the “Data Locality” of each node in your HCI solution. Nutanix XCP has many different models, you need to make sure you select the correct fit for your needs.
• Processes and Procedures – In my opinion, Hyper-Converged Infrastructure (HCI) is the future of Private and Hybrid Cloud infrastructure. If you are designing a greenfield data center and HCI is not on your list for serious consideration, then it should be. With that being said, every technology has its Pros and Cons; nobody rides for free. The advantages of HCI are many and outweigh the disadvantages, here is what you need to watch out for:
• People – If your server virtualization, network and storage teams operate in silos and are constantly at war with each other, then HCI is probably not for you at this time. Successful HCI projects are built upon a very close collaboration between these teams. In fact, it makes more sense to merge these three teams into one “Enterprise Infrastructure” team. It is also very important to cross-skill these team members and let them evolve into “Enterprise Architects”, “Enterprise Administrators” and “Enterprise Operators”. However, make sure you keep your Backup/Recovery/Archive responsibilities separate (see RBAC point below).
• Data Center Facilities – A data center full of legacy, 3-tier infrastructure is not the same as one packed with HCI. The resource density ratio is around 4-8 to 1 depending upon your current legacy infrastructure. You need to design for 25+kW racks with a matching cooling system. If you use a traditional, legacy data center (designed for 5-8kW per rack), then you will have problems down the road (hot spots and power exhaustion).
• Switch Fabric – By moving to HCI, you need a scalable LAN fabric that provides non-blocking throughput for East-West traffic. Legacy network switch design (Core – Distribution – Aggregation – Access layers) is not going to cut it for large scale HCI, which is optimized for North-South traffic. You may get away with it initially, however you will need plans to migrate to a non-blocking leaf and spine switched LAN. HCI has made Fiber Channel infrastructure obsolete, but the same principles that drove SAN design now apply to your LAN with the move to IP storage.
• Controller VM – The storage processor of legacy storage arrays has now become a virtual appliance running on the host itself. Make sure your administration/operations staff, Standard Operating Procedures and monitoring systems understand the importance and give it the respect it deserves. The current version of NOS with ESXi still allows vSphere administrators to modify the CVM (Nutanix Acropolis does not allow this for CVMs with Nutanix KVM). For example, an untrained vSphere Administrator powers off all Nutanix Controller VMs and reduces the RAM from 24GB to 8GB to provide additional resources for adding new VMs across the entire cluster.
• Role Based Access Control – When I consider failure scenarios for Business Continuity and Disaster Recovery, my nightmare risk is not a natural disaster, but the disgruntled rogue administrator, who has all of the keys to the kingdom, taking out every system. With HCI and the “Enterprise Administrator”, this risk is compounded. So it is very important to separate the administration/operations responsibilities for operational data and backup/recovery/archive. This way if either one is wiped out across all data centers, you still have the other to recover from. Apply this concept to physical data center security as well.
• Data Locality and the Working Set – “Data Locality” is the amount of local storage resources (capacity and performance) presented via the Controller VM to the Hypervisor for serving your virtual workloads. The “Working Set” is the active footprint (capacity and performance) of those virtual workloads. As an organization (architects, administrators and operators), you need to make sure that the “Working Set” of your virtual machines have the optimum fit with respect to the “Data Locality” of each node in your HCI solution. Nutanix XCP has many different models, you need to make sure you select the correct fit for your needs.
• Processes and Procedures – Moving from legacy, 3-tier infrastructure to HCI is a big change, so do not underestimate or ignore the imperative to update all of your processes and procedures. HCI will simplify and improve your infrastructure, consequently simplifying your operational procedures, but you will need to change how you do things with respect to people, process and technology.

What is most valuable?

How has it helped my organization?

What needs improvement?

For how long have I used the solution?

What do I think about the stability of the solution?

What do I think about the scalability of the solution?

How are customer service and technical support?

Which solution did I use previously and why did I switch?

How was the initial setup?

What about the implementation team?

What's my experience with pricing, setup cost, and licensing?

Which other solutions did I evaluate?

What is most valuable?

How has it helped my organization?

What needs improvement?

For how long have I used the solution?

What was my experience with deployment of the solution?

What do I think about the stability of the solution?

What do I think about the scalability of the solution?

How are customer service and technical support?

Which solution did I use previously and why did I switch?

How was the initial setup?

What about the implementation team?

What other advice do I have?

What is most valuable?

How has it helped my organization?

What needs improvement?

For how long have I used the solution?

What was my experience with deployment of the solution?

What do I think about the stability of the solution?

What do I think about the scalability of the solution?

How are customer service and technical support?

Which solution did I use previously and why did I switch?

How was the initial setup?

What about the implementation team?

What was our ROI?

What's my experience with pricing, setup cost, and licensing?

What other advice do I have?

Originally posted at vcdx133.com.

This post provides a Tech101 breakdown of the Nutanix Virtual Computing Platform. If you have heard the buzz-words “Nutanix”, “Web-Scale” and “Hyper-Convergence” and want to learn more about it, this post is for you.

The Nutanix Virtual Computing Platform (VCP) is a Hyper-Converged Infrastructure solution. Termed the “iPhone” of infrastructure, what is the big deal?

Short answer: Ease of use, fast deployment, linear scaling of performance with capacity and richness of product features.

Why should you care? If you live in the world of monolithic and vertically scaled storage, you may have experienced the initial, massive cost of purchasing your storage solution with a 5 year ROI. Then comes the honey-moon period where you experience amazing performance as workloads are moved to the new array. Well before the 5 year ROI mark, you will probably experience performance degradation as the capacity of the array is consumed. Which then initiates discussions with your storage vendor about how you need to spend another boatload of money adding additional controllers, SSDs and disks to the array.

Furthermore, think about how long it takes for the storage team, the server team and the network team to agree on a particular design for a project and the time it takes to order, deliver, install, configure and test that solution. Have you ever asked yourself, “There must be an easier way to do this?”

Enter Hyper-Convergence, where you can now buy individual blocks of infrastructure at an entry price and quickly scale-out your solution (performance and capacity) one unit at a time as you grow.

What is it? Nutanix has taken the “Web-Scale” infrastructure model of Google and Facebook and made it available to the Enterprise. You buy a Nutanix Block (rack mounted chassis) that contains Nutanix Nodes (hosts), storage, power and cooling. This creates a Nutanix Cluster (Software-Defined Storage), which are comprised of Controller Virtual Machines (CVM).

How does it work? The Controller Virtual Machines connect directly to the physical SCSI Controllers on each node (Pass-Through mode) and the CVMs are configured to create a “Nutanix Distributed File System (NDFS) Cluster” (storage pool) via the 10GbE network. All of the pooled storage is classified as local or remote from the point of view of each CVM and VM data is protected based upon a Replication Factor (RF) policy of two or three, which replaces the concept of RAID. RF-2 means a local and remote copy is maintained and RF-3, a local and two remote copies of said data.

Where is the magic? Since each node has its own Controller VM, SSDs and HDDs, NDFS can provide linear scaling of capacity and performance for locally stored data. When a CVM goes down, a remote CVM will takeover serving data I/O to the local hypervisor. When the local CVM comes online, it resumes service. This fundamental concept is what allows Nutanix to provide a “rolling upgrade” of the Nutanix Operating System (NOS). What is the caveat? During periods of CVM failure, local workloads that access remotely stored data will experience degraded performance until the local CVM is restored.

The major components of the Nutanix VCP solution are:

• Nutanix Node – a single host with CPU, RAM, SCSI Controller and Network that resides in a Nutanix Block
• Nutanix Block – rack mounted chassis that contains nodes, storage, power and fans
• Prism – User Interface and APIs
• Prism Central – aggregator of separate Prism Clusters to a single UI
• Cluster Virtual Machine – Software-Defined Storage that supports NFS, iSCSI or SMB3
• CVM operating system is known as “NOS” – Nutanix Operating System
• CVM file system is known as “NDFS” – Nutanix Distributed File System
• Hypervisor – VMware vSphere, Microsoft Hyper-V and KVM supported
• OEM Hardware – Supermicro and Dell with PCIe SSD and HDD
• Licencing – Starter, Pro & Ultimate

It comes in a number of OEM Hardware flavours (Dell and Supermicro) with the following uses:

• ROBO, Test & Development – NX-1000, Dell XC 720xd A5
• EUC/VDI – NX-3000, NX-7000 (GPU & PCoIP Offload), Dell XC 720xd B5/B7
• Compute Intensive – NX-3000, Dell XC 720xd B7
• Data Intensive – NX-8000, Dell XC 720xd C5/C7
• Business Critical & Mission Critical Apps – NX-9000 (All Flash)

The major features of the Nutanix Operating System (current version NOS 4.1):

• One-Click Upgrades of NOS and Hypervisor (rolling upgrade)
• Data Protection Cloud Connect (to AWS)
• Data At Rest Encryption
• Metro Availability (<5ms RTT)
• Deduplication
• Compression
• Data Tiering
• Snapshots and Clones
• Advanced Data Protection

Alternatives

• Server-Side Flash-Cache Acceleration – VMware vFRC, PernixData FVP, EMC XtremSF/XtremCache, etc.
• Converged Infrastructure Solutions – NetApp FlexPod, VCE vBlock, HP ConvergedSystem, Dell, etc.
• Other Hyper-Converged solutions – VMware EVO:Rail, Maxta, SimpliVity OmniCube, Scale Computing, Pivot3, etc.

Nutanix is hyper-converged infrastructure, which is currently a major concern in the world of IT. New players are appearing in hyper-converged infrastructure like Simplivity, VMware EVO: Rail, EMC ScaleIO, and so forth.

Waht is Hyper-Converged?

Hyper-Converged is the solution to combine Computer (Server), Storage and Networks in one solution or in one hardware. This solution will reduce complexity of the management and also the price. So like in Nutanix, the 2 U Server, in which there are four independent servers (nodes), who each have 2 Socket CPU, RAM, 2x10GbE, 2x1GbE, 6 Disks. Bringing the total in 2U Server (Series NX-3450) there are 80 CPU Cores, 2TB RAM (assuming 512GB per node), 8x10GbE and 16TB SATA Row, Row 3.2TB SSD.

Storage in Nutanix (Converged Storage), reduces the complexity of the storage layer, because it does not need LUN, Volume, Zoning, etc as when we configure FC Storage. Storage in Nutanix is NFS, NFS is the benefit of reducing the overhead of the disk groups and LUNs.

Nutanix Architecture

Inside there is a term Nutanix nodes and blocks.Block is a 2U rack mount chassis in which there are four nodes.Hardware Nutanix is OEM Super Micro or Dell.

In each of these nodes are Nutanix Controller VM (CVM). CVM is a storage appliance or controllers which use VMware Direct Path for direct access to the LSI Disk Controller. This is what allows the CVM to control the disks directly without passing through the hypervisor (ESXi or Hyper-V or KVM). Nutanix does not use RAID and no RAID Controller therein. Nutanix Distributed Files System (NDFS) algorithm is owned (proprietary) by Nutanix system for distributing files or replicating data to each Node. NDFS is a similar technology as used by Googles Files System, Facebook and other Web-Scale Technology.

Hypervisor is installed in the SATA DOM (Disk on Motherboard) module. So there is a flash chip in the motherboard 64GB per node (read article unboxing Nutanix). CVM is a VM running on the hypervisor, which has the configuration of 8 vCPUs and 16-24GB RAM.CVM has two vNICs that one vNIC to connect to the production network and one vNIC to connect to a private vSwitch. Private vSwitch VMkernel adapters have and currently do not have physical network adapter (Punic). Private network used for communication from the host to the CVM for mounting NFS datastore of Nutanix Storage Cluster.CVM uses public vNIX for communication between the CVM in other nodes and also for monitoring cluster plus replicate data.

The figure below is a diagram per Node:

And the image below is a diagram per Cluster Nutanix:

And some of the questions often asked are:

1. What happens when CVM failure in one node? Autopathing will redirect IO to the CVM in other nodes in the same cluster.
2. What happens when a disk failure on one node?Nutanix Storage Cluster will "sustain" the current single disk failure, and the system will redistribute data through Cluster.When the rebuild or redistribution is complete it will Cluster Healthy back.This process is very fast (ms).
3. What happens when one node failure?The process is almost the same as point 2, the data will redistribute within the Cluster.Only this process Nutanix uses MapReduce to speed up the process.If we have three nodes in one cluster, then only allowed one node failure.Therefore, at least 3 nodes required to form 1 Nutanix Cluster.

Management Console

Management Console in Nutanix is PRISM.PRISM is a web interface as shown below.

The first time we log in, then the initial menu as shown above.Visible top left is the ESXi hypervisor is installed, if you use Microsoft Windows, the look is Hyper-V.Then also seen versions of ESXi, below is the total storage capacity, a total of VMs (VMs 230), total node (5) and total blocks (2).In the middle column visible performance monitoring IOPS, bandwidth, CPU and Memory.And the rightmost column is the Events and Alerts.

Top left corner there is a drop down menu (Home), then I moved to the Storage menu Diagram View.It will appear as shown below.

Now we move to the VM menu, which will give us the details of the VM is running in the cluster.Seen there total VM, total CPU and Memory.Then there is also a report by IOPS, Latency IO, Memory Usage and CPU Usage.

Then we move to the VM Table, it is seen in detail per-VM, as shown below. If we look at the information that there is, then I think this function together with VMware vCenter Ops, which did not exist just Forecast Analysis. Here we can search the VM and then perform analysis per-VM performance based on CPU, Memory, IOPS, Latency, or Network.

And now we move to the Storage menu Diagram.Seen all information relating to such Usage Storage, IOPS, bandwidth IO, IO Latency. And in the top right corner, there are two menus of "Container" and "Storage Pool".

Storage Pool, is the union of all disks that exist in all Nodes.Container is as Datastore. We can make some Container in 1 Storage Pool.As with traditional storage, we can make some LUNs in 1 Disk Group.

Interestingly here is Nutanix without RAID but using NDFS, no volume or LUNs. For example, we will create one Container, then I click menu in the top right corner Container, the menu will appear below.

Interestingly, we we would like to mount the new container to multiple hosts / nodes, we just need to do the check as shown below.Imagine if we use the SAN, must do Zoning first, then setting WWN ID to each of the hosts, and so on.

And we can compress or dedup the container that we have made, and only one click then completed as shown below. Interesting is it not it?

And no less, that is build-in replication. We can do replication N-Way means can perform replication to 3rd place (A-> B-> C) upto 6th place (A-> B-> C-> D-> E-> F). Which greatly facilitates that we can do per-VM based and fixed per block-level replication. Imagine in traditional storage, replication must be done LUN to LUN.So if there are 50-VM per LUN, then total all 50 VM will be replicated to the LUN in the DR Site.Replication in Nutanix, can be integrated with VMware SRM.

Likewise reviews Nutanix, although not as complete but could be an additional complete our knowledge. And we will come back to a Nutanix detailed discussion at a later time.

Thanks to Nutanix for allowing me to "box-test" Nutanix lab in Singapore.
I would like to thank the team for greater Nutanix Reviews their help in providing the access to Reviews their lab and willingness to answer questions during our process.