VMware/Virtual SAN
VSAN
Virtual SAN is a software-based storage solution built into the hypervisor that aggregates the hosts' local storage devices (SSD and HDD) and makes them appear as a single pool of storage shared across all hosts.
Feature Highlights
Virtual SAN provides the following capabilities:
- Automated Storage Management via Intelligent VM based policies
- Dynamic scalability to grow storage and compute dynamically on an as-needed basis
- Integrated with vSphere and managed in vCenter
- Built-in resiliency with protection from multiple hardware failures
- Per-VM with SLA management via intelligent data placement
- Instant storage provisioning without complex workflows
Hardware Compatibility / Setup Requirements
- Minimum 3 vSphere Hosts with storage
- Combination of SAS/SATA HDD & SSD; at least 1 SSD and 1 HDD in every host contributing storage to the cluster
- SAS/SATA RAID Controller: Must work in “pass-thru” or “HBA” mode (e.g. LSI 2008)
- ESXi boot: SD card/USB/SATADOM is preferred (ESXi boot partition and Virtual SAN cannot co-exist on the same SSD/HDD)
- Network: 10G (preferred)
Licensing
vSAN will require seperate licensing from the Hosts and vCenter Server.
"When you enable Virtual SAN on a cluster, you must assign the cluster an appropriate Virtual SAN license before its 60-day evaluation period expires. Just like vSphere licenses, Virtual SAN licenses have per CPU capacity. When you assign a Virtual SAN license to a cluster, the amount of license capacity that is used equals the total number of CPUs in the hosts that participate in the cluster. For example, if you have a Virtual SAN cluster that contains four hosts with eight CPUs each, you need to assign the cluster a Virtual SAN license with a minimum capacity of 24 CPUs."
References:
- vSphere 5.5 Documentation Center - http://pubs.vmware.com/vsphere-55/index.jsp?topic=%2Fcom.vmware.vsphere.vcenterhost.doc%2FGUID-31AADB51-799C-40C2-882C-8D71B31C4BB9.html
Setup
Need at least 3 hosts, each with at least 1 free SSD and 1 free non-SSD. Additional hosts can be added as storage or as compute nodes.
Each SSD / non-SSD needs to be formatted clean, or at least have all partitions removed.
partedUtil get /dev/disks/eui.00000000000000 partedUtil delete /dev/disks/eui.00000000000000 1
Each host will need to have a vmKernel interface configured for vSAN. All of which will need to be on the same subnet.
Command Line
# esxcli vsan Available Namespaces: datastore Commands for VSAN datastore configuration network Commands for VSAN host network configuration storage Commands for VSAN physical storage configuration cluster Commands for VSAN host cluster configuration maintenancemode Commands for VSAN maintenance mode operation policy Commands for VSAN storage policy configuration trace Commands for VSAN trace configuration
Set auto mode:
esxcli vsan storage automode set --enabled="false"
List vSAN storage on local host:
esxcli vsan storage list
Get vSAN datastore name:
esxcli vsan datastore name get
Remove local disk from vSAN datastore:
esxcli vsan storage remove --disk mpx.vmhba2:C0:T1:L0 esxcli vsan storage remove --ssd mpx.vmhba2:C0:T1:L0 esxcli vsan storage remove --uuid 52afa1de-4240-d5d6-17f9-8af1ec8509e5
Leave cluster:
esxcli vsan cluster leave
References:
- vSAN command line | VMware Communities - http://communities.vmware.com/message/2245973
VM Storage Policy
Number of disk stripes per object: (1-12:1)
The number of HDDS across which each replica of a storage object is striped.
A value higher than 1 may result in better performance (for e.g. when flash
reach cache misses need toget services from HDD), but also results in higher
use of system resources.
Default value: 1, Maximum value: 12.
Number of failures to tolerate: (0-3:1)
Defines the number of host, disk or network failures a storage object can
tolerate. For n failures tolerated, "n+1" copies of the object are created and
"2n+1" hosts contributing storage are required.
Default value: 1, Maximum value: 3.
Flash read cache reservation (%): (0-100:0.0)
Flash capacity reserved as read cache for the storage object. Specified as a
percentage of the logical size of the object. To be used only for addressing
read performance issues. Reserved flash capacity cannot be used by other
objects. Unreserved flash is shared faily among all objects.
Default value: 0%, Maximum value: 100%.
Force provisioning: (No-Yes:No)
If this option is "Yes", the object will be provisioned even if the policy
specified in the storage policy is not satisfiable with the resources currently
available in the cluster. VSAN will try to bring the object into compliance if
and when resources become available.
Default Value: No.
Object space reservation: (0-100:0)
Percentage of the logical size of the storage object that will be reserved
(thick provisioned) upon VM provisioning. The rest of the storage object in
thin provisioned.
Default value: 0%, Maximum value: 100%.
Error:
The policy requires 2 replicas with 2 disks each with 0 bytes free each. Only found 0 such disks.
HCL
VSAN HCL - http://vmwa.re/vsanhcl
VSAN HCL - http://www.vmware.com/resources/compatibility/search.php?deviceCategory=vsan
References:
- VSAN and the AHCI controller (hint: not supported!) - http://feedproxy.google.com/~r/YellowBricks/~3/JoubaeDiX_k/
Pricing
$2,495 per CPU or $14,970 for a standard 3 node (2 CPU each) cluster.
VMware VSAN License Calculator | Virten.net - http://www.virten.net/2014/03/vmware-vsan-license-calculator/
VMware Virtual SAN Now Available | VMware Company Blog - VMware Blogs - http://blogs.vmware.com/vmware/?p=2311
VMware Virtual SAN Licensing and Pricing | Virten.net - http://www.virten.net/2014/03/vmware-virtual-san-licensing-and-pricing/
Beta
VMware Communities: Welcome to the vCloud Suite Beta Program - http://communities.vmware.com/community/vmtn/beta/vcloud_suite?view=documents
- VMware Communities: vSAN Workflows for Hosted Beta Only - http://communities.vmware.com/docs/DOC-22252
---
Coming in Beta2 Refresh - http://communities.vmware.com/servlet/JiveServlet/previewBody/22898-102-1-30527/Coming%20in%20Beta2%20Refresh.pdf
Released: Monday, April 29th, 2013.
Issues
storage policy 2 of 3 contributing
Error:
Cannot complete file creation operation. The policy requires 3 hosts contributing storage, only found 2 in the cluster. Failed to create object.
storage policy 2 replicas
Error:
Cannot complete file creation operation. The policy requires 2 replicas with 1 disks each with 0 bytes free each. Only found 0 such disks. Failed to create object.