Joint VMware and NetApp Best Practices for Running VI3 on IP Based Storage #TA2784
This will be another live blog session. Look for updates to this post every 15 minutes. I’m curious if any new best practices or insights will be mentioned that compliment the new vNetworking and vStorage pieces of the VMware VDC-OS.
Well, starting the session with the same disclaimer about the forward looking statements confirms my hopes! On the other hand, the forward statements could be new NetApp technologies. Either case would be cool!
A disclaimer before you read my notes: I have limited storage administration experience. Take my notes for what they are and understand the source (me). Help me out if I state something wrong.
This talk will be primarily on the NFS storage protocol. “NFS might be the best kept secret for virtualization”. To support this claim the speaker is talking about areas to consider when comparing storage protocols. Performance, Ease of Management, and High Availability are the main categories to consider.
Performance
- Scalability – How do you measure scalability? The speaker suggests the best place to measure scalability as at the ESX host level. By adding ESX hosts to a cluster you are scaling the I/O requests to the shared storage. A slide now shows the relative throughput comparison of ESX 3.5 with fiber channel storage performance as the baseline. The bar graph shows NFS and ISCI as being very close to FC in performance. When showing relative CPU utilization of the same protocols, the bar graph shows iSCSI and NFS being significantly higher.
Management
- NFS management benefits – when you create a VM on NFS storage it is thin provisioned by default. You can expand or decrease the NFS volume size on the fly. You can back up and restore whole VMs, or files within VMs. You can us snapshots (array or VMware). De-duplication works great with NFS
High Availability
This will be the part of the session that talks about the best practices?
- First design a redundant physical network for IP storage.
- Load Balancing options – within Data ONTAP there are VIF modes to support load balancing. Just as with networking be sure to match the proper physical switch options with the mode selected
- Need multiple datastores to get more aggregate throughput due to one connection made per datastore. This is true for all storage protocols.
-
How to accomplish multiple physical paths simultaneously and with scalability for future?
- Cross-stack Etherchannel – must have switches that have this capability and multiple storage IPs are needed. Without cross-stack etherchannel ESX will require two VMkernel ports on different IP subnets. Load balancing is achieved by selecting a different primary adapter in the vSwitch for each VMKernel portgroup connecting to the filer.
- Surviving Link Failure – multiple physical switches and multiple NetApp vif interfaces solve this
Best Practices
- Use dedicated switches or VLANs
- Use fast NICs (1GB or higher)
- Avoid over subscription issues
- Select “No” for NIC teaming failback
- Do not place sap space on other storage – do not place the vswap on VMFS. Follow best practices and keep the swap on the NFS storage.
- Match link aggregations on both sides of links. ESX supports static only so match this setting on the physical switch. Match the load balancing policies across the infrastructure. Use IP hash on ESX, the switch and the Filer.
- 10 GB is fully supported for IP storage
- Jumbo Frames is experimental. – but it works
- TOE – TCO Offload Engine –not supported in ESX and it is disabled when using VIF. The recommendation is to use Jumbo Frames instead
NFS Best Practices
- Mount the datastores exactly the same way on all hosts (UUID changes if hostname/fqdn/ip is not used consistently)
- Disable iSCSI on undesired storage ports
- Put the VMkernel port only on the storage network
- Use preferred paths to force traffic to storage network.
Conclusions
NFS is a viable, mature highly available choice for virtualization. “What protocol should I run my VMs on?” is the wrong question. It should be “which virtualization solutions enables me to leverage multiple protocols best?”. Performance implications at the ESX host are minimal between the different storage protocols.
Oh well, nothing really “new” was discussed in this session.
Related Posts
Tags: best practices, design, esx, netapp, NFS, storage, virtualization, vmware, vmworld2008
September 17th, 2008 at 8:25 am
[...] VMworld 2008 Tuesday Opening Keynote Q&A with Paul Maritz VirtualCenter Directions #TA3807 Joint VMware and NetApp Best Practices for Running VI3 on IP Based Storage #TA2784 Re-architecting Data Protection Processes with Data Deduplication and Virtualization Technologies [...]
September 22nd, 2008 at 4:14 pm
[...] Joint VMware and NetApp Best Practices for Running VI3 on IP Based Storage #TA2784 by Rich Brambley [...]
September 24th, 2008 at 8:15 am
The “Joint VMware and NetApp Best Practices for Running VI3 on IP Based Storage #TA2784” presentation and document state “TOE - TCO Offload Engine - not support in ESX and is disabled when using VIF…” which contradicts “What’s New in VMware Infrastructure 3: Performance Enhancements.” Which is it? Will NetApp support TOE using VIF and if so, when?
October 17th, 2008 at 11:23 am
[...] place swap space on non IP storage.” IP storage could mean iSCSI here, of course, but in the VM /ETC write up of TA 2784, Rich Brambley’s notes state: “Do not place [swap] space on other storage – do not [...]