vsphere_static_160x300
Free Business and Tech Magazines and eBooks
Badges

vexpert_logo_100x57

gestaltitbadge

follow-me-twitter

Subscribe to me on FriendFeed

Comments / DISQUS
Feedjit.com

vSwitch With Multiple VMKernel Portgroups for vSphere iSCSI Round Robin MPIO

vSphere has introduced several new features for storage performance enhancement. Most of the new features build on already accepted vSwitch standards and designs. An important example is the new Round Robin MPIO path policy for VMFS LUNs. However, based on what is the common vSwitch design today, the new iSCSI configuration needed for Round Robin multi-pathing may cause some admins to look twice.

I was motivated to write this post by 2 recently published storage vendor documents that both recommend the same basic iSCSI vSwitch with Round Robin MPIO configuration: create a single iSCSI vSwitch, assign 2 physical NICs, and then create as many as 8 VMKernel Portgroups each with their own ip address. The documents I am referring to are:

To give a visual of the recommended configuration (in case you are still doing a double take) here are screen shots of the configured vSwitch from:

NetApp’s PDF

netapp iscsi config

Dell’s PDF

equallogic iscsi switch config

Read the document(s) for the specific steps, requirements, and options to configure each vendor’s iSCSI storage with ESX 4 Round Robin multi-pathing, but the reoccurring process found in both documents goes like this (modeled after the table of contents of the Dell PDF):

Configure vSwitch and Enable Jumbo Frames
Add iSCSI VMkernel Ports
Assign 2 physical NICs to iSCSI vSwitch (at least)
Associate VMkernel Ports to physical NICs and override load balancing policy with dedicated pNICs
Enable VMware iSCSI Software Initiator
Bind VMkernel Ports to iSCSI Software Initiator
Connect ESX to iSCSI Storage
Enabling VMware Native Multipathing – Round Robin
Create VMFS Datastores

Why use multiple iSCSI VKernel Portgroups in vSphere? Vaughn Stewart explains in his blog post vSphere on NetApp Best Practices Released:

“iSCSI has made major gains in vSphere and in TR-3749 we detail how to aggregate the bandwidth of multiple network links. This solution allows customers with GbE based storage networks to achieve {omitted} the throughput needs of their most demanding applications without having to upgrade the storage network to FC, FCOE, or 10 GbE.”

Let me know if there are other whitepapers or technical notes published by iSCSI storage vendors other than Equallogic and NetApp that recommend the same configurations.

Related Posts

  • I have this working on the standard vSwitch configuration, but have been unable to get it working using dVswitch.

    Have you given this a crack yet ?

    Regards
    Brian
  • Brian,

    I have not tried with a vDS. The linked Equallogic PDF states as an assumption that a vDS is not in use for their vSwitch example. The NetApp PDF may also say the same. I should've mentioned the config was not for a vDS in the post. Thanks for bringing it up!
  • justingrote
    The main reason for the multiple vKernel ports is for Multipathing. By having multiple vkernel ports, they can show up as multiple paths in the Storage (go to manage paths on your storage and it'll look very similar to fibre channel). By setting all your LUNs to round-robin, they will more-or-less distribute across both NICs, allowing you to get multiple-gigabit storage performance out of your system (but never more than 1 gigabit per LUN unless you do Round-Robin Multipathing).

    I'm not sure why Dell talks about adding 6 vkernel ports for only two NICs, you should only have 1 vkernel port per NIC, you're not really getting any benefit out of those additional ports that I can see. You also get no benefit out of having more NICs connected than you do iSCSI ports on your SAN, as your SAN ports become the bottleneck in that situation.

    The multivendor iSCSI post does a great job of explaining this in detail, even though it was written for 3.5:
    http://blogs.vmware.com/vmtn/2009/06/a-multiven...
  • Aidan
    Can the same be accomplished with multiple vswitches with a single kernel on each?
  • Pasi Kärkkäinen
    Dell talks about multiple vmkernel ports per NIC because that way they get to utilize all of the eth-interfaces of the Equallogic arrays. Each separate iSCSI session from each vmkernel port will get redirected to the 'best' interface in the Equallogic group (of possibly many arrays).

    If you're doing random IO then you're not limited by the gigabit ethernet interface speed, but disk/spindle speeds and controllers.
  • Thanks for the info on Dell iSCSI
blog comments powered by Disqus
Hyper9 Cowabunga
Support VM /ETC
Support VMETC.com

Support VMETC.com

@rbrambley tweets
Advertisements
VMTN Roundtable Podcasts
Subscribe



Add to Google Reader or Homepage
Subscribe in NewsGator Online
Add to netvibes
Add to Plusmo