vsphere_static_160x300
Free Business and Tech Magazines and eBooks
Badges

vexpert_logo_100x57

gestaltitbadge

follow-me-twitter

Subscribe to me on FriendFeed

Comments / DISQUS
Tag Cloud
Feedjit.com
«
»

Trouble pinging multiple NIC ESX host after install

PostDateIcon May 26th, 2008 | PostAuthorIcon Author: Rich Brambley

A common issue after installing ESX servers that do not have all their NICs cabled is that you can not ping the host. For example, say you have an ESX host with 6 1GB network cards – 2 on board and 4 PCI. You would think that cabling the 2 on board cards would cover network connectivity. Unfortunately the order that ESX recognizes the NICs is not determined in a logical, expected order such as on board and then PCI cards. In fact, if you know how ESX determines how to order the network cards please comment and let us all know! In the meantime, here is how to use a few esxcfg- Console commands to make sure the cabled NICs are linked to the vSwitch that has the Service Console PortGroup (where the ip address is assigned).

Determine which NICs are enabled in ESX

#esxcfg-nics -l

displays all NICs and their status

You can see from the screen shot (click to see larger version) that ESX sees one of the PCI e1000 cards as vmnic0 (the first card). The 2 on board Braodcom cards are vmnic1 and vmnic 2, and the first PCI card is vmnic3.

You can also easily tell which vmics are hot.

Determine which vSwitch the Service Console is on and what NICs are liinked

#esxcfg-vswitch -l

In this screenshot I have already added vmnic1, but most likely after a fresh install all you will see is vmnic0 in the Uplinks column for the VM Network and Service Console PortGroups of vSwitch0.

To fix network connectivity you need to have the hot vmnics linked to the vSwitch.

Link enabled NICs to Service Console vSwitch

#esxcfg-vswitch -L vmnic2 vSwitch0

After linking a new vmnic repeat the #esxcfg-vswitch -l command to confirm the new vmnic is linked.

The esxcfg- commands are case sensitive. Repeat the command for each vmnic you want linked to the vSwitch.

Unlink disabled NICs from Service Console vSwitch

#esxcfg-vswitch -U vmnic0 vSwitch0

Once again, #esxcfg-vswitch -l confirms the command worked.

Unlink all vmnics that are not cabled.

Pinging the ESX host should now work! You should not have to restart networking, but if for some reason you need to use

#service network restart

Related Posts

PostCategoryIcon Posted in esx, how to | PostTagIcon Tags: , , , , ,
«
»
  • Lucas.
    Quick comment to why ESX is numbering PCI devices like that. As you know ESX is using RH for its service console. It is fairly old RH running kernel 2.4 series. On the top of that it's using old way of detecting devices - static /dev directory. Using udev would be nicer and solve all these problems. Now back to the problem.
    There is command available on most Linux systems: lspci (I would suggest 'man lspci'). You can get a lots of info out of it. All we need is the info about buses and slots. This is where your hardware vendor comes in. They decide how to organize your system, buses, slots and so on. Note that not only network cards are in your system, there is a lot more, eg. internally connected USB hubs, HBA's etc.
    So the service console when boots up is just walking through PCI list and detecting hardware without remembering ID's of the devices. So if you change devices in slots, next time you reboot the server it may appear that your eth0 is eth6 (or vmnic6) instead of expected eth1 (or vmnic1).
    Why? Moving card to next slot you could move it to different bus, and this bus is in different position on PCI hierarchy. The same situation applies when you add more cards to the server... cards numbering will change as mapping between PCI bus:slot -> device name will change ('shifting' devices)!

    Main answer, ask your hardware vendor how they assigned slots to buses and then reading lspci output you can determine which one is which.
    General rule, lower PCI bus:slot, lower network card device name.

    About 'udev'. It's latest version of 'low level device manager' in Linux, it has capability to remember devices ID's so even if you move device from slot to slot it will always remain as eg. eth0. Furthermore if you remove device from the server it will remember it as well so if by any chance you put it back it will be given eth0 again. This may or may not cause problems as you can imagine.

    Hope this help a bit.
  • Lucas,

    Thanks for Red Hat lesson! Hopefully newer versions of ESX will utilize 'udev'.
blog comments powered by Disqus
Hyper9 Cowabunga
Support VM /ETC
Support VMETC.com

Support VMETC.com

Special Offers

Save 10% on select VMware products!

@rbrambley tweets
Advertisements
Facebook
VMETC
VMETC
Promote Your Page Too
VMTN Roundtable Podcasts
Subscribe



Add to Google Reader or Homepage
Subscribe in NewsGator Online
Add to netvibes
Add to Plusmo

License
Creative Commons License
www.vmetc.com by Richard Brambley is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

Contact Us | Terms of Use | Trademarks | Privacy Statement
Copyright © 2009 VM /ETC. All Rights Reserved.

Powered by WordPress and WordPress Theme created with Artisteer.