Comments on: Trouble pinging multiple NIC ESX host after install http://vmetc.com/2008/05/26/trouble-pinging-multiple-nic-esx-host-after-install/ Go Green with Virtualization. Go UGLY Green with vmetc.com. Wed, 17 Mar 2010 10:29:58 +0000 http://wordpress.org/?v=2.9.1 hourly 1 By: Restore Connectivity To ESXi 4 Management Network After Adding Second NIC | VM /ETC http://vmetc.com/2008/05/26/trouble-pinging-multiple-nic-esx-host-after-install/comment-page-1/#comment-4165 Restore Connectivity To ESXi 4 Management Network After Adding Second NIC | VM /ETC Sun, 07 Mar 2010 13:59:14 +0000 http://vmetc.com/?p=392#comment-4165 [...] a little alien when an administrator is used to the Service Console of ESX Classic. Specifically, troubleshooting remote connectivity to the ESX management interface after adding a second NIC involves local esxcfg-vswitch commands on ESX, but with ESXi the esxcfg [...] [...] a little alien when an administrator is used to the Service Console of ESX Classic. Specifically, troubleshooting remote connectivity to the ESX management interface after adding a second NIC involves local esxcfg-vswitch commands on ESX, but with ESXi the esxcfg [...]

]]> By: Rich http://vmetc.com/2008/05/26/trouble-pinging-multiple-nic-esx-host-after-install/comment-page-1/#comment-657 Rich Mon, 02 Jun 2008 15:54:04 +0000 http://vmetc.com/?p=392#comment-657 Lucas, Thanks for Red Hat lesson! Hopefully newer versions of ESX will utilize 'udev'. Lucas,

Thanks for Red Hat lesson! Hopefully newer versions of ESX will utilize ‘udev’.

]]> By: Lucas. http://vmetc.com/2008/05/26/trouble-pinging-multiple-nic-esx-host-after-install/comment-page-1/#comment-656 Lucas. Mon, 02 Jun 2008 12:36:32 +0000 http://vmetc.com/?p=392#comment-656 Quick comment to why ESX is numbering PCI devices like that. As you know ESX is using RH for its service console. It is fairly old RH running kernel 2.4 series. On the top of that it's using old way of detecting devices - static /dev directory. Using udev would be nicer and solve all these problems. Now back to the problem. There is command available on most Linux systems: lspci (I would suggest 'man lspci'). You can get a lots of info out of it. All we need is the info about buses and slots. This is where your hardware vendor comes in. They decide how to organize your system, buses, slots and so on. Note that not only network cards are in your system, there is a lot more, eg. internally connected USB hubs, HBA's etc. So the service console when boots up is just walking through PCI list and detecting hardware without remembering ID's of the devices. So if you change devices in slots, next time you reboot the server it may appear that your eth0 is eth6 (or vmnic6) instead of expected eth1 (or vmnic1). Why? Moving card to next slot you could move it to different bus, and this bus is in different position on PCI hierarchy. The same situation applies when you add more cards to the server... cards numbering will change as mapping between PCI bus:slot -> device name will change ('shifting' devices)! Main answer, ask your hardware vendor how they assigned slots to buses and then reading lspci output you can determine which one is which. General rule, lower PCI bus:slot, lower network card device name. About 'udev'. It's latest version of 'low level device manager' in Linux, it has capability to remember devices ID's so even if you move device from slot to slot it will always remain as eg. eth0. Furthermore if you remove device from the server it will remember it as well so if by any chance you put it back it will be given eth0 again. This may or may not cause problems as you can imagine. Hope this help a bit. Quick comment to why ESX is numbering PCI devices like that. As you know ESX is using RH for its service console. It is fairly old RH running kernel 2.4 series. On the top of that it’s using old way of detecting devices – static /dev directory. Using udev would be nicer and solve all these problems. Now back to the problem.
There is command available on most Linux systems: lspci (I would suggest ‘man lspci’). You can get a lots of info out of it. All we need is the info about buses and slots. This is where your hardware vendor comes in. They decide how to organize your system, buses, slots and so on. Note that not only network cards are in your system, there is a lot more, eg. internally connected USB hubs, HBA’s etc.
So the service console when boots up is just walking through PCI list and detecting hardware without remembering ID’s of the devices. So if you change devices in slots, next time you reboot the server it may appear that your eth0 is eth6 (or vmnic6) instead of expected eth1 (or vmnic1).
Why? Moving card to next slot you could move it to different bus, and this bus is in different position on PCI hierarchy. The same situation applies when you add more cards to the server… cards numbering will change as mapping between PCI bus:slot -> device name will change (’shifting’ devices)!

Main answer, ask your hardware vendor how they assigned slots to buses and then reading lspci output you can determine which one is which.
General rule, lower PCI bus:slot, lower network card device name.

About ‘udev’. It’s latest version of ‘low level device manager’ in Linux, it has capability to remember devices ID’s so even if you move device from slot to slot it will always remain as eg. eth0. Furthermore if you remove device from the server it will remember it as well so if by any chance you put it back it will be given eth0 again. This may or may not cause problems as you can imagine.

Hope this help a bit.

]]>