VM /ETC

Archive for the ‘esxi’ Category

A few virtualization bloggers have reported that they received an alert email from VMware about an I/O failure issue involving iSCSI or Fiber Channel (FC) SANs. There is also an alert currently dispalyed at http://www.vmware.com/support. In summary, an indefinite block occurs between ESXi/ESX 3.5 Update 3 hosts and VMFS 3 Luns which results in all paths to the storage entering a standby state. The issue is apparently isolated to the Update 3 version only.

Eric Sloof is one blogger that received the email and he has published his copy on his NTPro.nl blog. Here’s a brief quote from the email about the issue:

PROBLEM STATEMENT AND SYMPTONS:

• ESX or ESXi Host may get disconnected from Virtual Center
• All paths to the LUNs are in standby state
• Esxcfg-rescan might take a long tome to complete or never complete (hung)
• VMKernel logs show entries similar to the following:
• Queue for device vml.02001600006006016086741d00c6a0bc934902dd115241 49442035 has been blocked for 6399 seconds.
• Please refer to KB 1008130.

SOLUTION:

A reboot is required to clear this condition.

VMware is working on a patch to address this issue. The knowledge base article for this issue will be updated after the patch is available.

VMware KB 1008130 is titled VMware ESX and ESXi 3.5 U3 I/O failure on SAN LUN(s) and LUN queue is blocked indefinitely and provides the pattern of vmkernel messages that identify you have this issue:

Error messages matching this pattern are repeated continually in vmkernel:
<date and time> <hostname> vmkernel: <server uptime> cpu6:1177)SCSI: 675: Queue for device vml.<Vol. Dev. ID> has been blocked for 7 seconds.
<date and time> <hostname> vmkernel: <server uptime> cpu7:1184)SCSI: 675: Queue for device vml.<Vol. Dev. ID> has been blocked for 6399 seconds.

As stated in both the email and the KB article, unfortunately the only solution is to reboot your ESXi/ESX 3.5 Update 3 hosts until VMware is able to provide a patch.

5 Linux-based Virtualization Companies to Watch on Ken Hess’s Linux Blog is a post about the 5 main server virtualization platforms based on Linux. Ken also mentions the only non Linux based hypervisor, Microsoft’s Hyper-V.

“There’s only one company that doesn’t use Linux for its server virtualization platform. Can you guess which one it is? If you guessed Microsoft, you’re correct. Microsoft is a newbie in the virtualization space but wants in and may make significant dents in the already well-established market that is significantly owned by VMware. For Windows-only virtualization, there may be some validity to the switch to Hyper-V.

For the rest of us, who are either too stubborn or too smart to make the shift to Hyper-V, what are our choices? The following is a list of 5 of the main players in Linux-based virtualization.”

Use the link above to read all of Ken’s original post for some brief info about each platform, but I am listing the 6 products and the links to their free versions for quick reference here. Ken does not discuss nor am I including free hosted platforms such as Microsoft Virtual PC or VMware Server. Read the rest of this entry »

Unfortunately this post is an update to my post from last week titled ESXi Update 3 enables full remote administration via RCLI. Today I have received new information from VMware about the current and future status of the Remote Command Line Interface’s (RCLI) ability to manage the free version of ESXi 3.5.

With the release of VMware ESXi 3.5 Update 3, some API calls became freely available.  As it turns out, these new APIs were opened up unintentionally.  This inadvertent change happened when VMware was resolving an API-related bug.

Only customers who downloaded the free version of ESXi have been affected.  VirtualCenter and VI (Foundation, Standard, Enterprise) customers are not affected.

My VMware contacts tell me that this bug will be fixed shortly, and the next release will lock down the RCLI again.

Remote administration and configuration of the free version of ESXi 3.5 remains possible from the VMware Virtual Infrastructure Client.

I have updated my original post with this information as well.

I’ve posted about dealing with ESX snapshots before, but it seems to always be a topic that resurfaces. It’s almost as if there is a “virtualization school of hard knocks” and dealing with open ESX snapshots is a required course. Maybe the misunderstanding is so widespread because a lot of administrators first use VMware Workstation before implementing production servers hosted on ESX?

Anyways, there is a lot of information about snapshots and the problems they can get you in already on the web, but this week Jason Boche posted Know thy open snapshots
which I found to be very informative and helpful. The title of my post was taken from an eye opening quote from Jason:

“Unfortunately in the current builds, VMware doesn’t give us real good (or automated) visibility of open snapshots. I liken it to handing a loaded gun to a child – it’s only a matter of time before an accident happens. That analogy is quite extreme but it gets my point across …”

Read Jason’s post in it’s entirety for much more information, but I particularly like his explanation Read the rest of this entry »

Unlike Windows Sever and various Linux distributions that have separate installation media for 32 bit and 64 bit versions, there is only one VMware ESX installation CD. If you have 64 bit hardware you need to download and install the same media available at https://www.vmware.com/download/vi/ that you would use for 32 bit hardware. There is an additional requirement and configuration for 64 bit guest VMs, however.

To enable hosting of virtual machines running 64 bit operating systems and applications you must have an ESX host that has CPUs with modern virtualization assist technologies. This would be either the Intel-VT or the AMD-V processors. Once you have hardware with these processors you need to enable them in the server BIOS. A common example for enabling this feature would be to hit whatever key your hardware requires duing boot (F2, F12, DEL, F9, etc), and then once in the BIOS go to Advanced Features > Processors > Virtualization Assist. This will vary from manufacturer to manufacturer.

Note that you can install ESX with virtualization assist disabled, and you can P2V or build 32 bit VMs without issue, but once you try to add a 64 bit guest Read the rest of this entry »

If you are like me you’ve probably wondered “if VMware’s ESXi hypervisor is supposed to be so small, then why is the downloadable installation media so huge?” As you can see from the image to the right, the current version is 241 MB! A download that size does not appear to be the tiny, embeddable hypervisor minus the Linux based Service Console we’ve all heard so much about. Fortunately, I was lucky enough to participate in a phone conversation a few weeks back with Amir Sharif, VMware Senior Product Manager, where I got the answer to this very question.

Amir explained to me that the VMware ESXi downloads (both for installation media and patches) contain 4 major components.  These components and their approximate sizes are:
Read the rest of this entry »

The problem surfaced from a ESX 3.5 U2 host fiber connected (FC) to a NetApp filer. An AutoSupport email was generated (NetApp’s filer “phone home” feature) with the following information:

This AutoSupport indicates that there is a configuration issue in the FCP partner path.

Information:
============

[hostname: scsitarget.partnerPath.misconfigured:error]: FCP Partner Path Misconfigured.
[hostname: scsitarget.partnerPath.misconfigured:error]: FCP Partner Path Misconfigured – Host I/O access through a non-primary and non-optimal path was detected.

The AutoSupport email even explained what causes the misconfigured path.

This message occurs when the system detects that host I/O access to logical units (LUNs) is not through a primary path. NetApp clustered storage controllers allow access to LUNs through primary, optimized paths, and secondary, non-optimized paths.  Secondary(non-primary) paths provide access to LUNs through the partner storage controller’s FCP target ports. Under normal operating conditions, a host should not perform I/O to LUNs using a non-primary path. Access through a non-primary path should only occur when a host’s MPIO software detects a failure of all primary paths.

The mystery for me was “what caused a failure of the primary path?“. Believe it or not, Read the rest of this entry »