VM /ETC

Archive for the ‘esx’ Category

A new patch for VMware ESX 3.5 Update 3 was released last week that fixes the previously reported iSCSI and FC Alert – “Queue for device has been blocked” issue. The fix was part of a bundle of patches released for ESX 3.5, ESXi 3.5 and ESX 3.0.x products.

Duncan’s summary post over at yellow-bricks.com pointed me to the new VMware KB article specific to the patch with the I/O alert fix: VMware ESX 3.5, Patch ESX350-200901401-SG: Updates VMkernel, VMX, and hostd

Upon reading the KB article I noticed that the VI Client NTP configuration issue I have posted about, and continue to run into, has also been fixed.

Read VMware’s KB for all “the other issues” corrected along with some information about applying the patches with or without Update Manager.

Jason Boche’s post titled Idle Memory Tax is a great read if you are trying to understand ESX memory allocation between virtual machines (VMs). Specifically, the post does a great job explaining how it works when you over commit your physical host’s memory. In other words, the sum of all the RAM assigned to the VMs running on a host is greater than the actual physical RAM of the ESX server.

Here’s a quote from Jason that briefly explains part of the technology that makes over commit possible: the Idle Memory Tax (IMT).

“Quite simply it’s a mechanism to take idle/unused memory from guest VMs that are hogging it in order to give that memory to another VM where it’s more badly needed. Sort of like Robin Hood for VI. By default this is performed using VMware’s balloon driver which is the more optimal of the two available methods. Out of the box, the amount of idle memory that will be reclaimed is 75% as configured by Mem.IdleTax under advanced host configuration. The VMKernel polls for idle memory in guest VMs every 60 seconds.”

Read the entire post for much more technical details and examples.

I’ve blogged before about the symptoms when the IMT and the ESX balloon driver can no longer keep up and it’s time to add another ESX host and spread the VM load.

I believe that ESX 3.x changed the need to Read the rest of this entry »

A few virtualization bloggers have reported that they received an alert email from VMware about an I/O failure issue involving iSCSI or Fiber Channel (FC) SANs. There is also an alert currently dispalyed at http://www.vmware.com/support. In summary, an indefinite block occurs between ESXi/ESX 3.5 Update 3 hosts and VMFS 3 Luns which results in all paths to the storage entering a standby state. The issue is apparently isolated to the Update 3 version only.

Eric Sloof is one blogger that received the email and he has published his copy on his NTPro.nl blog. Here’s a brief quote from the email about the issue:

PROBLEM STATEMENT AND SYMPTONS:

• ESX or ESXi Host may get disconnected from Virtual Center
• All paths to the LUNs are in standby state
• Esxcfg-rescan might take a long tome to complete or never complete (hung)
• VMKernel logs show entries similar to the following:
• Queue for device vml.02001600006006016086741d00c6a0bc934902dd115241 49442035 has been blocked for 6399 seconds.
• Please refer to KB 1008130.

SOLUTION:

A reboot is required to clear this condition.

VMware is working on a patch to address this issue. The knowledge base article for this issue will be updated after the patch is available.

VMware KB 1008130 is titled VMware ESX and ESXi 3.5 U3 I/O failure on SAN LUN(s) and LUN queue is blocked indefinitely and provides the pattern of vmkernel messages that identify you have this issue:

Error messages matching this pattern are repeated continually in vmkernel:
<date and time> <hostname> vmkernel: <server uptime> cpu6:1177)SCSI: 675: Queue for device vml.<Vol. Dev. ID> has been blocked for 7 seconds.
<date and time> <hostname> vmkernel: <server uptime> cpu7:1184)SCSI: 675: Queue for device vml.<Vol. Dev. ID> has been blocked for 6399 seconds.

As stated in both the email and the KB article, unfortunately the only solution is to reboot your ESXi/ESX 3.5 Update 3 hosts until VMware is able to provide a patch.

health_check.sh script report .html

If you are looking for a free and quick method to document your ESX 3.0.X  and ESX 3.5 server configurations without installing a scripting environment then don’t forget about A. Mikkelsen’s ESX Healthcheck script. An oldie but still a goodie, this script is simply transferred to your ESX 3.5 Console and then just run manually or via a scheduled job. The end result is a .html report that is saved on your ESX host, or if you utilize the optional send_smtp.pl script the report can be mailed to you automatically.

I’m not providing any new information in this post. In fact I am way behind  as Duncan Epping has covered this script on his blog at Yellow-bricks.com. Duncan first posted about the esx healthcheck script in January 2008 and recently mentioned it again in his post about his tools he uses when conducting customer Healthcheck services. The comments thread on Duncan’s January post walk through a year’s worth of feature development and support.

Download your copy of the script at  SourceForge.net: ESX Healthcheck Script

Once downloaded, the .zip file contains several helpful text files along with the script. The rest of this post walks through the install and configure steps and then explains how to run the script when ready. Read the rest of this entry »

I’ve posted about dealing with ESX snapshots before, but it seems to always be a topic that resurfaces. It’s almost as if there is a “virtualization school of hard knocks” and dealing with open ESX snapshots is a required course. Maybe the misunderstanding is so widespread because a lot of administrators first use VMware Workstation before implementing production servers hosted on ESX?

Anyways, there is a lot of information about snapshots and the problems they can get you in already on the web, but this week Jason Boche posted Know thy open snapshots
which I found to be very informative and helpful. The title of my post was taken from an eye opening quote from Jason:

“Unfortunately in the current builds, VMware doesn’t give us real good (or automated) visibility of open snapshots. I liken it to handing a loaded gun to a child – it’s only a matter of time before an accident happens. That analogy is quite extreme but it gets my point across …”

Read Jason’s post in it’s entirety for much more information, but I particularly like his explanation Read the rest of this entry »

So you’ve downloaded your free version of ESXi. Maybe you’ve built a couple of ESX Servers but you haven’t purchased a full VI 3.5 Enterpise license. You might even have an enterprise license but your VirtualCenter (VC) server will be a virtual machine (VM) and you want to build a template VM as a gold image first. These are some of several possible scenarios where it helps to have VC -like functionalities without VirtualCenter. This post is a summary of 3 freely available tools that can provide some of VirtualCenter’s features.

First let me state that there is not an exact substitute for VirtualCenter, and the automation and ease of management VC provides is well worth it’s cost. Now for some alternatives. Read the rest of this entry »

Unlike Windows Sever and various Linux distributions that have separate installation media for 32 bit and 64 bit versions, there is only one VMware ESX installation CD. If you have 64 bit hardware you need to download and install the same media available at https://www.vmware.com/download/vi/ that you would use for 32 bit hardware. There is an additional requirement and configuration for 64 bit guest VMs, however.

To enable hosting of virtual machines running 64 bit operating systems and applications you must have an ESX host that has CPUs with modern virtualization assist technologies. This would be either the Intel-VT or the AMD-V processors. Once you have hardware with these processors you need to enable them in the server BIOS. A common example for enabling this feature would be to hit whatever key your hardware requires duing boot (F2, F12, DEL, F9, etc), and then once in the BIOS go to Advanced Features > Processors > Virtualization Assist. This will vary from manufacturer to manufacturer.

Note that you can install ESX with virtualization assist disabled, and you can P2V or build 32 bit VMs without issue, but once you try to add a 64 bit guest Read the rest of this entry »