VM /ETC

Posts Tagged ‘issues’

The product expiration time bomb that was mistakenly left in the first versions of the ESX 3.5 and ESXi 3.5 Update 2 download media is no doubt an embarrassing and horrible mistake by VMware. The timing of this disaster couldn’t be worse with Microsoft Hyper-V, Citrix XenServer, and others starting to be considered as an alternative virtual infrastructure platform for companies just beginning to explore the benefits of virtualization. How could this have happened and what are some lessons to be learned, not just for VMware, but for VI administrators around the world? Read the rest of this entry »

Based on the email I received from VMware at 1:14 am EST last night, the express patches for the ESX 3.5 and ESXi 3.5 Update 2 product expiration bug have been released and are available for download. This availability would have been 10:14 pm PST and slightly later than the 9:00 pm promised deadline. This is still impressive customer response in my opinion as the ESX team is no doubt working around the clock to provide a fix around the world. VMware’s next deadline is 6 pm PST tonight (August 13) to provide the reissued upgrade media.

There are now 3 KB articles related to this bug:

• Fix of virtual machine power on failure issue, refer to KB 1006716
• For VI 3.5, refer to KB 1006721 for deployment consideration and instruction
• For VI3.5i, refer to KB 1006670 for deployment consideration and instruction

The whole email is copied at the end of this post, but here is the important information from the email about the express patches:

We have released the express patches for the product expiration issue. Please
go to http://www.vmware.com/go/esxexpresspatches for more information.

1. What do the express patches do?

There are two express patches: one for ESX 3.5 Update 2 and one for ESXi 3.5
Update 2. They are specifically targeted for customers who have installed or
fully upgraded to ESX/ESXi 3.5 Update 2 or who have applied the
ESX350-200806201-UG patch to ESX/ESXi 3.5 or ESX/ESX 3.5 Update 1 hosts. For
customers who haven’t done either, these express patches should not be applied.

To be noted is that these patches have been validated to work with esxupdate.
However, testing with the VMware Update Manager is still under way. In
subsequent communications, we will provide confirmation whether the patches work
with VMware Update Manger or if a re-spin is required.

We are currently testing an option to apply the patch without requiring
VMotion or VM power-off and re-power-on at the point of patch application. To
immediately refresh vmx on the VM, one can VMotion off running VMs, apply the
patches and VMotion the VMs back. If VMotion capability is not available, VMs
can be powered off before the patches are applied and powered back on
afterwards.

The following is the email I received from VMware in it’s entirety. Read the rest of this entry »

I just received word that the following email is being sent (or will be sent) to all ESX 3.5 and ESXi 3.5 customers likely to be impacted by the Aug 12 time bomb BUG. I assume the 6:00 pm deadline is PST as VMware’s headquarters is in Palo Alto, California?

Dear VMware Customers,

Please find the latest update about the product expiration issue. From this point on, we’ll provide an update every two hours. Thanks.

Problem:

An issue has been discovered by many VMware customers and partners with ESX/ESXi 3.5 Update 2 where Virtual Machines fail to power on or VMotion successfully. This problem began to occur on August 12, 2008 for customers that had upgraded to ESX 3.5 Update 2. The problem is caused by a build timeout that was mistakenly left enabled for the release build.

Affected Products:

- VMware ESX 3.5 Update 2 & ESXi 3.5 Update 2

- Reports of problems with ESX 3.5 U1 with the following 3.5 Update 2 patch applied.

1. ESX350-200806201-UG

- No other VMware products are affected.

What has been done?

Read the rest of this entry »

I just ran across this rapidly growing thread in the VMware Community forums. Apparently, the date August 12, 2008 is literally a bug in ESX 3.5 Update 2. If you have already upgraded be prepared to experience the following issues as described at VMware Communities: BIG bug in ESX 3.5 Update 2 – If you’re … (check this thread for the latest as it is updating frequently as of this posting):

The bug:

Starting this morning, we could not power on or VMotion any of our Virtual Machines. The VI Client threw the error “A general system error occurred: Internal Error”.

Further digging lead us to messages like this one in /var/log/vmware/hostd.log, and the log file for any virtual machine we tried to power on or VMotion:

Aug 12 10:40:10.792: vmx| http://msg.License.product.expired This product has expired.
Aug 12 10:40:10.792: vmx| Be sure that your host machine’s date and time are set correctly.
Aug 12 10:40:10.792: vmx| There is a more recent version available at the VMware Web site: “http://www.vmware.com/info?id=4″.

A call to tech support confirmed this as a known problem with a temporary workaround.

The work-around: Read the rest of this entry »

I was banging my head against my desk trying to make shared folders work in VirtualBox 1.64 on my WinXP notebook inside an Ubuntu 8.04 guest. I kept getting a “protocol error” failure.

Here’s the scenario I was working with:

• I created a Windows folder on my notebook to share to the guests – f:\shared2vms
• I added the folder to the shared folders properties of the Ubuntu VM and named it shared2vms
• I created a folder at /home/username/shared2vms to be the mount point of the VirtualBox shared folder in the Ubuntu guest

For a Linux guest in VirtualBox the command to use the shared folder is “mount -t vboxsf [shared folder name] [mount point]”

So, the command I was using

#sudo mount -t vboxsf shared2vms /home/user/shared2vms

After some creative Google -ing I luckily found this Virtualbox.org forum thread that solved the issue – Read the rest of this entry »

I was involved with a customer support issue today where multiple ESX hosts were experiencing random restarts. Although I did not personally get to troubleshoot the servers, the customer was confident that the RAM, all ordered at the same time, was the issue and probably a bad batch. This scenario is extremely difficult to troubleshoot, and extremely costly when multiple guests are hosted on a production ESX server. Of course, best practice when building new ESX hosts is to thoroughly test for bad memory before hosting VMs, but there is also a utility installed with ESX made for RAM testing in the background too. This post covers both options. Memtest86 should be used before an ESX host is in production while Ramcheck can be used if a problem develops after hosting running virtual machines. Read the rest of this entry »

Yes, this post uses another movie reference.

In the film 28 Days Later the Rage virus infects the Island of Great Britain turning all but a few survivors into zombie-like monsters called “The Infected”. The virus was unleashed when animal activists released medical research chimpanzees which ended up attacking the activists and scientists. This post is about what could cause a similar rage 730 days after installing VirtualCenter, potentially causing VI administrators to become lifeless, rabid, and insane.

After installing VirtualCenter (VC), you should check the installed SSL certificate used by the VI Client because you will most likely need to manually replace it. After a fresh installation the default certifcate expires in 730 days (or 2 years). If the certificate expires you will be unable to log in to the VirtualCenter Management Server using either the VI Client or the web administration interface.

Unfortunately, it is unclear to me at this writing if upgrading the VC Server within the 730 day period updates the certificate store. Read the rest of this entry »