VM /ETC Go Green with Virtualization. Go UGLY Green with vmetc.com.

Earlier this week the VMware VI Team Blog reported that as of ESX version 3.5 Update 1 Storage VMotion is now officially supported for iSCSI SANs. This means that administrators can reorganize virtual machines without down time in order to match any storage needs. The Storage VMotion and 10Gb Ethernet support for iSCSI SAN’s post explains:

“Although Storage VMotion is designed to work with any type of storage, it was initially supported only with Fibre Channel SANs. As of Update 1, Storage VMotion is supported with iSCSI SAN’s for moving virtual machine disk files in the following scenarios:

- From iSCSI SANs to other iSCSI SANs

- From iSCSI SANs to FibreChannel SANs

- From FibreChannel SANs to iSCSI SANs

In addition, we now support the use of 10Gb Ethernet for iSCSI in a VMware Infrastructure environment.”

In my experience Storage VMotion has worked great, but be careful with VMs that have higher I/O utilizations.

• Continue reading or Leave a comment... (0)

Lately, I have had several clients and peers ask me how to get disk usage and performance statistics from their current virtual infrastructure of ESX hosts and virtual machines . Some needed data for planning and sizing a new SAN, while others needed statistics for upgrading, adding more disks, or for optimizing multi path and VMFS performance. In one case the customer was trouble shooting poor VM performance issues. Regardless of the objectives there are some built in tools in both VirtualCenter and ESX server that can get this information for you. This post explains 2 native methods:

• Using the VI Cleint to access the Performance data in VirtualCenter
• Using esxtop from the ESX Service Console

I have included plenty of screenshots. As always, click on them for larger views. The rest of this post is in an outline format, but should be easy enough to follow.

• Continue reading or Leave a comment... (1)

Michael Healey over at InformationWeek has written an article that demonstrates real world pricing scenarios comparing the cost of a SAN with replacing physical servers and local RAID 5 storage. ROI Analysis: Virtualization uses an example of 10 physical servers at their “end of life” and focuses on hard cost savings that will even make sense to the CIO. Read the entire article, but here’s the results:

“Scenarios under consideration:

1. Replace with 10 traditional servers: Ten HP ML350 boxes with mirrored boot drives and RAID 5 storage–$62,100.

2. Virtualize 10 servers using three physical servers with local storage: Three HP ML350s with more memory and RAID 5 storage running VMware ESX (licenses included)–$49,300.

3. Same virtualized servers in a cluster that boots to a 1-TB SAN: Three HP ML350s with more memory, boot drives, and VMware Enterprise, plus a Hitachi SAN for storage–$52,730.”

As Michael points out, migrating to virtual infrastructure and spending $3500 extra for a SAN (scenario 2 versus 3) are options that should make sense to even the most cost conscious IT departments!

• Leave a comment... (0)

Based on best practices, this post is a “formula” for planning and sizing SAN storage for maximum VMFS performance. This is intended to be for all protocols where VMFS volumes are utilized ( FC, iSCSI ) and conservative enough to allow general sizing estimates while ensuring high performance of the running VMs.

WARNING: The storage design resulting from following these recommendations will not be the most cost effective solution. As storage performance generally requires the most spindles (disks) possible and this formula requires as many dedicated LUNs as possible, the cost for storage will be maximized. More often than not, compromises between performance and cost have to be reached that keep the design within the expected budget.

This post is split into 2 sections. The first section lists the VMFS Maximum Performance Rules while the second section uses a 25 server example to walk through the design rules.

• Continue reading or Leave a comment... (8)

This post is a supporting post to the discussion earlier this week sparked by Scott Lowe’s tip on avoiding ESX snapshots when using SAN device snapshots. NetApp’s SnapManager for Virtual Infrastructure (SMVI) is discussed as a solution for streamlining the coordination of snapshots between ESX and the SAN. Luckily, Nick Triantos from NetApp joined the discussion on Scott’s blog with some “under the hood” information:

• Continue reading or Leave a comment... (3)

Avoiding storage array snapshot pitfalls in a VMware environment is an article and tip published by Scott Lowe for Searchvmware.com. Scott discusses the design challenges and implications of combining the snapshot abilities of VMware ESX with the SAN based snapshot features of storage devices. The tip points out that incorrect configuration of VMware ESX with the storage device could lead to inconsistent and unusable images when trying to recover VMs.

“Because these snapshots are not, by default, integrated in any way with VMware ESX Server, we have to perform a few extra steps to ensure consistently reliable and usable storage array snapshots.”

Read all of Scott’s tip at the link to the article above.

My “2 cents” on this is that trying to configure the combination of the two snapshots manually might not

• Continue reading or Leave a comment... (1)

Move over Lefthand Networks VSA, xtravirt.com has provided a free alternative for creating a virtual iSCSI SAN. Xtravirt Virtual SAN (XVS) is a virtual machine appliance that runs on two of your ESX hosts’ local VMFS datastores to create a single, synchronized iSCSI SAN. XVS allows the creation of ESX clusters for VI3 Enterprise features without purchasing a physical shared storage solution.

“The Xtravirt Virtual SAN (XVS) appliance for VMware ESX3 Server is a free solution to provide the benefits of shared VMFS storage without the cost of a SAN – this allows the utilisation of otherwise unused local storage in the ESX server to facilitate enterprise level features such as vMotion, DRS and HA normally only available through the use of a shared storage device. All volume data is synchronously replicated between hosts, providing full fail-over capability with data integrity in the event of host, disk or appliance failure.”

XVS is the perfectly priced storage solution for the home ESX test lab, small and mediium businesses, or the small remote branch office.

To download a copy of the virtual appliance and for more about XVS go to xtravirt.com.

updated 5.24.08

Currently XVS is only configurable as a single LUN across paired ESX hosts. A third ESX hosts can use the virtual ip address for it’s SAN, but the additional host(s) would not be using their local storage as part of the synchronized SAN. Future editions will hopefully expand the storage across more than 2 ESX hosts.

• Leave a comment... (4)

I was helping a customer P2V a large development SQL server this week and ran into a VMFS configuration issue that failed the conversion. We were using the Converter Enterprise for VirtualCenter 2.5 plugin. Almost as soon as we kicked off the job it failed with an error starting with “file size is larger than the maximum size supported by datastore”. The VMFS LUN we were using as the target was an empty 1.5 TB volume, and the new VM consisted of 2 virtual disks that totaled roughly 450 GB. We had plenty of room, but the problem was not the available storage space. Instead, the issue was that we exceeded the maximum possible .vmdk size for the default VMFS 1MB block setting.

When you add new storage to an ESX host and you format the LUN with the VMFS file system you have to choose what block size setting you want to use. See the screenshot for the dropdown box used to make this choice. Notice the Maximum file size description supposedly provided to help you understand this setting. It’s hardly intuitive in my opinion, so let me try to translate - Choosing the block size determines what maximum possible .vmdk size can be created on this LUN.

If you do not change the default setting when you format a VMFS LUN

• Continue reading or Leave a comment... (3)

VMware VirtualCenter comes with built in alerting and a handful of alerts preconfigured. Unfortunately, alerting for disk space usage of either the ESX hosts or the virtual machines is not included. Administrators continue to use common physical infrastructure monitoring and reporting applications such as NetIQ and MOM for VMs, or SNMP capable programs like HP Openview or IBM Director for ESX host monitoring. A less complex and less expensive ( cost of installing and configuring agents on each VM OS ) alternative would be to tap into VirtualCenter’s central management ability to monitor, alert, and report on disk space. This post lists a few free solutions that can already use VC2.x or quickly be configured for ESX hosts and therefore save administrators time and money. Hopefully, a future feature of VC2.x will include vital disk space metrics and alerting.

• Continue reading or Leave a comment... (0)

When planning P2V migrations there is always a handful of servers that have huge disk requirements. A network file share could be as large as several terabytes. User Home Directories can also cause large, cumbersome servers that present unique challenges when migrating to virtual infrastructure. Given that VMware VI3 has a 2 TB VMFS volume limitation as well as a 2 TB .vmdk limitation, there is a better option. Consolidate your shares to a

• Continue reading or Leave a comment... (1)

VMware Communities: New Script for moving vm to another … contains a thread from this summer about cold migrating a VM from local storage of an ESX server to iSCSI storage. The post contains the code (original and updated versions) for a script to automate this process. The final version allows the migration of a VM to any ESX storage location. I have added the vm-relocate.sh script to the Files page as well.

This script is handy if you do not have

• Continue reading or Leave a comment... (0)

My customer had a couple of ESX hosts. Both ESX hosts had access to (4) 250 GB VMFS LUNs on fiber attached IBM storage. After upgrading their SAN controller in order to allow them to have the more storage partitions for booting several blades from SAN, the LUNs were no longer available to the ESX hosts. The LUN IDs were changed when the storage controller was upgraded and ESX began complaining that the LUNs were snapshot volumes.

The resolution was simple enough and is documented in VMware’s KB article titled “VMFS Volume Can Be Erroneously Recognized as a Snapshot“:

To resolve issues with invisible LUNs on certain arrays:

1. In the VI Client, select the host in the inventory panel.

2. Click the Configuration tab and click Advanced Settings.

3. Select LVM in the left panel and set LVM.DisallowSnapshotLUN to 0 in the right panel.
   Warning: When LVM.DisallowSnapshotLUN is set to 0, no snapshot LUNs should be presented to the ESX Server host. Otherwise, data corruption may result. For details, see “State 3 - EnableResignature=no, DisallowSnapshotLUN=no” on page 110 of the SAN Configuration Guide at www.vmware.com/pdf/vi3_esx_san_cfg.pdf.

4. Rescan all VMFS volumes.

After the rescan, all VMFS volumes are available.

The issue resurfaced in slightly different form when I added 3 new ESX hosts to the environment. The existing 4 VMFS LUNs never appeared as available storage to the new hosts even though the zoning was correct. Rescanning from the storage adapters did not help. When I went to manually add the storage I could see the LUNs but I was warned:

The current disk layout will be destroyed. All file systems and data will be destroyed permanently.

That was enough to make me stop and research again. It turns out that the LUNs were appearing to the new hosts as snapshot volumes as well, so I had to make the same LVM.DisallowSnapshotLUN change as above.

If you actually do use snapshot LUNs then there is another fix to this issue detailed in VMware’s KB article titled “Resignaturing VMFS3 Volumes That Are Not Snapshots“.

• Leave a comment... (0)