I’ll get right to it. You can make a backup copy of physical SQL databases with Veeam BaR. Both vCenter and Veeam BaR have a SQL backend. You can’t schedule this as a job, but there are several scenarios where you could take advantage of a quick and easy, one time, manual backup  – before an upgrade or patch, for example.

Although I work for Veeam, this is not necessarily an intended or fully supported usage of the product. This is an easy alternative for the VMware admin to CYA , however.

I’ll start with a brief introduction on how the SQL U-AIR wizard is supposed to work, and then I will explain how you can use an admin switch to make a backup copy of SQL database whether on a VM or a physical server. VMware vCenter and Veeam BaR/Monitor/Reporter all have SQL back ends. 

The U-AIR Up There

To do this you can use the SQL U-AIR wizard. U-AIR stands for Universal Application Item Recovery, and there are 4 stand alone .exes for the various U-AIR wizards of Veeam BaR – AD, Exchange, SQL, and Universal. All of these wizards can be installed on the Veeam BaR server or on any Windows system that can communicate with Veeam. They could be installed on an admins desktop or the SQL, Exchange, or Domain Controller servers too.

Normally, the purpose of the U-AIR wizard is to request and kick off a workflow for a Veeam vPower Virtual Lab. Once the request is approved and managed by the VMware/Veeam administrator and the “Lab Manager–like” virtual lab is ready with the fenced off, running backup copy of the VM(s), the U-AIR wizards allow for the restore from the backup copy VM to the original production VM. For SQL VMs in particular, the restore options are shown in the following screen shot:

Watch this 4 minute video to see the normal SQL restore functionality of the wizard. This video skips the workflow request, skips the wait for approval and virtual lab start up, and just shows what is possible from a backup copy of a SQL VM. I also want to mention that this is an agentless solution. You do not need to install and manage agents anywhere with Veeam BaR.

Trick The System for Physical SQL backups

You can skip the workflow process of starting and using the vPower Virtual Lab if you use an undocumented (as far as I know) Admin Switch for the U-AIR wizards. I’ll focus on the SQL U-AIR wizard for the rest of this post, but it is the same for the Exchange and AD wizards as well.

To use the U-AIR Admin Switch do the following:

1. Download and Install the SQL U-AIR wizard on the Veeam BaR server, a SQL server, or your desktop.
2. Browse to the install folder – standard location is “C:\Program Files\Veeam\Application Restore\SQL Restore”
3. Create a shortcut – right click Veeam.AIR.SQLrestore.exe and choose “Create Shortcut”
4. Right click the new shortcut and choose Properties
5. Modify the “Target” field by adding “/admin” (see the screen shot to the right. click for a larger image)
6. Save the shortcut.
7. Move the modified shortcut to where ever you want (the desktop)

Now, when you run the wizard from the shortcut you go right to the restore options.

Making A Backup of a physical VMware vCenter SQL

Here’s where it gets interesting! Because the wizard prompts you for the SQL Server\Instance to connect to, you can choose any SQL instance that is configured for remote connections. Yes, SQL Express works too!

In my lab environment, vCenter is not included in a Veeam Backup job. It is a “virtual physical server”. I call it “virtual physical” because I use a VMware Workstation lab (3 VMs = 2 vSphere 4 ESXi + 1 vCenter). The Veeam BaR server is a nested VM along with the 5 other VMs in the environment which it backs up. So vCenter is the equivalent to a physical server in a real datacenter. Clear as mud?

To back up my vCenter database I can do the following:

• start the SQL U-AIR wizard from the shortcut with the /admin switch
• On the SQL Servers screen click the Browse button
• From the Choose SQL Server pop up window pick the vCenter instance (screen shot below. click for larger view. Note the Veeam BaR and Veeam One instances available too)

• For performing just a backup, copy the newly populated contents of the Backup server name field to the Production server name field (screen shot below, click for larger view)

• For the Restore Task pick “Restore Database Tables” (see screen shot at the beginning of this post)
• For the Restore Target pick “Bulk Copy Program (BCP)”

• From the Database screen pick the vCenter database.

• From the Objects to Restore screen leave all selected

• Pick an Output folder. (I created a Restore folder on the Desktop)
• Finish the wizard and export the database tables to the selected folder.

Now you have a CYA, backup copy of the vCenter database. In case you didn’t notice this was done while vCenter was running.

If you need help with the Microsoft SQL Bulk Import and Export process, check out the MSDN pages on the subject.

The Tip Of The U-AIR Backup / Restore Iceberg

Hopefully you can now see the potential of the Veeam BaR U-AIR wizards with the /admin switch? Imagine what else you can do by placing different SQL instances on your network in the Production and Backup server fields of the wizard. Once again, this functionality goes beyond the intent of the product, but the flexibility of it’s use is endless in the hands of the right admins!

“I wanted to remind everyone, of what I have already seen floating around the internet, but still important enough to remind.  Our next release of SRM is going to require a 64 bit OS.  This is the same as our next release of VC as it too will require a 64 bit host OS.  This change is required to support the increased capabilities of our products.  As we scale our products to match our customers needs, generally 1 – 2 years in advance of where they will need all the capabilities of a given product we have had to use a 64 bit OS.  This will show itself in increased numbers in things like more simultaneous vSphere client connections.”

To me these new operating system (OS) requirements mean we will see even more instances of vCenter as a VM (virtual machine). It only seems logical that a least path of resistance is to virtualize the management server in order to upgrade, especially considering all have already invested in 64 bit hardware for their hypervisors if they decided to upgrade to vSphere 4 in the first place. To go a step further, I’m willing to argue that it will be more common for an IT Department to justify the cost of additional ESX hosts, even if only dedicated for management, then it will to deploy new servers for physical instances of vCenter.

The looming transition to a console-less ESXi eventually means more management virtual appliances in the future too. Solutions which will continue to need a ESX console or similar will have to substitute their own appliance to operate with ESXi. This means even more justification for additional ESX/ESXi hosts and thus greases the decision to virtualize vCenter as well. I expect to see management clusters of ESX hosts become more common in the future than even the use of management networks today.

ESX hosts have bigger and badder hardware now than ever before allowing for higher consolidation ratios and larger applications to easily run in virtual machines, but it will be interesting to see if the vCenter as a VM best practices change over time. I personally feel that continuing to separate the database from the virtualized vCenter will continue to be a smart choice. Running a separate, and even virtualized, SQL instance ensures not only better performance of vCenter as a VM but enhances DR scenarios. In fact, those that already have the vCenter database on a remote instance will likely have a safer upgrade to the 64 bit vCenter.

The new 64 bit requirements will no doubt make for an interesting migration scenario, and I’m sure we will see some positive and negative opinions. Let me know your thoughts on a 64 bit vCenter as a VM in the future!

“The resource ’1′ is in use. DVS [vDS Name] port 1 is reserved by to entity <not-found.> vnic 4000, type:vmVnic”

The screen shot shows the actual error I received.

After thinking about that message, I decided to check to see if any ports were in fact being used on the vDS. Although PortID 1 dvPortGroups did not show it being used as expected, other PortIDs were, in fact, in use. See image below.

I tried messing around with disabling the ports (blocking was the terminology in the vSphere Client, I believe) but I still could not delete the vDS. It was Saturday and I had family stuff to do so I reluctantly gave up at the time. However, 3 days later I was able to easily remove it via the Remove link on vDS Configuration screen just like I was supposed to be able to do all along. Needless to say I was a bit perplexed.

After researching I discovered that

there is a 24 hour time out on the previously mentioned ports in use that was preventing the vDS removal. This is a guess because I am drawing a conclusion between my error message and the information from the following VMware KB article Changing the default timeout for locked dvPorts. The resolution section of this KB explains:

“Before you can remove a vNetwork Distribued Switch or dvPort group, all unused dvPort reservations must time out. You can either wait for the default 24-hour timeout period before removing the vNetwork Distributed Switch or dvPort group, or temporarily change the default timeout for dvPorts to free the ports before the default 24-hour timeout period. If you change the default timeout, be sure to change it back after the ports have been freed.”

The details section of the KB  is not exactly a clear match to my specific scenario, but this seems to explain what actually was preventing me from deleting the vDS.

The locked port timeout can be changed by following the KB articles instructions for changing the setting to 10 minutes:

To remove a vNetwork Distributed Switch or dvPort group with locked dvPorts

1. In vpxd.cfg, add the line <vpxd><dvs><portReserveTimeout>10</portReserveTimeout></dvs></vpxd> and save the file.
2. Restart vCenter. The default timeout is now set to ten minutes.

3. After the port reservation has timed out, remove the vNetwork Distributed Switch or dvPort group.

4. Reset the default timeout by removing the line you previous added to vpxd.cfg.

5. Restart vCenter.

A vDS is created on the vCenter 4 server, so the vpxd.cfg file was in C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter on my vCenter 4. Although I used a custom install to change the default installation directory, I believe that is the default location for the vpxd.cfg file. If you have to search for it be sure to look for hidden files too.

You can see from the following screen shot I added the line in the already existing <vpxd> section just before the closing “</vpxd>” tag. Copy everything between the <vpxd></vpxd> tags in step 1 above. I’ll point out that I first placed the entire line as it’s own section just before the final </config> file, but that caused problems and the VMware VirtualCenter Service would not start. Don’t do that!

I tested the new config changes by creating a new vDS again, adding my hosts, migrating the VMs to use the vDS, powered on and off some VMs, performed a VMotion, migrated the VMs back to the standard vSwitch, removed the hosts from the vDS, and then deleted the vDS. Interestingly enough, I did not have to wait 10 minutes to delete it.

In the end this post may only be valuable for pointing out that a 24 hour period exists by default for locked portgroups on a vDS. I can’t say for sure that the vpxd.cfg change to 10 minutes means you can remove a stubborn vDS more quickly, but I’m drawing the conclusion that it helps. Let me know in the comments if you have a definitive explanation for this scenario!

VMware VirtualCenter 2.5 Update 6 is Generally Available

We are pleased to inform you that VMware VirtualCenter Server 2.5 Update 6 (English and localized) is generally available as of late night January 29, 2010.

VirtualCenter 2.5 Update 6 provides the following improvements:

• Guest Operating System Customization Improvements
• Support for Firefox 3.x Browsers with VirtualCenter Web Access
• Bug and security fixes

For details regarding the new fixes and improvements, please refer to the release notes.

VirtualCenter Server 2.5 Update 6 is available for download.

For details regarding compatibility, please view vSphere Compatibility Matrixes.

Thanks,
VMware Infrastructure Product Management Team

Some of the highlights for me when reading the Release Notes are:

From the Resolved Issues section:

• I/O error occurs when downloading a large file from the datastore browser or when exporting a virtual appliance to OVF (KB 1017614)

I’ve never experienced the issue with transferring .ISOs and other large files with the data store browser, but several VM /ETC readers have. VMware’s KB Article still doesn’t explain exactly when and why it happens, but it’s good to see there’s now a fix. This is critical for ESXi implementations without a service console and SSH access to transfer files. The data store browser will have to become the tool most often used to move files to and from ESX datastores.

From the What’s New section

Guest Operating Systems Customization Improvements

Customization support for the following guest operating systems has been added in this release:
For more complete information about supported guests included in this release, see the VMware Compatibility Guide.

• Windows 7 Enterprise (32-bit and 64-bit)
• Windows 7 Ultimate (32-bit and 64-bit)
• Windows 7 Professional (32-bit and 64-bit)
• Windows 7 Home Premium (32-bit and 64-bit)
• Windows Server 2008 R2 Standard Edition (64-bit)
• Windows Server 2008 R2 Enterprise Edition (64-bit)
• Windows Server 2008 R2 Datacenter Edition (64-bit)
• Windows Server 2008 R2 Web Server (64-bit)

Support for Firefox 3.x Browsers with VirtualCenter Web  Access

This release adds support for Firefox 3.x browsers with VirtualCenter Web Access. Firefox 3.x is not a supported browser for ESX Server 3.5 Update 5 Web Access with this release.

VirtualCenter 2.5 Update 6 also provides Update Manager Plug-In updates:

This release of the VMware Infrastructure 3 software suite includes an update to VMware Update Manager. For more information, see the VMware Update Manager 1.0 Update 6 for VirtualCenter 2.5 Release Notes. VMware Converter Enterprise Update 4 is compatible with VirtualCenter 2.5 Update 6.

Finally, Update 6 also improves security with updates to JRE and Tomcat

Security

• Updates JRE version to 1.5.0_22This release fixes the following security issues by updating the JRE version to 1.5.0_22:
  • The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned the following names to the security issues fixed in JRE 1.5.0_22:
    CVE-2009-3728, CVE-2009-3729, CVE-2009-3864, CVE-2009-3865, CVE-2009-3866, CVE-2009-3867, CVE-2009-3868, CVE-2009-3869, CVE-2009-3871, CVE-2009-3872, CVE-2009-3873, CVE-2009-3874, CVE-2009-3875, CVE-2009-3876, CVE-2009-3877, CVE-2009-3879, CVE-2009-3880, CVE-2009-3881, CVE-2009-3882, CVE-2009-3883, CVE-2009-3884, CVE-2009-3886, and CVE-2009-3885.
  • The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned the following names to the security issues fixed in JRE 1.5.0_20:
    CVE-2009-2625, CVE-2009-2670, CVE-2009-2671, CVE-2009-2672, CVE-2009-2673, CVE-2009-2675, CVE-2009-2676, CVE-2009-2716, CVE-2009-2718, CVE-2009-2719, CVE-2009-2720, CVE-2009-2721, CVE-2009-2722, CVE-2009-2723, and CVE-2009-2724.
  • The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned the following names to the security issues fixed in JRE 1.5.0_18:
    CVE-2009-1093, CVE-2009-1094, CVE-2009-1095, CVE-2009-1096, CVE-2009-1097, CVE-2009-1098, CVE-2009-1099, CVE-2009-1100, CVE-2009-1101, CVE-2009-1102, CVE-2009-1103, CVE-2009-1104, CVE-2009-1105, CVE-2009-1106, and CVE-2009-1107.

• Updates Apache Tomcat version to 5.5.28This release of VirtualCenter Server updates the Tomcat server package from 5.5.27 to 5.5.28, to address multiple security issues. The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned the names CVE-2008-5515, CVE-2009-0033, CVE-2009-0580, CVE-2009-0781, and CVE-2009-0783 to these issues. For more information on security vulnerabilities, refer to the Apache Tomcat 5.x Vulnerabilities page.

If you are still running VMware VI 3 you should consider upgrading to VirtualCenter 2.5 Upgrade 6 as soon as possible. ESX/ESXi 3.5 remains at Update 5 to the best of my knowledge at this time, however.

When you install Update Manager, you can select the location for downloading patches. To change the location after installation, you must manually edit the vci-integrity.xml
file.

Procedure

1. Log in to the Update Manager server as an administrator.
   
2. Stop the Update Manager service.
   
   1. Right-click My Computer and click Manage.
      
   2. In the left pane, expand Services and Applications and click Services.
      
   3. In the right pane, right-click the VMware Update Manager Service and click Stop.
      
3. Navigate to the Update Manager installation directory and locate the vci-integrity.xml file.
   
   1. The default location is C:\Program Files\VMware\Infrastructure\Update Manager.
      
4. Create a backup copy of this file in case you need to revert to the previous configuration.
   
5. Edit the file by changing the following fields:
   
   1. yournewlocation
   2. The default patch download location is: C:\Documents and Settings\All Users\Application Data\VMware\VMware Update Manager\Data\
      
   3. The directory path must end with \.
      
6. Save the file in UTF-8 format, replacing the existing file.
   
7. Copy the contents from the old patchstore directory to the new folder.
   
8. Restart the Update Manager service.
   

Note that integrating the Sysprep files are still required in all versions of vCenter to customize VMs. This includes vCenter 4 for vSphere. Sysprep is no longer used for Server 2008, however, but VMware has added native customization of Server 2008 VMs in vCenter 4 without adding any additional files.

KB Article 1005593 titled Sysprep file locations and versions not only provides download links and extract to locations but also explains the common symptoms when Sysprep is not installed correctly on vCenter.

• When attempting to customize the deployment of a virtual machine the radio buttons are disabled (greyed out).
• When a virtual machine (VM) is deployed from a Template, you find that the SID is always the same, despite the fact that you chose the option to generate a new SID during Template deployment and guest operating system customization.
• When attempting to create a new virtual machine from a Template in ESX v3.5 you receive the following error message

Warning: Windows customization resources were not found on this server

• Message in the guestcust.log:

deploy doesn’t contain known sysprep files

The KB article explains the cause

Microsoft has a different version of Sysprep for each release and service pack of Windows. According to Microsoft, “You need to use the version of Sysprep specific to the operating system you are deploying”. The differences are not immediately visible in the packaging and documentation of the service packs, so it is necessary to manually investigate.

Use either of the following methods to obtain the appropriate Sysprep files. All instructions in this post assume vCenter has been installed in the default location.

Follow the link above to find the table of Sysprep downloads for each Microsoft operating system. Extract those downloads to the exact locations provided in the same table. Basically, all Sysprep versions go in the C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\sysprep directory but in a separate sub folder specific for each OS.

At the time of this writing KB Article 1005593 has been updated as of August 14, 2009 so the download links seem current.

Obtain Sysprep from the Microsoft Install Media

Whether or not KB Article 1005593 remains updated or not, the good news is that you can always get the Sysprep files from the Windows Install CD/DVD too.

Assuming your CDrom drive is E:\

1. Use Windows Explorer to browse to E:\Support\Tools on the Windows (insert your OS here) CD/DVD and find the deploy.cab file
2. Double-click deploy.cab to open it (like a .zip file)
3. Copy all of the contents of the deploy.cab file
4. Paste the copied files on the vCenter server in the appropriate subfolder at C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\sysprep
   

The sub folders are named after the various Windows operating systems and are obvious, but use the table in KB Article 1005593 if you need to confirm the exact file location.

P2V and V2V Customizations with VMware Converter Stand Alone

The free VMware Converter Stand Alone version also let’s you integrate the Sysprep tools and customize multiple VM migrations from a single physical or virtual server. If you install Converter in it’s default location then obtain Sysprep in the same methods already discussed and place the files in the same locations as vCenter. Of course, this location is now on the server where you installed Converter, and you will only need to use the Sysprep version of the installed OS. You can’t install Converter on a Windows 2003 Server, put the Windows XP Sysprep tools in it’s correct directory, and then expect to P2V/V2V a new Windows XP VM!

Starting with VMware Converter 4 Stand Alone there is a Client/Server install option which allows Converter to utilized as a centralized tool for deploying new VMs. Creative administrators are able to duplicate most of vCenter’s deploy from template and VM cloning features using this install mode.

When the green circles do not appear, the Maps view helps troubleshoot what is wrong because you can see exactly which pieces are not connected correctly in the diagram.

What I did not realize, and Eric’s post points out, is that 

there are different shades of green that appear on the Maps tab.

“If one of your ESX servers is under a heavy load it’ll show a light green colour. On the other hand, when one of your ESX server is not CPU intensive at all, it will show up dark green.”

Now that I know this, recognizing what hosts have a darker green circle from the Maps tab is a quick and convenient way to pick a host that has the best available resources for a manual VM migration.

First let me state that there is not an exact substitute for VirtualCenter, and the automation and ease of management VC provides is well worth it’s cost. Now for some alternatives.

ESX Manager 2.3

As described on it’s web page at ESXGuide.com:

Features:

• Virtual Center independent
  Keeps track of Virtual Machine Host Registration, Migrations and Status
  Manage Virtual Machine Configuration
  Display and work in the Virtual Machine Console
  Kill Virtual Machine Process (if the VM can’t be powered off)
  Rename Virtual Disks of registered Virtual Machines
  Move Virtual Disks and keep the disk attached to the Virtual Machine
  Virtual Machine Registration, Start , Restart , Reset, Power Off, Suspend
  Extend Virtual Disks
  View-Search-Filter Logfiles on the ESX Host.
  Define and save custom SSH commands with Parameter handling

VMX – Trilead VM Explorer

I discovered VMX – Trilead VM Explorer from the professionalvmware.com post titled Free ESX and ESXi Backups – Trilead VM Explorer and it quickly has become one of my favorites. It is much more than a back up tool however. Here’s more info from theTrilead web page.

Features

• Multi-Server Management
  The graphical user interface of VM Explorer allows to connect to multiple ESX, Linux and FreeBSD servers at the same time in the same interface.
• Server to Server fast copy
  Allows you to copy files directly (drag&drop) between ESX, Linux and FreeBSD servers at full speed: throughput is only limited by the available network/storage system bandwidth.
• ESX to ESX/Windows/Linux/FreeBSD/SAN Backup & Restore
  Our unique ESX to any backup feature allows to backup virtual machines to other ESX Servers, Linux or FreeBSD servers. Of course, virtual machines can also be stored to a SAN (ISCSI or NFS) directly attached to an ESX. Furthermore, backups can also be saved on the Windows machine where the VM Explorer is running.
• Support for ESX 3i
  VMX version 1.5 includes beta support for the next generation of ESX servers.
• File Browser
  Browse instantly your ESX/Linux/FreeBSD servers’ file system and copy files from one to the other just by drag & drop. Supports also up- and download to the management station.
• Integrated SSH Client
  The integrated SSH Terminal Client allows you to configure all your Unix/Linux based Servers trough SSH, all within one software.
• Job Scheduler
  The integrated job scheduler allows you to perform unattened, periodic backups of your virtual infrastructure.
• Integrated ESX Snapshot Manager
  Create and remove snapshots directly from within VMX.
• Dynamic Compression
  To make use of slow network links (e.g., for wide-area replication) or to save space on the target system, backups can be dynamically compressed.

VI Client Datastore Browser

The VI Client (VIC) comes with every version of ESX/ESXi. It’s really surprising all that you can actually do with it. Although not meant to be a post about the VIC, Jason over at boche.net provides some great examples of how to simulate some VirtualCenter functionality with VMware’s standard client. Check out Jason’s post titled Virtualization rematch corrections and clarifications to see examples of how the VIC can:

• Upload and download files
• Rename objects
• Clone VMs

Let me know if there are other free alternatives that are worth adding to my VI tools usb stick.