There are definite pros and cons for using both VMFS and RDMs. This post suggests the 3 most common practical reasons (in my opinion) to use RDMs. That is, reasons that benefit VI administrators, leverage a VM’s native operating system,  or take advantage of technologies and process designed for physical environments. I also provide some links to a few other blogs that have recently explored the same RDM vs VMFS topic as supporting examples.

First, if proof is still needed that performance should not be a factor in the VMFS versus RDM decision, check out VMware’s own PDF titled Performance Characteristics of VMFS and RDM.

So what are some practical reason for using RDMs?

Use RDMs when VMFS virtual disk would become too large to effectively manage.

For example, When a VM needs a partition that is greater than the VMFS 2 TB limit is a reason to use an RDM. Large file servers, if you choose to encapsulate them as a VM, are a prime example. Perhaps a data warehouse application would be another.

Stephen Beaver provides his guidelines on when RDMs make sense for large file management in a recent post on the Virtual Black Hole blog:

To Use RDM’s or Not To Use RDM’s

“I try to use RDM’s on anything larger than one terabyte.  This is strictly based on management of the data and nothing else.  For example, let’s say we have a file server with a one terabyte data drive.  If I ever have to move that virtual machine to another cluster or had to move to another LUN, or insert your own reason here, then how long would it take to move that vmdk file?   Way too long is the correct answer.”

Leading into the next reason, if you know that you may have to move a large VM between LUNs, then an RDM may make your task a lot easier when the time comes. (if moving between LUNs on the same SAN)

Use RDMs to leverage native SAN tools

SAN snapshots, direct backups, performance monitoring, and SAN management are all possible reasons to consider RDMs.

Again, In the previously mentioned Virtual Black Hole post, Stephen used a virtualized instance of an Exchange Server to illustrate RDM advantages when using SAN snapshots:

“If that terabyte of data was on a RDM then I could use the native SAN tools to snap the LUN and move the data in a much quicker rate.”

Sean Clark also mentions RDMs for taking advantage of SAN utilities in his post VMFS vs RDM? Does Miller Lite taste great or is it less filling?

“…except in instances where we would like to take advantage of application aware SAN utilities for backups and/or SAN snapshots.”

Use RDMs for virtualized MSCS Clusters

Actually, this is not a choice. Microsoft Clustering Services (MSCS) running on VMware VI require RDMs. Clustering VMs across ESX hosts is still commonly used when consolidating hardware to VI. The actual need for VM MSCS Clusters is debatable with VMotion and DRS/HA, but that’s another topic.

Jason Boche recently posted Setup for Microsoft cluster service which explains the needs for RDMs. Jason provides a great run through of all the requirements for MSCS on VMware as well.

The section on disk I/O made me think – specifically about whether creating RDMs to raw disks was really a better choice than carefully planning VMFS LUNs. There is just too much convenience with using VMFS in my opinion, and I’ve never really been convinced that any report or testing has proven RDMs offer better performance than carefully planned VMFS. So, instead of a long comment on the vm0 post I decided to blog about it here at VM /ETC.

Be sure to read the entire post for the RAM and CPU recommendations, but here is the section that inspired me:

“I/O

I/O is by and large the biggest concern
when running a database on an ESX cluster. Your main concern here is
going to be ensuring that your database files are not impacted by I/O
operations on your other VMs. We can ensure this through Raw Device
Mappings.

Generally, for a high-performance database, you’ll want your data files, log files, and program/OS/backup files totempdb all live on separate spindles. To accomplish this, we do the following:

Data Files – RAID5 or RAID10 (RAID5 is more economical but you will suffer a hit on write performance).

Log Files – RAID10

TempDB – RAID10

OS/Program Files/Backup/File Share – This can be run from a standard VMDK.”

But do you really need RDMs? What about multiple, but dedicated and separate LUNS on the same, larger RAID5 or RAID10 arrays? Give the SQL server it’s own separate VMFS LUNs to spread out .vmdks across. The multiple .vmdks controlled by different virtual SCSI adapters combined with the fact that each LUN has it’s own I/O queue should provide for more than sufficient I/O performance. Add as many disks as possible to the larger arrays so you increase your spindle count.

Sure, if you have such a big data warehouse that tweaking the .vmdks and VMFS as described above won’t help it then probably the next best thing to do would be to dedicate an entire ESX host just for that “monster SQL VM”. Actually, dedicate 2 ESX hosts so you can have a “Monster SQL VM ESX Cluster”. You’d be looking at spending the cash for a multi-node physical cluster anyways, right? This way you get the mobility and conveninence of VMotion and all the administrative benefits of a VM.

As always, I ask, YOU, the vmetc.com readers to chime in with your opinions, preferences, and experience with heavy I/O virtual infrastructure design.