How RDMA Increases Virtualization Performance Without Compromising Efficiency

Virtualization, , ,

Virtualization has already proven itself to be the best way to improve data center efficiency and to simplify management tasks. However, getting those benefits requires using the various services that the Hypervisor provides.  This introduces delay and results in longer execution time, compared to running over a non-virtualized data center (native infrastructure). This drawback hasn’t been hidden from the eyes of the high-tech R&D community seeking ways to enjoy the advantages of virtualization with a minimal effect on performance.

One of the most popular solutions today to enable native performance is to use the SR-IOV (Single Root IO Virtualization) mechanism which bypasses the Hypervisor and enables a direct link between the VM to the IO adapter. However, although the VM gets the native performance, it loses all of the Hypervisor services.  Important features like high availability (HA) or VM migration can’t be done easily.  Using SR-IOV requires that the VM must have the specific NIC driver (that he communicates with) which results in more complicated management since IT managers can’t use the common driver that runs between the VM to the Hypervisor.

As virtualization becomes a standard technology, the industry continues to find ways to improve performance without losing benefits, and organizations have started to invest more in the deployment of RDMA enabled interconnects in virtualized data centers. In one my previous blogs, I discussed the proven deployment of RoCE (RDMA over Converged Ethernet) in Azure using SMB Direct (SMB 3.0 over RDMA) enabling faster access to storage.

In addition, few companies have already started to work on advanced solutions that will allow running VM-to-VM communication over RDMA.  Among them, VMware that announced at the 2014 OpenFabrics International Workshop that they are working on virtualization support for host and guest services over RDMA.  They shared preliminary performance numbers showing they were able to get latency numbers close to native performance.

RDMA interconnects are also behind Dell’s Fluid Cache for SAN solution, which uses a distributed cache mechanism to minimize access to the physical storage. The solution demoed live at Dell World 2013, maximizes the cluster’s performance by enabling a VM access to a data stored in another VM’s cache. This requires high bandwidth and low latency interconnect.

Motti Blog 061614

Source: Dell


The end results are pretty impressive.  In 2013, Dell showed that the solution reached 5 million random read IOPS* (input/output per second) at their Dell World event. In Dell lab tests of an 8 node cluster test of an online transaction processing (OLTP) Oracle workload simulation, on a Dell hardware stack, which according to the company, Dell Fluid Cache for SAN achieved a 99% faster response time, a -4X increase in transactions per second, while also supporting a 6X increase in concurrent users – increasing from just 2,200 to 14,000 concurrent users. Dell has also seen significant performance improvements via Dell lab tests of a hardware stack enabled with Dell Fluid Cache for SAN running Microsoft® SQL Server® database on VMware® software.

The Dell Fluid Cache for SAN solution takes advantage of RDMA enabled interconnect, and we are not far from the day that all enterprise applications will run over virtualized environment will enjoy native performance without compromising on the efficiency and ease of management.


* A 2013 Dell lab test of Dell Fluid Cache for SAN achieved over 5 Million IOPS using a random read workload with a 4 KB sized block with a hardware stack consisting of: eight Dell PowerEdge R720 servers each containing two Dell Express Flash 350 GB PCIe SSDs, one Mellanox ConnectX-3 40GE RoCE card, and Fluid Cache software; all connected to Dell Networking and Dell Compellent Storage solutions. Actual performance will vary based on configuration, usage and manufacturing variability.

Motti Beck
Motti Beck is Director of Marketing, EDC market segment at Mellanox Technologies. Before joining Mellanox, Motti was a founder of several startup companies including BindKey Technologies that was acquired by DuPont Photomask (today Toppan Printing Company LTD) and Butterfly Communications that was acquired by Texas Instruments. He was previously a Business Unit Director at National Semiconductors. Motti holds a B.Sc in computer engineering from the Technion – Israel Institute of Technology.

Follow Motti on Twitter: @MottiBeck


About Motti Beck

Motti Beck is Sr. Director Enterprise Market Development at Mellanox Technologies Inc. Before joining Mellanox Motti was a founder of BindKey Technologies an EDC startup that provided deep submicron semiconductors verification solutions and was acquired by DuPont Photomask and Butterfly Communications a pioneering startup provider of Bluetooth solutions that was acquired by Texas Instrument. Prior to that, he was a Business Unit Director at National Semiconductors. Motti hold B.Sc in computer engineering from the Technion – Israel Institute of Technology. Follow Motti on Twitter: @MottiBeck

Comments are closed.