Enable Higher IOPS while Maximizing CPU Utilization
As virtualization is now a standard technology in the modern data center, IT managers are now seeking ways to increase efficiency by adopting new architectures and technologies that enable faster data processing and execute more jobs over the same infrastructure, thereby lowering the cost per job. Since CPUs and storage systems are the two main contributors to infrastructure cost, using fewer CPU cycles and accelerating access to storage are keys toward achieving higher efficiency.
The ongoing demand to support mobility and real-time analytics of constantly increasing amounts of data demands that new architectures and technologies be used, specifically those with smarter usage of expensive CPU cycles and as a replacement of old storage systems that were very efficient in the past, but that have become hard to manage and extremely expensive to scale in modern virtualized environments.
With an average cost of $2,500 per CPU, about 50% of compute server cost is due to the CPUs. On the other hand, the I/O controllers cost less than $100. Thus, offloading tasks from the CPU to the I/O controller frees expensive CPU cycles, increasing the overall server efficiency. Other expensive components, such as SSD, will therefore not need to wait the extra cycles for the CPU. This means that using advanced I/O controllers with offload engines results in a much more balanced system that increases the overall infrastructure efficiency.
RDMA (Remote Direct Memory Access) is one of the most effective offload engines to be integrated into Mellanox’s ConnectX® I/O controller family. This engine enables significant I/O acceleration by allowing application software to bypass most layers of software and communicate directly with the hardware. Plus, it enables servers to directly place information into the memory of another computer, thereby reducing the application latency and minimizing the CPU overhead.
As for the storage system, the exponential growth of data has forced the industry to replace traditional SAN with higher capacity, easier-to-scale, and lower cost scale-out systems. This new architecture requires lower latency, which can easily be achieved when RDMA is used to access a distributed cache storage (mostly SSD-based). These new scale-out systems, such as Storage Spaces over SMB Direct, provide the same RAID capabilities as traditional SAN. Moreover, by using I/O adapters with offload engines such as erasure coding, which is often used instead of traditional RAID because of its ability to reduce the time and overhead required to reconstruct data, these scale-out systems can further increase the performance and lower the cost.
In order to meet customer needs, Dell and Mellanox worked together on a joint solution that increases CPU utilization and accelerates access to storage. The solution includes standard off-the-shelf components that reduce the Total Cost of Ownership. The solution components Included:
To compare the efficiency boost that RoCE enables, we compared the IOPS when running different workloads over 4KB and 8KB block sizes. Analysis of the results shows that in all workloads, the number of IOPS when running Storage Spaces over SMB Direct was higher than when running over TCP/IP.
To compare the efficiency in CPU utilization, we ran different workloads using a 512KB block size. The results show that when running over SMB Direct, Storage Spaces uses less than 50% of the CPU it uses when running over TCP/IP.
Faster access to storage with lower CPU usage enables more jobs per second to run over the same infrastructure, thereby maximizing the ROI.
Running Microsoft’s Storage Spaces over Dell PowerEdge servers and Mellanox 10GbE RoCE interconnect provides a much better alternative to TCP/IP-based networking by: