Feeding the Data Beast with PCIe 4.0

 
Adapters, Uncategorized

With growing Ethernet speeds, PCIe 4.0 is ready to stay ahead of the curve as a x16 slot can provide enough bandwidth for a 200 Gb/s adapter.

Summary

Artificial Intelligence, Virtual Machines, containerization, and 5G mobile wireless networks are key drivers for next-generation high-performance systems. However, current servers with PCIe Express (PCIe) 3.0 require wide busses to keep up with the latest Ethernet or InfiniBand speeds or with the performance demands of new NVMe solid-state drives (SSD).

For example, the bandwidth of an 8-lane PCIe 3.0 interface supports a single 40 Gigabit Ethernet connection but creates a bottleneck with dual ports and at greater speeds. Likewise, PCIe 3.0 is already seen as a speed limitation for SSDs. A faster solution is required since increasing lane width with 3.0 is not efficient in terms of cost, complexity, higher power, complex circuit board layout, and component fanout.

To meet the requirements of data-intensive applications, PCIe 4.0 doubles the bandwidth of servers, creating a superhighway to increase a server’s data handling capacity to 64 GBytes/s. This will dramatically enhance data access capabilities so that servers will be able to analyze more data, more effectively for real-time insights, and access shared data quicker to fully utilize NVMe drives. Other advantages include a smooth transition for 4K and 8K video enabling four times sharper video footage and a platform that can manage the traffic that will be created by new 5G mobile networks.

Why is  PCIe 4.0 needed?

CPUs and GPUs are manipulating ever-increasing data sets. Flash storage and NVMe drives are far faster than spinning media from the past therefore, media and entertainment companies must support higher-definition content and interconnect speeds are quickly moving towards 200 GBytes/s. All this means an improved local I/O mechanism is needed to keep system latency low and to prevent bandwidth bottlenecks. PCIe 4.0 provides faster speeds to handle the greater bandwidth that new and more powerful component and applications demand.

To explain this further, let’s look at a 40 GBytes/s Ethernet adapter to understand why Gen 4 is needed. The industry standardizes on PCIe 3.0 x8 for a standard 40 Gb adapter. The max bandwidth of a single PCIe 3.0 lane is 985 MBytes/s (or 7.88 Gb/s). Multiply that by 8 to get the max bandwidth of a x8 slot and a PCIe 3.0 x8 slot can provide 63.04 Gb/s, plenty to handle a single port adapter. However, there is not enough bandwidth for a dual-port adapter. Another example is the huge growth in M.2 NVMe SSDs that utilize PCI Express connectivity. The x4 M.2 NVMe SSDs using PCIe 3.0 peak at 3.94 GBytes/s, a number which will double to 7.88 GBytes/s with PCIe 4.0. It’s easy to see that most data center will see a real benefit from PCIe 4.0. With Ethernet moving from 100 Gb/s speeds, which are currently available, to 200 Gb/s at the end of this year the additional bandwidth of PCIe 4.0 arrives just in time to stay ahead of the curve as a x16 slot can provide enough bandwidth for a 200 Gb/s adapter.

Conclusion

In today’s era of data-centric computing, to truly deliver superior performance, data center servers must be designed differently. They need advanced I/O buses with enhanced bandwidth and latency capabilities to deliver higher capacity for data-intensive workloads. PCIe 4.0prepares a data center to handle the new demands that specialized applications need and provides a foundation to support high-performance and efficient network connectivity to support a wide range of link speeds and evolving network acceleration engines such as SmartNICs. In turn, SmartNICs will complement the performance boost by offering acceleration engines for specific functions like networking, security, and storage to alleviate the CPU from the burden of I/O tasks.

The exponential growth of data demands not only the fastest throughput but also smarter networks. Mellanox intelligent interconnect solutions incorporate advanced acceleration engines that perform sophisticated processing algorithms on the data as it moves through the network. Intelligent network solutions greatly improve the performance and total infrastructure efficiency of data-intensive applications in the cloud such as cognitive computing, machine learning and Internet of Things (IoT). Mellanox stands out as the first company with adapters to support PCIe Express Gen 4.0 and achieved that milestone in 2017!

 

Supporting Resources:

 

 

 

 

About John Biebelhausen

John Biebelhausen is Director, OEM Marketing at Mellanox Technologies, Inc. Before joining Mellanox, John had an extensive career as a Marketing Executive in a range of technology companies including IBM, Lenovo, Dell and Sharp Labs. John was a founder in multiple start-up companies pioneering Direct Relationship Marketing and SaaS applications. John holds a MS of Science in Finance from Colorado State University and a BBA in Economics and Finance from Kent State University.

Leave a Reply