Another milestone in the life of NVM Express over Fabrics (NVMe-oF) technology has occurred. NetApp has become the first large enterprise external storage system vendor to support NVMe-oF. Late last month they announced their new E-5700 hybrid storage array and EF-570 all-flash array. Both feature front side NVMe-oF connections to clients. Only four months have passed since I wrote about the last NVMe-oF milestone (first UNH NVMe-oF interoperability plugfest), a successful multi-vendor interoperability test at the University of New Hampshire InterOperability Laboratory (UNH-IOL). It is not unusual for a new technology’s milestones to cluster closer together as its evolution to mainstream nears completion and accelerates. Large enterprise storage vendor support is extremely important for a new technology because it signals to main stream data centers that it is becoming mature and less risky to try out and deploy. It also stokes the competitive fires with the other enterprise storage vendors, creating a domino effect of initial support, then expanding features, and innovation.
NVMe-oF Technology
For a refresher on NVMe-oF, my colleague John Kim, has authored several blogs detailing how the technology enables the new high performance SSD interface, Non-Volatile Memory Express (NVMe), to be connected across RDMA-capable networks. A YouTube video description is also available.
I attended NetApp’s Insight conference a few weeks ago and got a firsthand look at the E-5700 and EF-570.
Figure 1: The NetApp EF-570 all-flash array now supports NVMe-oF
On the conference show floor, NetApp had an impressive demo of the EF-570’s NVMe-oF capabilities, showing 21GB/s of bandwidth and 1 million IOPs with less than 100µs of latency.
Several startups and Original Design Manufacturers (ODMs) are already shipping NVMe-oF solutions with impressive performance using a Just-a-Bunch-of-Flash (JBOF) or hyperconverged (Server SAN) configuration. And some enterprise storage vendors, such as Pure, have NVMe-oF now at the “back end” of their storage arrays, connecting storage controllers to shelves of NVMe SSDs. But, until now, none of the major array vendors offered generally available NVMe-oF connectivity to client/servers.
Figure 2: NetApp EF-570/E-5700 can connect to Linux servers at 100Gb/s using Mellanox InfiniBand adapters and switches.
Conclusion
With the completion of another millstone for NVMe-oF, the technology seems to now be accelerating to mainstream success. A widely supported open industry standard, tested for multi-vendor interoperability, supported by mainstream enterprise storage vendors, with performance to match the new Non-volatile memory storage technologies. What’s not to like?
Resources:
- Read more on: NetApp EF-series product information
- Learn about: NetApp EF-570
- Check out: NetApp E-5700 and EF-570 press release
- Read: Pure blog NVMe-oF
- Catch up with the Rob Davis: NVMe over Fabrics Standard is Released
- John Kim NVMe-oF blogs:
The open source NVMe-oF Linux driver software was used on both the Target and Initiator platforms. The NICs used the RoCE (RDMA over Converged Ethernet) protocol to provide the RDMA capability required by the NVMe-oF driver software. All NIC vendors were aggressively encouraged to attend by UNH-IOL. BTW, at the first Fibre Channel UNH-IOL event, there were just a handful of startups, Ancor (mine), Gadzooks (great name), and perhaps a couple others. The NVMe-oF testing this week was completely successful with our ConnectX®-4 adapter and other vendors’ NICs being totally interoperable as ether the NVMe-oF Target or Initiator. Near line rate performance for these adapters of 25Gb/s was also achieved.
Figure 1: Newer storage is exponentially faster than older storage.
Figure 3: RDMA allows direct, zero-copy and hardware-accelerated data transfers to server or storage memory, reducing network latencies and offloading the system CPU. 
Figure 5: Mellanox 
