Ethernet Storage Fabric – Part 1

Ethernet Storage Fabric

Simple, Efficient and High-performance Network Fabric for Scale-out Storage and Hyperconverged Infrastructure

The volume of data, in both structured and unstructured forms, is growing rapidly in datacenters. Applications that generate or consume this data are being developed and deployed across geographic regions. All these are calling for a storage infrastructure that is fast to expand with rapid data growth, agile to accommodate application performance requirements and virtualized infrastructure, and efficient to operate at scale. Traditional storage, exemplified by complex, expensive, and proprietary SAN-based storage systems, cannot meet these requirements. As a result, modern datacenters are breaking away from the “Big-Box” storage model, and migrating to scale-out, software-defined storage (SDS) and hyperconverged infrastructure (HCI), where it’s fast to deploy, elastic in scale, and flexible in provisioning.

Scale-out SDS and HCI build on industry standard servers, with a control plane allocating and managing resource pools on demand. Distributed and software-defined in nature, they are designed to deliver guaranteed performance to applications and services, expand on-demand to handle exponential data growth, and to simplify operations and management. By removing the bottlenecks and complexity in standard storage, scale-out storage and HCI lay the foundation for today’s cloud infrastructures – private, public or hybrid – to achieve ultimate cost and operational efficiency while meeting ever-increasing demands for performance and capacity. However, by adopting scale-out storage or HCI, you are only halfway through your data center transformation. The network fabric connecting scale-out storage and converged infrastructure also needs to be “modernized” before one can fully realize these benefits. Herein comes the reason you need an Ethernet Storage Fabric (ESF).

 What is an Ethernet Storage Fabric?

An Ethernet Storage Fabric, or ESF in short, is the fastest and most efficient way to network storage. It leverages the speed, flexibility, and cost efficiencies of Ethernet with the best switching hardware and software. It comes packaged in ideal form factors to provide performance, scalability, intelligence, high availability, and simplified management for storage.

An ESF is optimized for scale-out storage and HCI environments because it is designed to handle bursty storage traffic, route data with low latencies, provide predictable performance to maximize data delivery and allow for simplified scale-out storage architectures, and support storage aware services. These are all crucial attributes for today’s business-critical storage environments. In particular, the switches must support new, faster speeds including 25, 50, and 100GbE. They must have an intelligent buffer design that ensures fast, fair, consistent networking performance using any combination of ports, port speeds, and packet size.

Additional ESF attributes include support for not just block and file storage, but also for object based storage, along with storage connectivity for the newest NVMe over Fabric arrays. Additionally, an ESF must provide support for storage offloads, such as RDMA, to free CPU resources and increase performance. Not only is an ESF specifically optimized for storage, but it also provides better performance and value than traditional enterprise storage networks.

I’ve mentioned the need to support simplified scale-out designs, which will be expanded on

I’ve mentioned the need to support simplified scale-out designs, which will be expanded on in Part 2 of this blog.

The Benefits of an Ethernet Storage Fabric

ESF delivers the scale-out storage/HCI traffic in a faster, smarter and much simpler way.

Faster, guaranteed Performance. The ESF is a dedicated network fabric for scale-out storage and HCI. The congestion, increased latency, and unpredictable performance caused by traffic aggregation in the traditional three-tier network is now gone. Within the datacenter, any storage/HCI I/O transverses the ESF in a single hop if the end points are in the same rack, or in just three hops if across racks. As long as dedicated ESF switches are used to construct the fabric (we will come back to this point later on), storage and HCI traffic, including bursty I/Os, always reaches its destination with a predictable response time. With RDMA over Converged Ethernet (RoCE) offload and native NVMe over Fabrics (NVMe-oF) acceleration, applications are serviced at a highest performance level, in accordance with SLAs or predefined policies.

Simple to Deploy, Manage, and Scale. Ethernet is ubiquitously used in datacenters, and easy and rapid to expand. By converging all network and storage traffic within scale-out storage and HCI environments onto Ethernet, ESF eliminates network silos (such as Fibre Channel used with legacy SAN), resulting a single network fabric to manage. Beyond the boundary of a single datacenter, the use of overlay technologies such as VXLAN/EVPN which create efficiencies allowing expansion across multiple datacenters.

Automation, Security, and Storage-aware QoS. An ESF provides automated network provisioning, monitoring and management for virtualized workloads and storage traffic. Seamlessly integrated with clouds, ESF supports secure and isolated workspace for multiple tenants on scale-out storage and HCI. Combined with the intelligence in auto-discovering storage devices on the fabric, and allocating proper network resources for storage-aware QoS, the ESF delivers a non-disruptive and transparent network fabric for business continuity of business applications.

Cost-Effective. Ethernet is de-facto network in datacenters and clouds. Wide usage and high volume shipments have driven down the hardware cost, while ensuring rapid technology innovation and enterprise-class quality. Furthermore, innovative and scalable management tools and automation software for configuration, monitoring and troubleshooting have grown out of the huge Ethernet networks deployed by both enterprise and cloud customers. These management tools significantly reduce operational cost for managing scale-out storage and HCI. Easy application migration over a single fabric with automation tools maximizes uptime and resource utilization, also lowering operations costs.

Containers and Docker. The move to modern datacenters is driving new and dynamic operation models. An ESF must also provide a wide range of tools to address these needs. For example, support for Docker containers which enable software to be run in isolation. This provides faster and secure delivery of customized applications, giving customers a unique edge to quickly integrate and improve development cycles and share storage resources between containers.

Ethernet Storage Fabric vs. Fibre Channel

Naturally, as a seasoned IT professional, you may ask what has happened to Fibre Channel (FC)? Why is ESF, and not FC, the de-facto fabric in modern datacenters today?

My colleague, John Kim, bravely called out the demise of FC in his inspirational blog a couple years ago. As shown in the charts on the right, FC port shipments have continued their downward spiral, while Ethernet shipments have kept rising. Technology developments John listed in his blog have already taken place, including the arrival of mainstream 25/100GbE, the emergence of fast storage such as NVMe and 3D Xpoint SSDs, the growing adoption of object storage, and the convergence to public, private, and hybrid clouds.

Fibre Channel innovation has stagnated and it remains a block-only storage solution deployed only in the enterprise; it has no use in the cloud, big data, machine learning, or HCI. ESF is the only fabric adapted for these technological advances within modern and future datacenters and cloud deployments. And it does so at one third the cost of Fibre Channel at three times the performance.


An Ethernet Storage Fabric (ESF) leverages the speed, flexibility, and cost efficiencies of Ethernet to provide the foundation for the fastest and most efficient way of networking storage. The ESF has everything a traditional SAN offers but …faster, smarter, & much simpler.

An ESF is run on purpose-built switches which are optimized to deliver the highest levels of performance, lowest latencies and zero packet loss, with unique form factors and storage-aware features. Continue to read Part 2 of this blog, where we will discuss how to build an ESF network in the right way.

You can find more technical details about Mellanox Spectrum Switches in the Mellanox Community and on

Follow us on Twitter: @MellanoxTech

Related Resources

About Jeff Shao

Jeff Shao is Director, Ethernet Alliances at Mellanox Technologies. Prior to Mellanox, he held senior product management and marketing roles at LSI (Avago), as well as Micrel, Vitesse Semiconductor & Promise Technology. He holds a MBA from University of California, Berkeley and a Bachelor of Science in Physics from University of Science & Technology of China.

Comments are closed.