E8 Storage uses Mellanox’s Ethernet Storage Fabric in Record Breaking Benchmark!

Ethernet Storage Fabric

The following is a guest blog by E8 Storage.

There’s nothing like an established, peer reviewed benchmark such as SPEC SFS®2014 to prove what you’ve been saying all along: E8 Storage delivers high performance and low latency at a fraction of the cost of competing solutions. This means that data-intensive applications can rely on E8 Storage to deliver real results and not just hype.

What is SPEC SFS2014?

SPEC SFS has long been the benchmark of choice for measuring file system storage performance and a strong resource for comparing competitive file storage offerings.  SPEC SFS2014 is the 3rd generation of the benchmark and was a major rewrite to move the focus away from measuring the storage component itself, and instead focuses on the end-to-end storage solution for specific workloads.  SPEC.org requires all benchmark results to disclose the hardware, software, and network configuration used, and mandates that only generally-available solutions be used to generate the published results. Results are reviewed by a SPEC committee before publication on the SPEC.org web site.

The benchmark can be run against any of the 5 available workloads which are meant to simulate a wide range of applications needing storage performance. E8 Storage chose to focus on the Software Build (SWBUILD) workload primarily because the results could be applied to various High Performance Computing (HPC) workloads.


Mellanox’s Ethernet Storage Fabric – the foundation for the record-breaking performance

A high-performance storage solution requires a network fabric that is designed to deliver consistently low latency and high bandwidth, with available offloads that remove CPU bottlenecks.  Not every datacenter switch can be used to create an Ethernet Storage Fabric (ESF). An ESF is run on purpose-built switches which are optimized to deliver the highest levels of performance, lowest latencies and zero packet loss, with unique form factors and storage-aware features that provide access to remote data with the same performance as access to local data, from the application’s perspective. This translates into close-to-local predictable latency, line-rate throughput with QoS, and linear scalability to accommodate dynamic, agile data movement between nodes – Mellanox’s Spectrum switches provided the end-to-end network foundation for an ESF and contribute tothe record-breaking performance achieved by E8 Storage.

Mellanox’s ConnectX®-5 NICs were used in each of the 16 hosts to generate storage traffic, and a Mellanox SN2700 100G Ethernet switch was used to connect these hosts to the single E8-D24 Storage Appliance.  The E8-D24 has 8 *100G Ethernet ports (4 ports to each of its 2 controllers) which are powered by ConnectX-5 NICs within. All of the equipment was connected using Mellanox LinkX® DAC cable products. E8 Storage’s multi-patented scale-out software architecture and technologies enabled the NVMe-oF-based storage solution to use Mellanox end-to-end RoCE (RDMA over Converged Ethernet) technology, as shown below, to deliver the record-breaking storage system performance.

The Results

E8 Storage smashed the existing benchmark!   Not only did they complete more builds than any previous results, the solution delivered more throughput and IOPS at a fraction of the latency of previous submissions. E8 Storage achieved the lowest ever latency (overall response time) at the time, and still has one of the only sub-microsecond response times of all submissions to date.


E8 Storage achieved great performance on the SPEC SFS2014_swbuilds benchmark. This chart shows how the response time (latency) of the solution goes up as the number of IOPS increases, until latency becomes unacceptably high. Using the E8 and Mellanox solution, the benchmark achieved:

  • More builds – over 2.5x more than the previous IBM Spectrum Scale test
  • More performance – Almost 300,000 IOPS
  • Lowest latency – a record breaking 0.69ms!


Now, one of the criticisms of standardized benchmarks is that vendors can try to game the system with special performance tuning that isn’t available—or isn’t affordable–to customers.  Not so with our results.  The E8 Storage Controller had no performance tuning; that is no special tweaks and no impractical configurations: just great performance out of the box.

Here’s what we tested:

  • 1 x E8-D24 NVMe Appliance – Our flagship appliance with 24 HGST SN200 NVMe drives and dual controllers for high availability
  • E8 Storage Software version 2.1.1 – Our latest GA software release
  • IBM Spectrum Scale 5.0 – The most up to date software for IBM’s high performance distributed file system
  • 16 x Spectrum Scale Nodes – The Spectrum Scale cluster nodes all connected directly to the shared read / write volume on the E8-D24
  • 16 x Mellanox ConnectX5 NICs – one per Spectrum Scale cluster node
  • 1 x Mellanox SN2700 100GbE Switch – high-performance interconnect
  • 8 x Mellanox LinkX 100GE AOC cables
  • 16 x Mellanox LinkX 100GE to 2*50GE DAC splitter cables.



Mellanox’s Ethernet Storage Fabric and E8’s D24 NVMe Appliance were the foundation for this small, energy-efficient and relatively inexpensive system that smashed the existing SPECsfs benchmark.

Mellanox’s ConnectX-5 NICs delivered high bandwidth and provided a robust implementation of RoCE that is a key component of this NVMe-oF-based system.

Mellanox’s SN2700 Ethernet Switch delivered the non-blocking high performance and consistently low latency across all of its ports that was also necessary to enable the new benchmark.

Finally, here’s what some of the press have been saying about this performance:


Comments are closed.