New Low-Loss DAC Cables Enables Reduced FEC and Low Switch Latency for Server Links

 
Link-X

Mellanox’s LinkX line of Ethernet and InfiniBand Direct Attach Copper (DAC) cables is adopting new low-loss cable material based on the IEEE 802.3bj and by spec called CA-N and CA-L.

Offering improved shielding, it significantly reduces the 25G signal losses in the cable. This delivers several key benefits:

  • Swapping the Forward Error Correction (FEC) RS-FEC algorithm in the switch to a low-latency FireCode (BASE-R) FEC saving approximately 30% latency or manually turning off FEC entirely saving upward of 100- 120ns latency in each direction.
  • Use of thinner 30AWG wire in place of thicker 28AWG wire making the cable easier to bend and route in crowded racks, with less weight and increased airflow.
  • Maintains the 5-meter reach at 25G and provides better signal integrity to support future 50G PAM4 signaling for 200G and 400G DAC networks
  • Continues offering pre-FEC Bit Error Ratio (BER) rating of 1E-15, which is about 1,000 times better than the IEEE standard specification of 1E-12 post-FEC

 

Green Cables – that are actually Black…Huh?

We made the cable “greener” meaning lead-free, halogen-free and the cable jacket material switched from PVC to Low-Smoke, Zero-Halogen (LSZH), important when things catch fire in data centers. However, they are still black!  In short, PVC is on its way out as a cabling material.

 

New Pull Tab Design

We are changing the pull tabs styles to reduce air flow restrictions and the tab fluttering vibrations that often happens in racks with high air flow.  Also, we are using different colored tabs for break-out link ends for easier identification.

 New Open-Air Flow, Colored Tabs

New Open-Air Flow, Colored Tabs

 

The new cable material is being used in all our InfiniBand and Ethernet 25G and soon 50G PAM4-based DAC cables for 100G, 200G and 400G.

Mellanox 25G NRZ-based Ethernet DAC Cable Product Line

Mellanox 25G NRZ-based Ethernet DAC Cable Product Line

 

What the Heck is FEC?

In the electrical PHY transmitter portion of a switch port, an Error Correction algorithm in run against the data in the packet before transmission. It is then Forwarded with the data and decoded in the receiver to detect and even correct bit errors in the transmitted stream. Hence, name FEC or Forward Error Correction.

However, constructing and deconstructing FEC take approximately 100-120ns per direction, 60ns to initially calculate at the transmitter and 60ns to compute at the receiver and that is for only one direction of a 2-way link.

The new Phase-3, type-N cables (N=No-FEC) enables automatically configuring to a low-loss FEC (BASE-R) or one can manually configuring the switch to turn off the FEC altogether.

A few nanoseconds here and there times 25 billion bits per second, times 10,000 cables, operating at 24/7 for 5 years can really add up as this area is where all the sever data transfer action is!

 

New IEEE Specs

The new specification created three different types of Cable Assemblies (CA-) based on the overall signal losses.

  • Up to 2 meters: CA-N (No FEC) or reduced FEC algorithms (15.5bB max loss)
  • 5 and 3-meters: we offer two versions supporting either CA-25G-N or CA-25G-L
  • 3-to-5 meters: CA-L (Long) with RS-FEC (Reed Solomon) enabled (22.48 dB max loss)

Mellanox uses only two specs as the Type CA-S (16.48 dB max loss) is for only the 4-meter case which is not very popular.

The new cables don’t save latency themselves but when used in the switch-to-adapter link is where the benefit is derived by reducing or eliminating the FEC delay in the switch link.

 

Low BER Rating More Important than Ever

Unlike many optical transceiver and switching system manufacturers, Mellanox designs and manufactures its own DAC cables as well as optics, network adapters and switching systems. This gives Mellanox incredible control over quality control and cable testing.

Mellanox builds its InfiniBand and Ethernet DAC cables to a bit error ratio of 1E-15 without the use of FEC in the switch. That is one-bit error every 10^15th bits transmitted expressed as 1E-15. The IEEE 25G/100G 802.3 Ethernet spec requires the use of FEC with post FEC ratings of a minimum of 1E-12.  Pre-FEC this computes to 1E-5 which is an incredible number of errors for electronics to deal with and can easily be overloaded forcing repetitive data retransmissions.

The IEEE recently introduced the CA-N new standard for short reach DACs that reduces the FEC requirements for less than 2 meter likes to servers and storage. This is where DAC cable quality becomes incredibly important because there is no FEC to correct any bit errors.

Mellanox uses the same DAC manufacturing techniques for both InfiniBand and Ethernet which enables building Ethernet DACs to the 1E-15 BER pre-FEC rating. Over 1,000 times lower bit error rates than competitors DAC cables at 1E-12.  Lastly, as Mellanox also builds switches, we test every cable in actual switching systems under live running conditions found in harsh data center environments and not simulated with benchtop test equipment and manual operators as many competitors do.

This guarantees the highest DAC cable signal integrity and reliability possible. DAC links are when most of the rack traffic take place between servers, switches and storage so reducing latency is very beneficial as this is the highest value-add traffic in the network.

 

Summary

DAC cables are the lowest-cost, lowest power high-speed interconnect for linking switches to network adapters used in high traffic servers and storage subsystems. Now, with the new cable material, DACs enable building the lowest latency switch links, thinner cables, colored pull tabs and LSZH jackets for the best in-rack DAC cabling infrastructure where the highest value-add date traffic takes place.

 

Supporting Resources:

 

About Brad Smith

Brad is the Director of Marketing at Mellanox, based in Silicon Valley for the LinkX cables and transceivers business focusing on hyperscale, Web 2.0, enterprise, storage and telco markets. Recently, Brad was Product Line Manager for Intel’s Silicon Photonics group for CWDM4/CLR4 and QSFP28 product lines and ran the 100G CLR4 Alliance. Director of Marketing & BusDev at OpSIS MPW Silicon Photonics foundry. President/COO of LuxSonar Semiconductors ( Cirrus Logic) and co-founder & Director of Product Marketing of NexGen, a X86-compatible CPU company sold to AMD - now the X86 product line. Brad also has ~15 years in technology market research as Vice president of the Computer Systems group at Dataquest/Gartner; VP/Chief Analyst at RHK and Light Counting networking research firms. Brad started his career at Digital Equipment near Boston with the VAX 11/780 and has served as CEO, president/COO and on the board-of-directors of three start-up companies. Brad has a BSEE degree from the University of Massachusetts; MBA from University of Phoenix and holds 2 optical patents

Comments are closed.