June 2, 2019 Brad Smith, Director of Marketing, LinkX Cables and Transceivers
- Early first-generation 50G PAM4 lasers and ICs had some “issues” causing engineers to resort to using powerful DSPs to clean up the signals in the receiver.
- Next generation 50G PAM4 lasers and ICs have improved significantly raising the question, “Do we still need high-cost, power and latency consuming DSPs?”
- Returning to traditional analog electronics in the receiver CDRs yields benefits of lower cost, size, complexity, power and latency and return to a simpler, lower cost test procedure.
- IEEE is holding firm on DSPs, but a splinter group formed that believes DSPs are overkill.With the high number of optical transceivers used in data centers, there are tremendous pressures to lower the CapEx (product cost) and the OpEx (operating costs such as power consumptions) and maximize performance in reducing latency delays in the systems.The top 30 most important items on the transceiver buyers list of requirements are:#1-27 Low cost#28 Low power#29 Low latency#30 InteroperabilityThe Open Eye MSA is attempting to target all of these factors at once. All of these factors will be critically important with future co-packaged optics with chiplets and next-generation switch ICs for 25Tb and 51Tb.
Open Eye MSA Participants
The IEEE holds standards meetings that last several years in duration to get all parties to agree, then the standard is set and those that didn’t get their way or want to do something different go off and form splinter groups. Interestingly, sometimes these groups hit upon the most popular products such as the 100G PSM4, CWDM4 widely adopted in hyperscale systems.
Founding members of the Open Eye group are MACOM and Semtech.
Promoters and Contributors as of this writing include:
Analog-based receivers offer lower cost, lower power and lower latency – all benefits derived from not using a DSP and simpler testing techniques particularly dropping TDEQ testing in favor of more traditional Open Eye techniques, hence the group’s name.
The group plans to start with 53G single-mode optics, then multi-mode and later to target 100G PAM4 per lane for the next generation 400G and 800G transceivers. The plan is to be fully compatible with existing data center and enterprise infrastructure as the industry doesn’t need two versions of DR4, FR4, LR4 etc. transceivers. Making the interoperability happen will be challenging. The goal is to have the first specifications out by September ECOC and CIOE trade shows.
Key product targets would be the 50G PAM4-based multi-mode 100G/200G/400G SR2, SR4, SR4.2, SR8 and single-mode 400G DR4 (500m), FR4 (2 km), LR4 (10 km).
Signal “EYE” Diagrams
On an optical scope, overlapping displaying a pulse that rises from digital 0-to-1-to-0 with a signal starting from a 1-to-0-to-1 yields what engineers call a signal eye diagram. So-called because the pattern looks like a pair of eyes between two rails.
NRZ Signal with Open Eyes and Distorted Eyes
This displays any noise, reflections, rise and fall time edges, jittering and other parameters that distort transmission signals. The goal is to have wide open eyes that clearly distinguish each signal bit and approximate the square pulses. Noise makes this determination harder for the electronics and increases the probability of error bits.
PAM4 signal modulation
When PAM4 modulation first came into play for optical transceivers, it created a lot of controversary in the transceiver engineering and standards community on the viability of implementing it – especially at higher speeds such as 100G PAM4. PAM4 modulates the signal to four levels in the same time and amplitude space instead of two as with NRZ, hence is much harder to detect the data signals from the noise.
PAM4 is like Playing Tennis with 4 Nets!
To simplify the problem, think of signaling as a tennis game. With older NRZ all one had to do was hit the ball over the net. One side was a zero, the other was a one. Didn’t matter how far the ball went over the net 1” or 100 feet. Just get it over the net and it’s a 0 or a 1.
With PAM4, it’s a tennis game with 4 nets (4-signal levels)! And you can’t stray beyond the individual net area! Additionally, the tennis court is the same size, so everything is compressed. This makes signal detection much more difficult as noise can interfere much more.
PAM4 with DSPs
Digital Signal Processors (DSPs) have been used to convert the analog signal to digital and compute mathematics in the digital processor (a type of microprocessor) to clean up the signals before converting back to analog. This gives the programmer a lot of variable and math to play with. DSPs are very accurate but requires a DSP chip in the receiver which introduces higher cost, power and latency delays.
Today, early samples of 200G and 400G transceivers almost all use DSPs, but the debate is starting to rage on over whether DSPs are really needed going forward given the advancements in the quality of second generation of core lasers and other control ICs. As there are a hundred different variables to consider on thing for sure is this debate will go on for quite a while and even more so at 100G PAM4 where things are expected to get very hard.
The Open Eye MSA wants to develop transceiver specifications that interoperate with IEEE and DSP-based transceiver specifications but using analog electronics at lower cost, power, and latency and still maintaining interoperability with other devices, thereby meeting the buyers 1-30 criteria!
To be fair, a second-generation of DSPs are on the horizon that will offer perhaps lower power form smaller IC geometries and maybe a lower cost than the current generation, but the latency is likely to still exist the DSP is still a digital computer and the analog signals need to be converted to digital back and forth. Also, the next generation cost and power targets are still likely to be significantly above an analog approach as the DSP transceiver approach still requires an additional chip, additional power, latency and cost over the analog approach. The gaps may shrink but we’ll have to see how much as new ICs roll out and what the actual Open Eye implementation come out with.
Moving from non-return-to-zero (NRZ) with digital 1,0s to PAM4 brought on a new set of test techniques. The goal is to have nice clean open eyes. However, measuring the eye diagram became very complicated at 50G PAM4 as three eyes were compressed into the same voltage height as with NRZ. So, a new technique was introduced called TDECQ. TDECQ testing forces almost of the equalizations to be done in the DSP receiver instead of being split between the transmitter and receiver. Additionally, some companies have found that while the device may pass the TDECQ test, it can still fail in the field. Lastly, this type of testing is complicated and more expensive than the traditional Open Eye method the MSA wants to return to now that newer higher-quality lasers and control ICs are available.
MSA Time-to-market is essential
Many standards bodies with large groups can take years to come to a consensus – sometimes outliving the product usefulness of the product itself before the next generation kicks in. Both the hyperscalers and HPC communities are pushing very hard on the next generations.
The Open Eye group plans to have a small core group to develop the baseline draft specification in order to product the final specification as soon as possible and not get delayed with a large group as minimal time-to-market is essential. Once initial specifications are established the group will open up to a wider audience of contributors.
MSA Open Eye: Minimal Changes & Large
The change is in the transceiver electronics circuitry and should not impact the other areas of the optical and electrical infrastructure.
- Fibers: Compatible with existing single-mode and multi-mode fiber infrastructures and link budgets
- Optical connectors: compatible with existing MPO-12, MPO-16, LC Duplex, MOP/APC and the new Corning MDC and Senko CS/SN optical connectors for 400G
- Interoperability: The group believes that both analog and DSP-based transceivers should be able to interoperate if both are tested to the same test standards.
- Optical only: Both DSP and analog types of circuitry are used on the optical side so should not interfere with the electrical side in switches or network adapters.
- Testing: Changing testing requirement from using TDECQ to testing via Eye Opening technique equals simpler and fast testing and reduced costs since optical testing is a major contributor to cost.
- Re-timer usage: Enables using retimers
- Package indifference – supporting QSFP/QSFP-DD, SFP/SFP-DD, and OSFP/DSFP
The Open Eye group rough estimates with the following benefits:
- Significantly lower latency – no A/D and D/A conversion or DSP digital computations. Low latency is critical in High Performance Computing and hyperscale application such as maps, search and now becoming very popular with Ethernet AI applications. Estimates of 40-100X lower latency.
- 10%-25% lower power –no A/Ds, D/As or DSP electronics – fewer circuitry, less power
- Simpler module construction – besides the DSP, other components could be deleted
- Simpler manufacturing testing – optical testing is a major cost component in optical transceivers manufacturing accounting for 35-50% of the total cost. OE modules should be faster to test hence lower cost.
Mellanox’s Long History of Building Analog Transceiver Electronics
What is not well known is Mellanox designs and builds its own single-mode and multi-mode transceiver ICs using an 8- and 12-inch wafer, fabless manufacturing model. Additionally, we design and build our own DAC, AOC and transceivers as well. We have been designing our own transceiver ICs since the acquisition of IPtronics in 2013. We use these ICs in our SFP and QSFP transceivers and AOC cables and splitters.
Steen Gundersen, Vice President of LinkX cable and transceiver products said, “By taking advantage of the inherent benefits of fully analog module architectures enabled by the Open Eye MSA, Mellanox and our industry peers are helping datacenter and High-Performance Computing operators achieve the low latency necessary for latency-sensitive computing applications. Low-power and low-cost are two key attributes every transceiver buyer demands. Low-latency for both Mellanox’s Ethernet and InfiniBand product lines is critically important in High Performance Computing and Ethernet hyperscale systems. Mellanox has a long history in designing, fabless manufacturing, and selling analog transceiver ICs.”
In the next few months, both next generation DSP and new analog receiver designed transceivers will appear and the market place will make its call with its dollar votes based on the real value proposition. Both camps will argue strongly for their respective cases. Promises of “tomorrow’s IC will be better” will be bantered back and forth. The marketing “food fights” will start up! The real benefits versus estimated benefits will sort themselves out. No doubt, the debate will not end soon. But there appears to be a significant gap in cost, power, latency and simplicity between the two groups and these elements are key in the buyer’s minds. Interoperability and whether the Open Eye approach can be used at 100G PAM4 will likely be at the forefront.
About Mellanox & LinkX™
Mellanox Technologies, Ltd. (NASDAQ: MLNX), a leading supplier of optical transceivers and high-performance, end-to-end smart interconnect solutions for data center servers and storage systems. Mellanox offers a full line of cables and transceivers for hyperscale and datacenter applications:
- 25G/50G/100/200/400Gb/s DAC & AOC cables and transceivers
- 200G InfiniBand Quantum switches – 40 ports of QSFP56
- 200G Ethernet Spectrum-2 50G PAM4 switches from 16-to-64 ports
- 200G ConnectX-6 network adapters with 2 ports of 200Gb/s QSFP56 or SFP56
- Optical transceiver IC components TIAs/Laser Drivers, etc.
You can find more technical details about Mellanox DAC, AOCs cables, transceivers and ICs:
Contact your Mellanox sales representative for availability and pricing options and stay tuned to my blog more interconnect news and tips.
Sign up for free ebook download here: LinkX Cables and Transceivers ebook