Going faster and faster encounters increasing complexities in electronics and optical physics. So, the transceiver industry is introducing several changes. Sending 2 data bits per clock, increasing the number of channels, and changing the transceiver packaging, and optical connectors. These changes combined with future line rate advances of 100G PAM4 will enable 400G and 800G per transceiver in the future. 1. New Signaling Scheme: For the last 35+ years, digital signaling has been based digital ones or zeros
with one bit per defined clock pulse (called NRZ). Now, the industry is moving to 2-bits per clock pulse by varying the amplitude of the pulse to four levels of 00,01,10,11 (instead of just 1,0) called PAM4 for Pulse Amplitude Modulation 4-channels. The entire infrastructure of switches, network adapters cables and transceivers must change to adopt this new technology. 50G PAM4 allows keeping the same low-cost 25GHz electrical infrastructure but transferring data at twice the rate with 2-bits per clock pulse or 2x25G=50G.
2. Move to 8-Channels: Increasing the number of channel from four to eight provides more aggregate bandwidth but makes everything larger and requires more electrical paths and thermal dissipation. 3. New Transceivers Packaging: QSFP28
supports 3.5-4.5W with four-channels whereas the new packages offer 8-channels and 12-15W support. In July, 2017, the SFP-DD MSA was announced to advance the SFP design from one channel to two channels and from 2W to 3.5W enabling up to 100G and 200G in an SFP-DD form-factor using 50G or 100G PAM4 signaling. Several industry groups are battling for leadership over the next transceiv- er package type. Cisco is leading the QSFP-DD charge and Arista is leading
the OSPF group. Each offer backward compatibility, cooling and connector different options and more cryptic buzzwords than ever before!
| page 33
eBook: New Developments in Data Center Cables and Transceivers
<< Back to TOC
Summary
Mellanox is one of the few companies in the business that designs silicon ICs and complete systems for network switches and network adapters as well as ICs for transceivers. This gives Mellanox a unique view into the complicating issues in optimizing total system performance.
In the middle of the diagram below are four of our most popular 25G and 100G Spectrum 32-bit 25G NRZ-based switches.
New is the SN2010 switch which offers eighteen 25G SFP28 ports and four 100G QSFP28 ports. This switch is designed for Ethernet storage as it is becoming clear, especially with NVMe FLASH over Fabrics (NVMeoF), that a network switch is different from a storage switch and plays to the new trend of storage-optimized, memory-centric computing. The SN2100 and SN2010 are half-wide switches and enable mixing and matching in a singe 1RU bay. Every Ethernet rack and row application for network switching can be addressed with these four products. A similar product line is available for InfiniBand switching.
The bottom side middle of this figure illustrates the various Mellanox ConnectX network adapters for 10G-25G line rates and 40G/100GbE quad channel adapters in both single and dual port configurations. Adapters are most often used to uplink servers and HDD, SSD and NVMe storage subsystems to Top- of-Rack switches. See the ConnectX sections of the Mellanox website for many more products and options.
On the left side of the adapters are shown DAC cables in the server/storage rack or what we call “DAC-in-the-Rack”. Virtually every combination of interconnects can be seen in this diagram from 10G/ 25G and 100G single DAC cables to dual-50G and quad-25G splitter cables that enable linking at any line speed to system servers and storage subsystems. All Mellanox switches, network adapters and transceivers are backwards compatible with 10G, 14G, and 25G line rates supporting quad channel versions of 40G, 56G and 100G.
Shadowing DAC cabling on the right side are AOC cables – also in 25G/40G/100G single AOCs as well as dual-50G and quad-25G splitter cables. While AOCs used in the rack are more expensive than DAC cabling, many system builders need to link subsystems residing is multiple racks to a common Top-of- Rack switch and spanning at reaches greater than 3-5 meters maximum of DAC cabling. Subsystems and storage bays are often as far as 10-30 meters away from the server complex along the row and adjacent rows. AOCs and splitters are a perfect, low-cost solution for these applications and less expensive than optical transceivers.
While not displayed in the diagram, SR and SR4 multi-mode transceivers and LR and PSM4 single- mode transceivers can also emulate the AOC splitters using dual MPO and 8-LC fiber splitter cables for applications that need to disconnect the fiber from the transceivers using optical connectors. Again, at a higher cost than AOCs but with the added benefit of being able to disconnect the fibers from the transceivers to rout in infrastructures such as structured cabling.
On the top are aqua-colored, 25G/100G multi-mode transceivers and AOCs. AOCs most often used for 10-30 meter reaches and connectorized SR and SR4 transceivers for structured cabling and reaches up to 100m. To the left are single-mode transceivers in 25G/100G in different types for reaches spanning 500m, 2km and 10km. The longer the reach in optics, the more complex and expensive the transceivers become so there are multiple design types (PSM4, CWDM4, LR4) to optimize for the lowest cost.
| page 35
eBook: New Developments in Data Center Cables and Transceivers
| page 37
eBook: New Developments in Data Center Cables and Transceivers
<< Back to TOC
The modern data center has focused on DAC and AOC cables with multi-mode and single-mode transceivers in 10G and 25G line rates, in single SFP and quad channel QSFP form-factors. Large and hyperscale data center builders are driving down prices and up the rate of change to unprecedented levels. The next phase of the industry starting in 2018 will bring many changes in line rate, modulation techniques, number of channels and new packaging schemes. The jump from 10G/40G to 25G/100G was fairly smooth as it had a minimum number of changes. However, the jump to 50G/200G and 100/200/400G entails very significant changes on nearly every aspect of switching, network adapters, DAC, AOC cables and optical transceivers and optical connectors.