Cisco, Nvidia team up to build fatter, faster pluggable

AI datacenters are a rats nest of fiber optic cables stitching together thousands of AI accelerators, NICs, switches, and everything else required to train and run the latest and greatest modes. Cisco hopes to untangle this mess with a new pluggable optic developed in collaboration with Nvidia that will pack between four and eight times the bandwidth of existing pluggables into a single port. Announced at the Open Compute Project (OCP) global summit this week, the ultra-dense small form factor pluggable (USFP) promises to pack up to 32, 400 Gbps lanes into a pluggable roughly the size of existing OSFP modules. Each of these pluggables would boast 12.8 Tbps of Ethernet bandwidth, as much as a high-end enterprise switch just a few years ago. To put that in perspective, the most sophisticated pluggable optics today cap out at around 1.6 Tbps using eight 200 Gbps links. Such a pluggable has the potential to significantly simplify cabling across AI datacenters where it isn't uncommon for each GPU to be paired with an 800 Gbps and soon 1.6 Tbps NIC. A single USFP pluggable could consolidate eight GPUs worth of cabling into a single fiber optic bundle. Meanwhile, on the switch side, a future 204.8 Tbps switch would only require 16 USFP ports to support 128 1.6 Tbps endpoints. Cramming all that connectivity into a single pluggable won't be easy. Today a high-end pluggable optic might pull on the order of 20 watts. Cisco's USFP spec calls for up to 100 watts per pluggable. There are also design constraints that have to be taken into consideration. Four times as many links per pluggable means 4x as many electrical contacts that have to be perfectly seated. Because of this Cisco says USFP modules will employ a two phase insertion mechanism in which the pluggable is first inserted into the cage and then a second mechanism is actuated to make the electrical connection. Speaking of reliability, USFP would have a bigger blast radius than conventional pluggables. A single bad laser or digital signal processor (DSP) has the potential to take out far more than a single GPU, but entire systems. While that might sound bad, it's nothing compared to co-packed or near-package optics where one failure could drop an entire rack's worth of accelerators from the network. Nvidia's participation in the development of the USFP standard is notable as the Ethernet networking titan has largely moved away from pluggables in its most recent set of switches. Many of its Spectrum-X and Quantum-X switches now use co-packaged optics and for good reason. Cutting out the pluggables on the switch side nets a roughly 30 percent power savings. But while Nvidia's latest switches may use CPO, its NICs still rely on pluggables in part because it offers integrators greater flexibility. Not everyone is building 100,000-plus GPU clusters for training. For racks utilizing a top of rack switch, lower power direct attach copper cabling can be used without much issue. Given Nvidia's investments in CPO we can only speculate on its interests in USFP, but it likely comes down to densification. The AI infrastructure giant is rapidly moving toward 600 kW racks packing 144 GPUs on board. First teased in early 2025, Nvidia's Vera Rubin Ultra platform promised 115.2 TB/s scale out bandwidth per rack or about 3.2-6.4 Tbps of bandwidth per GPU package depending on whether they're counting bidirectional bandwidth or not. It just so happens that Cisco's initial USFP spec calls for 16, 400 Gbps links per pluggable, or about 6.4 Tbps of bandwidth. While Cisco and Nvidia may be leading the charge on the development of an ultra dense optical pluggable, like most things Ethernet, USFP will be an open standard anyone can use in their designs. Microsoft taps Cisco for multi-datacenter AI Alongside Cisco's latest tie up with Nvidia, Switchzilla also revealed that Microsoft is now using its Silicon One P200 routing chips to distribute workloads across multiple datacenters. Nvidia and others have taken to calling this kind of multi-site computing: scale across networking. We discussed Cisco's datacenter-scale routing platform at length late last year, but the silicon competes directly with products like Nvidia's Spectrum-XGS and Broadcom's Jericho 4-based offerings. Cisco touts the P200 as the first 51.2 Tbps systems "deployed in a production environment by leading hyperscalers." With enough qualifiers you can be the first at anything. As we reported at the time, Microsoft was among the first major hyperscalers to embrace scale-across networking as a means of overcoming local power constraints with its Fairwater clusters in Wisconsin and Georgia. And it seems we finally know the hardware good old Redmond is using to stitch them together. Cisco gives SONiC some love Speaking of hyperscalers, one thing you probably won't find in a hyperscale network is Cisco's Nexus network operating system (NOS). Hyperscalers like Microsoft typically employ vendor agnostic NOS platforms to avoid vendor lock-in and make managing networks across multiple Ethernet vendors cheaper and easier. Originally developed by Microsoft and later open sourced, SONiC is among the most popular and widely deployed of the hyperscale NOSes. SONiC support, we should note is nothing new for Cisco which contrary to popular belief is more than willing to sell you any one piece of its networking stack, assuming you've got a big enough purse - the hyperscalers certainly do. As such Cisco is one of the biggest contributors to the SONiC project. Instead of forcing customers to use its software stack, Cisco's approach to SONiC and the hyperscale is to extend the NOSes capabilities to better take advantage of ASIC's unique capabilities and allow for integration with its services. At OCP this week, Cisco announced support for its Live Protect platform in SONiC to make it easier for customers to keep up with security patches in the wild. This capacity is expected to roll out next year. (R)