Speaker
Description
Network protocols for AI/ML are a hot item right now. UEC (Ultra Ethernet Consortium) gets a lot of press, but it’s clearly designed for Scale Out. The landscape for an open community supported Scale Up protocol is much murkier. The difference between Scale Out and Scale Up is stark. The performance expectations of Scale Out are high, but for Scale UP they're much higher. Nvidia’s NVlink is the industry lead but it’s not open. UAlink is one consortium's answer to NVlink but it’s not Ethernet which is becoming an obvious liability. Broadcom's Scale Up Ethernet (SUE) is interesting, but far from a complete protocol and seems to lean towards being a proprietary protocol. In short there’s really no good option for an open Scale Up protocol.
Enter the Scale UP EtheRnet protocol, or SUPERp. SUPERp is an open networking protocol for Scale Up that runs over Ethernet. The primary purpose is to support Remote Memory Access (RMA) operations and other HPC operations among a set of tightly coupled systems in AI/ML Scale Up networks. For instance, several co-located racks of GPUs, or more generically XPUs or DSAs, may be connected to create a scalable cluster. SUPERp leverages the scalability and ubiquity of Ethernet while maintaining high performance for Scale UP.
A major goal of SUPERp is that it’s able to be implemented by a programmable device without sacrificing performance that could be attained by fixed function hardware. Programmability affords flexibility and allows the protocol to be changed or features added as needed. The protocol is purposely kept simple to be amenable to programmability and the programming model encourages the use of hardware accelerations to maintain high performance.
In this talk we introduce SUPERp. We’ll discuss the SUPERp implementation in XDP2 that can compile to a bunch of different backend targets including the likes of eBPF and custom hardware. We’ll then demo the PoC of SUPERp. Finally, we’ll discuss the plan and development path for SUPERp.