Enfabrica 3.2 Tbps ACF SuperNIC Chip: Boosting AI Knowledge Heart Efficiency And Reliability For Large AI Cluster Scale

At Supercomputing 2024 (SC24), Enfabrica Company unveiled a milestone in AI information heart networking: the Accelerated Compute Cloth (ACF) SuperNIC chip. This 3.2 Terabit-per-second (Tbps) Community Interface Card (NIC) SoC redefines large-scale AI and machine studying (ML) operations by enabling large scalability, supporting clusters of over 500,000 GPUs. Enfabrica additionally raised $115 million in funding and is predicted to launch its (ACF) SuperNIC chip in Q1 2025.

Addressing AI Networking Challenges

As AI fashions develop more and more giant and complex, information facilities face mounting pressures to attach giant numbers of specialised processing models, equivalent to GPUs. These GPUs are essential for high-speed computation in coaching and inference however are sometimes left idle attributable to inefficient information motion throughout present community architectures. The problem lies in successfully interconnecting 1000’s of GPUs to make sure optimum information switch with out bottlenecks or efficiency degradation.

Conventional networking approaches can hyperlink roughly 100,000 AI computing chips in an information heart earlier than inefficiencies and slowdowns grow to be important. Based on Enfabrica’s CEO, Rochan Sankar, the corporate’s new expertise helps as much as 500,000 chips in a single AI/ML system, enabling bigger and extra dependable AI mannequin computations. By overcoming the constraints of standard NIC designs, Enfabrica’s ACF SuperNIC maximizes GPU utilization and minimizes downtime.

Key Improvements within the ACF SuperNIC

The ACF SuperNIC boasts a number of industry-first options tailor-made to trendy AI information heart wants:

1. Excessive-Bandwidth, Multi-Port Connectivity: The ACF SuperNIC delivers multi-port 800-Gigabit Ethernet to GPU servers, quadrupling the bandwidth in comparison with different GPU-attached NICs. This setup supplies unprecedented throughput and enhances multipath resiliency, guaranteeing strong communication throughout AI clusters.
2. Environment friendly Two-Tier Community Design: With a high-radix configuration of 32 community ports and as much as 160 PCIe lanes, the ACF SuperNIC simplifies the general structure of AI information facilities. This effectivity permits operators to assemble large clusters utilizing fewer tiers, lowering latency and bettering information switch effectivity throughout GPUs.
3. Scaling Up and Scaling Out: The Enfabrica ACF SuperNIC, with its high-radix, high-bandwidth, and concurrent PCIe/Ethernet multipathing and information mover capabilities, can uniquely scale up and scale out 4 to eight latest-generation GPUs per server system. This considerably will increase AI clusters’ efficiency, scale, and resiliency, guaranteeing optimum useful resource utilization and community effectivity.
4. Built-in PCIe Interface: The chip helps 128 to 160 PCIe lanes, delivering speeds over 5 Tbps. This design permits a number of GPUs to hook up with a single CPU whereas sustaining high-speed communication with information heart backbone switches. The result’s a extra environment friendly and versatile structure that helps large-scale AI workloads.
5. Resilient Message Multipathing (RMM): Enfabrica’s proprietary RMM expertise boosts the reliability of AI clusters. By mitigating the impression of community hyperlink failures or flaps, RMM prevents job stalls, guaranteeing smoother and extra environment friendly AI coaching processes. Sankar notes the significance of this characteristic, particularly in giant setups the place hyperlinks to switches failures grow to be frequent.
6. Software program-Outlined RDMA Networking: This distinctive characteristic empowers information heart operators with full-stack programmability and debuggability, bringing the advantages of software-defined networking (SDN) into Distant Direct Reminiscence Entry (RDMA) setups. It permits customization of the transport layer, which might optimize cloud-scale community topologies with out sacrificing efficiency.

Enhanced Resiliency and Effectivity

Conventional techniques usually require one-to-one connections between GPUs and numerous parts, equivalent to PCIe switches and RDMA NICs. Nevertheless, because the variety of GPUs in a system will increase, the chance of hyperlinks to switches failures grows, with potential disruptions occurring as usually as each 23 minutes in setups with over 100,000 GPUs, based on Shankar. 

The ACF SuperNIC addresses this concern by enabling a number of connections from GPUs to switches. This redundancy minimizes the impression of particular person element failures, boosting system uptime and reliability.

The SuperNIC additionally introduces the Collective Reminiscence Zoning characteristic, which helps zero-copy information transfers and optimizes host memory management. By lowering latency and enhancing reminiscence effectivity, this expertise maximizes the floating-point operations per second (FLOPs) utilization of GPU server fleets.

Scalability and Operational Advantages

The ACF SuperNIC’s design isn’t solely about scale but in addition about operational effectivity. It supplies a software program stack that integrates with customary communication, present interfaces, and RDMA networking operations. This compatibility ensures environment friendly deployment throughout various AI compute environments composed of GPUs and accelerators (AI chips) from completely different distributors. Knowledge heart operators profit from streamlined networking infrastructure, lowering complexity and enhancing the flexibleness of their AI information facilities.

Availability and Future Prospects

Enfabrica’s ACF SuperNIC shall be out there in restricted portions in Q1 2025, with each the chips and pilot techniques now open for orders by means of Enfabrica and chosen companions. As AI fashions demand greater efficiency and bigger scales, Enfabrica’s progressive method may play a pivotal function in shaping the subsequent technology of AI information facilities designed to help Frontier AI models.

Filed in Computers. Learn extra about AI (Artificial Intelligence), Chip, generative AI, Semiconductors, Server, SoC and Supercomputer.