|

Data Center Switches for Gen AI/ML

⏱️ Estimated reading time: Approx. 5 minutes (5 min read)

Artificial intelligence (AI) is being used across all industries and sectors, becoming a universal requirement. AI-based solutions are essential for analyzing continuously incoming data, gaining insights, and supporting real-time decision-making using task-specific AI models, such as machine learning and natural language processing (NLP).

인공지능 사용 사례

Infrastructure Requirements for GenAI (Generative AI)

Large-scale GenAI models require massive amounts of GPU-powered computing to support terabyte-sized training datasets containing billions of parameters accessed through ultra-low-latency memory and storage systems. The networks supporting these large-scale GenAI models are tailored to deliver optimized, power-efficient, and predictable performance across LLM multi-tenant workload environments.

The fabric and topology are highly tuned to support the entire system, with all aspects of the network including compute, data processing units (DPUs), I/O, cabling and optics, acceleration software, and networking (e.g., InfiniBand 400/800G Ethernet switches).

Preparing Data Center Networks for AI

AI architectures require a dedicated network fabric that combines high-performance and low-latency connectivity to ensure the fastest training, inference, and modeling task completion times. Early HPC and AI training networks initially required high-speed, low-latency, and dedicated connectivity for fast and efficient communication between servers and storage systems. InfiniBand The network gained popularity.

Today, 100/200/400G+ Leaf/Spine Ethernet switching is gaining significant traction in supporting networking for HPC/AI clusters by providing an open, standards-based alternative, and is expected to become the most cost-effective and popular alternative for many AI use cases.

AI 네트워킹 시장

“As AI bandwidth increases, the share of Ethernet switching connected to AI/ML and accelerated computing will shift to a significant portion of the market by 2027. As AI/ML-enabled products reach production scale, shipments of 800Gbps-based switches and optical modules will reach record levels.”

Alan Weckel, founder and technology analyst at 650 Group

Modern AI applications require high-bandwidth, lossless, low-latency, and scalable multi-tenant networks interconnecting hundreds or thousands of GPUs at speeds ranging from 100G to 400G and beyond. An Ethernet-based networking fabric delivers the reliability and performance required for AI workload clusters with hundreds or thousands of GPUs.

Backend vs. Frontend Differences in AI Fabric Networks

Gen AI/ML를 위한 데이터센터 스위치 - image 30
  • Frontend network: It is built using existing Ethernet networks and is built as a lossless environment to support shared storage.

  • GenAI Fabric: Workloads where the GPU moves workload traffic using both the internal PCIe bus and the backend fabric.

  • Backend network: Used to interconnect AI servers and workloads, and may also include dedicated storage. Built with no link oversubscription between workloads and the fabric, and optimized for GPU workloads using low latency, DCB, RoCE, etc.

GenAI Backend Fabric Network Design Method
Multi-layer Clos
  • ToR switches connect each server and provide connectivity to other racks through aggregation switches.
  • Spine switches provide connections to other PODs
  • Best suited for CPU-intensive workloads
Rail Optimized
  • Built on a GPU-centric cluster with two different communication paths to each GPU.
  • One path is NVIDIA® NVSwitch1Through, other routes connect via rail switches.
  • NVIDIA NVSwitch on individual servers creates high-speed interconnects to form high-bandwidth (HB) domains.
  • These rail switches are connected to the Spine switches to form a full-section, any-to-any Clos network topology.
Rail Only
  • Similar to Rail Optimized
  • Instead, remove network connections between GPUs of different ranks within the Rail.
  • HB domain2Communication is still possible by passing data through
Gen AI/ML를 위한 데이터센터 스위치 - image 31
Rail-only architecture

AI-enabled data center switches from HPE Aruba Networking

HPE Aruba Networking can help you design and build a dedicated AI network fabric.

HPE Aruba CX 8325 Switch Series
Gen AI/ML를 위한 데이터센터 스위치 - image 32

The CX 8325 switch series is an entry-level GenAI switch solution that supports 1/10/25/100GbE ports.
It supports rail-only architecture and is suitable for general data center environments as a 1U switch with up to 6.4Tbps.

HPE Aruba CX 9300 Switch Series

CX 9300 Switch Seriesis a next-generation 25.6Tbps, 1U switch that supports 32 100/200/400GbE ports.

HPE Aruba Networking CX9300-32D

The CX 9300 offers AI/HPC-optimized features, including low latency, lossless network quality of service (QoS), ROCEv2, ECN, and PFC, all of which are essential for AI/HPC connectivity. It also supports both rail-only and rail-optimized architectures, and the CX 9300 switch can be used for either Spine or Rail purposes.

Gen AI/ML를 위한 데이터센터 스위치 - image 15

For more information, please see the link below.

  1. Supports high bandwidth but short-distance interconnection ↩︎
  2. Subset of GPU ↩︎

Similar Posts

  • A New Definition of SDN: Self-Driving Network

    ⏱️ Estimated reading time: Approx. 5 minutes (5 min read) The biggest topic that has recently shaken the automotive industry is undoubtedly Tesla's FSD (Full Self-Driving). This technology, which minimizes driver intervention and allows the system to make decisions on its own to safely reach a destination, has imprinted upon us the convenience and efficiency that 'autonomous driving' brings. However, this wave of change is now...

  • Prepare for the Golden Week with HPE Instant On

    ⏱️ Estimated reading time: Approx. 3 minutes (3 min read) The Golden Week holidays are fast approaching. While this is an exciting time for the hospitality industry, such as hotels, it also presents its own challenges. As guests flock to enjoy the holidays, the demand for stable and fast Wi-Fi also surges. This is especially true for small hotels with limited resources, where providing seamless and inconvenience-free… for all guests

  • New security as data center traffic patterns change

    East-west traffic has recently become increasingly important within data centers, driven by the proliferation of virtualization and cloud computing. This shift introduces security vulnerabilities and operational inefficiencies, necessitating microsegmentation and internal traffic control to ensure data center security.

  • Why did HPE choose Juniper, even though it already has Central AI?

    ⏱️ Estimated reading time: Approx. 3 minutes (3 min read) When news of the merger between HPE and Juniper Networks broke, the biggest question in the industry was this: “Why spend a massive sum of $14 billion (approx. 19 trillion won) to acquire Juniper when the industry’s second-largest player, Aruba Central, already exists?” This is not merely an acquisition to increase size. This involves a network paradigm…

  • HPE Aruba Ethernet-based storage fabric

    ⏱️ Estimated read time: Approx. 4 minutes (4 min read) Deploying next-generation data centers and private clouds requires 100/200/400GbE fabrics to support these new workloads and data-centric applications. Recent storage architectures are rapidly evolving to improve performance and provide cloud-like scalability and efficiency to data centers. Disaggregated Storage is…

  • Core Features of Zero Trust Network Security

    ⏱️ Estimated reading time: Approx. 6 minutes (6 min read) October is Cyber Security Awareness Month, focusing on protecting individuals and organizations from cybercrime. One way for organizations to address cyber security issues is to adopt “Zero Trust Network Security.” However, if a Zero Trust Network Security strategy is fully implemented, the problem is…