Ethernet Instead of InfiniBand: HPE Networking Supports OCI's Mega-sized AI Cluster

On September 2, 2026, HPE and Oracle officially announced a significant expansion of their strategic cooperation to expand gigawatt (GW)-scale AI infrastructure.I did.

Across Oracle Cloud Infrastructure (OCI)'s next-generation AI supercluster HPE Networking The core idea is to fully deploy the solution on a global scale over several years. In particular, the bond between the two companies takes the form of a strategic alliance that goes beyond a simple vendor-customer relationship, to the extent that HPE has granted Oracle a warrant to purchase its common stock.

From an engineering and business perspective, this announcement carries exceptional weight.

Breaking the barrier of InfiniBand, which was considered the exclusive property of AI clusters, the fact that “hyperscalers” massive AI fabrics are also perfectly converging to Ethernet,” and HPE has demonstrated unrivaled capabilities to complete AI Data Center (AIDC) networking as a 'full-stack' solution from the front-end to the back-end.is the point.

AI Fabric at a Crossroads: Why Ethernet (RoCEv2) over InfiniBand?

In distributed learning environments for AI workloads, particularly for very large language models (LLMs), networks are no longer merely simple data transmission channels.
It is a core component that determines computing power itself.

In the distributed learning process, tens of thousands of GPUs are inevitably All-ReduceCollective communication such as this is constantly performed. The most critical problems at this time are tail latency and synchronization barriers.

If a delay caused by packet drops or microbursts occurs on a single link, tens of thousands of other GPUs will stop computations and idle while waiting for a single flow.
This directly leads to a sharp decline in GPU Model FLOPs Utilization (MFU) and a surge in production costs per token.

Initially, InfiniBand, which guaranteed lossless and ultra-low latency, became the de facto standard, but as infrastructure expanded to gigawatt (GW) scale, it encountered clear limitations.

• Vendor Lock-in Supply chain risks and distorted TCO structure of a closed ecosystem relying on a single supplier
• Physical Limitations of Scale-Out: Fabric complexity and cost burden when bundling hundreds of thousands or more nodes
• Disconnection from Scale-Across: Connecting regions and distributed hubs beyond the interior of the data center Operational inefficiency caused by disconnection from existing IP/Ethernet infrastructure in a wide-area AI Grid environment

The solution OCI chose for this massive buildout is High-performance Ethernet fabric based on RoCEv2 (RDMA over Converged Ethernet v2)It was. As Ultra Ethernet (UET) aims for It has secured a robust supply chain for a standard open ecosystem while enabling a large-scale implementation of an InfiniBand-level lossless, ultra-low latency environment through advanced congestion control technology.

The 'Full-Stack' Architecture of AIDC Fabric Presented by HPE

The strongest message this order sends to the market is that HPE has proven it possesses a “full-stack portfolio capable of completing everything from the deepest GPU backend fabric of AIDC to the outermost WAN/edge routing with a single architecture.”.

While many existing solutions were limited to spine-to-leaf switching or restricted to specific segments, the HPE networking architecture deployed in OCI penetrates all boundaries of the data center.

Architecture Domain	Applicable Platforms	Engineering Role and Key Mechanisms
AI Backend Fabric (GPU Cluster)	QFX Series	• Provides high-density 400G/800G port integration • RoCEv2-based lossless transmission optimization (PFC, ECN) • Dynamic Load Balancing: Prevents flow-unit skew and performs packet spraying/load balancing
Regional DC Fabric (Spine/Leaf)	QFX & EX Series	• Connecting internal computing and storage fabrics within regional data centers • Minimizing operational complexity through a consistent enterprise operating system
Edge & Inter-DC (Core Routing)	PTX & MX Series	• Terabit-class backbone routing connecting massive AI clusters, global regions, and customer networks • High-reliability core traffic processing supporting gigawatt-class infrastructure
Telemetry & Observability (Intelligence)	Intelligent Telemetry	• Preemptive detection of queue buildup and packet loss in microseconds (µs) • Real-time tracking of traffic imbalance and optical transceiver/cable hardware degradation

The Core of Backend Fabric: QFX-based Dynamic Traffic Engineering

The latest QFX series deployed in the backend has not simply expanded bandwidth.

To mitigate irregular traffic patterns (such as incast) of large-scale AI workloads at the silicon ASIC level Sophisticated ECN marking과 PFC Deadlock Prevention, ...and performs hardware-based dynamic load balancing (DLB). It is a design that suppresses hash bias caused by Elephant Flow concentrating on specific links and maximizes bandwidth utilization across the entire fabric.

Weapon to Prevent GPU Idling: Deep Fabric Telemetry

What deserves more attention than the hardware specifications is the “visibility collaboration based on Intelligent Telemetry” emphasized by both companies.

In a cluster where tens of thousands of GPUs operate in tandem, a single physical cable failure, minute signal degradation in optical transceivers, or intermittent CRC error becomes a "silent killer" that halts an entire AI training job or delays it by hours. From the perspective of token economics, this represents a fatal waste of resources.

OCI and HPE achieve the following by streaming telemetry of traffic behavior occurring at the device and fabric levels in microseconds (µs):

1. Queue Buildup Early Warning: Detect signs of queue congestion and bypass traffic before the buffer overflows and packet drops occur.
2. Proactive Remediation: Preemptively isolate (Drain) links with degraded signal quality without interrupting GPU training sessions
3. GPU MFU Optimization: Maximize infrastructure TCO by fundamentally eliminating latency caused by network bottlenecks

“The Triumph of Ethernet, and the Completed Power of HPE Networking”

This announcement clearly shows where the landscape of the AI infrastructure market is headed.

first, The 'Ethernet dominance' of AI backend networks is no longer a futuristic hypothesis but an ongoing standard.no see.

The massive trend that began with the inception of the Ultra Ethernet Consortium (UEC) has been fully proven by the large-scale adoption of OCI, the most aggressive AI cloud builder.

Second, Proof of HPE Networking's Full Stack Powerno see.

Companies with a single vendor portfolio covering the entire spectrum from edge to core and AI backend are extremely rare in the market.
Through this collaboration, HPE has established an unrivaled position as a 'proven AIDC full-stack partner' that meets the most demanding requirements of hyperscalers.

In the era of gigawatt-class AI infrastructure, networks are no longer merely an auxiliary means of connecting compute resources, but a core engine directly linked to the yield of AI factories. At the heart of this, Ethernet and HPE Networking's full-stack fabric technology are setting new industry standards.