Uncompromised Ethernet: Performance and Benchmarking for AI/ML Fabric

[ad_1] Right now, we’re exploring how Ethernet stacks up towards InfiniBand in AI/ML environments, specializing in how Cisco Silicon One™ manages community congestion and enhances efficiency for AI/ML workloads. This submit emphasizes the significance of benchmarking and KPI metrics in evaluating community options, showcasing the Cisco Zeus Cluster outfitted with 128 NVIDIA® H100 GPUs and cutting-edge congestion administration applied sciences like dynamic load balancing and packet spray.

Networking requirements to satisfy the wants of AI/ML workloads AI/ML coaching workloads generate repetitive micro-congestion, stressing community buffers considerably. The east-to-west GPU-to-GPU visitors throughout mannequin coaching calls for a low-latency, lossless community material. InfiniBand has been a dominant know-how within the high-performance computing (HPC) atmosphere and recently within the AI/ML atmosphere.

Ethernet is a mature various, with superior options that may tackle the rigorous calls for of the AI/ML coaching workloads and Cisco Silicon One can successfully execute load balancing and handle congestion. We got down to benchmark and examine Cisco Silicon One versus NVIDIA Spectrum-X™ and InfiniBand.

Analysis of community material options for AI/ML Community visitors patterns fluctuate based mostly on mannequin dimension, structure, and parallelization methods utilized in accelerated coaching. To guage AI/ML community material options, we recognized related benchmarks and key efficiency indicator (KPI) metrics for each AI/ML workload and infrastructure groups, as a result of they view efficiency by completely different lenses.

We established complete assessments to measure efficiency and generate metrics particular to AI/ML workload and infrastructure groups. For these assessments, we used the Zeus Cluster, that includes devoted backend and storage with an ordinary 3-stage leaf-spine Clos material community, constructed with Cisco Silicon One–based mostly platforms and 128 NVIDIA H100 GPUs. (See Determine 1.)

<img class="wp-image-471543 size-medium_large" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure3-768x294.png" alt="" width="768" height="294" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure3-768x294.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure3-300x115.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure3-1024x392.png 1024w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure3-1536x588.png 1536w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure3.png 2046w" sizes="(max-width: 768px) 100vw, 768px"/> Determine 1. Zeus Cluster topology We developed benchmarking suites utilizing open-source and industry-standard instruments contributed by NVIDIA and others. Our benchmarking suites included the next (see additionally Desk 1):

Distant Direct Reminiscence Entry (RDMA) benchmarks—constructed utilizing IBPerf utilities—to guage community efficiency throughout congestion created by incast NVIDIA Collective Communication Library (NCCL) benchmarks, which consider utility throughput throughout coaching and inference communication part amongst GPUs MLCommons MLPerf set of benchmarks, which evaluates essentially the most understood metrics, job completion time (JCT) and tokens per second by the workload groups

<img loading="lazy" class="wp-image-471535 size-medium_large" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Table_1-768x422.png" alt="" width="768" height="422" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Table_1-768x422.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Table_1-300x165.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Table_1-1024x563.png 1024w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Table_1-1536x844.png 1536w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Table_1.png 2000w" sizes="(max-width: 768px) 100vw, 768px"/> Desk 1. Benchmarking key efficiency indicator (KPI) metrics Legend:

JCT = Job Completion Time

Bus BW = Bus bandwidth

ECN/PFC = Specific Congestion Notification and Precedence Circulate Management

NCCL benchmarking towards congestion avoidance options Congestion builds up through the again propagation stage of the coaching course of, the place a gradient sync is required amongst all of the GPUs taking part in coaching. Because the mannequin dimension will increase, so does the gradient dimension and the variety of GPUs. This creates large micro-congestion within the community material. Determine 2 exhibits outcomes of the JCT and visitors distribution benchmarking. Word how Cisco Silicon One helps a set of superior options for congestion avoidance, corresponding to dynamic load balancing (DLB) and packet spray methods, and Knowledge Middle Quantized Congestion Notification (DCQCN) for congestion administration.

<img loading="lazy" class="size-medium_large wp-image-471536" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_2-768x506.png" alt="" width="768" height="506" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_2-768x506.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_2-300x198.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_2-1024x674.png 1024w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_2-1536x1011.png 1536w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_2.png 2000w" sizes="(max-width: 768px) 100vw, 768px"/> Determine 2. NCCL Benchmark – JCT and Visitors Distribution Determine 2 illustrates how the NCCL benchmarks stack up towards completely different congestion avoidance options. We examined the commonest collectives with a number of completely different message sizes to focus on these metrics. The outcomes present that JCT improves with DLB and packet spray for All-to-All, which causes essentially the most congestion because of the nature of communication. Though JCT is essentially the most understood metric from an utility’s perspective, JCT doesn’t present how successfully the community is utilized—one thing the infrastructure staff must know. This information may assist them to:

Enhance the community utilization to get higher JCT Know what number of workloads can share the community material with out adversely impacting JCT Plan for capability as use instances enhance

To gauge community material utilization, we calculated Jain’s Equity Index, the place LinkTxᵢ is the quantity of transmitted visitors on material hyperlink:

The index worth ranges from 0.0 to 1.0, with larger values being higher. A price of 1.0 represents the right distribution. The Visitors Distribution on Material Hyperlinks chart in Determine 2 exhibits how DLB and packet spray algorithms create a near-perfect Jain’s Equity Index, so visitors distribution throughout the community material is nearly excellent. ECMP makes use of static hashing, and relying on stream entropy, it could actually result in visitors polarization, inflicting micro-congestion and negatively affecting JCT.

Silicon One versus NVIDIA Spectrum-X and InfiniBand The NCCL Benchmark – Aggressive Evaluation (Determine 3) exhibits how Cisco Silicon One performs towards NVIDIA Spectrum-X and InfiniBand applied sciences. The information for NVIDIA was taken from the SemiAnalysis publication. Word that Cisco doesn't understand how these assessments have been carried out, however we do know that the cluster dimension and GPU to community material connectivity is just like the Cisco Zeus Cluster.

<img loading="lazy" class="size-medium_large wp-image-471537" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_3-768x506.png" alt="" width="768" height="506" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_3-768x506.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_3-300x198.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_3-1024x674.png 1024w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_3-1536x1011.png 1536w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_3.png 2000w" sizes="(max-width: 768px) 100vw, 768px"/> Determine 3. NCCL Benchmark – Aggressive Evaluation Bus Bandwidth (Bus BW) benchmarks the efficiency of collective communication by measuring the pace of operations involving a number of GPUs. Every collective has a particular mathematical equation reported throughout benchmarking. Determine 3 exhibits that Cisco Silicon One – All Cut back performs comparably to NVIDIA Spectrum-X and InfiniBand throughout varied message sizes.

Community material efficiency evaluation The IBPerf Benchmark compares RDMA efficiency towards ECMP, DLB, and packet spray, that are essential for assessing community material efficiency. Incast eventualities, the place a number of GPUs ship information to at least one GPU, usually trigger congestion. We simulated these circumstances utilizing IBPerf instruments.

<img loading="lazy" class="size-medium_large wp-image-471538" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_4-768x506.png" alt="" width="768" height="506" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_4-768x506.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_4-300x198.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_4-1024x674.png 1024w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_4-1536x1011.png 1536w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_4.png 2000w" sizes="(max-width: 768px) 100vw, 768px"/> Determine 4. IBPerf Benchmark – RDMA Efficiency Determine 4 exhibits how Aggregated Session Throughput and JCT reply to completely different congestion avoidance algorithms: ECMP, DLB, and packet spray. DLB and packet spray attain Hyperlink Bandwidth, bettering JCT. It additionally illustrates how DCQCN handles micro-congestions, with PFC and ECN ratios bettering with DLB and considerably dropping with packet spray. Though JCT improves barely from DLB to packet spray, the ECN ratio drops dramatically because of packet spray’s best visitors distribution.

Coaching and inference benchmark The MLPerf Benchmark – Coaching and Inference, printed by the MLCommons group, goals to allow honest comparability of AI/ML programs and options.

<img loading="lazy" class="size-medium_large wp-image-471539" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_5-768x710.png" alt="" width="768" height="710" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_5-768x710.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_5-300x277.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_5-1024x947.png 1024w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_5-1536x1420.png 1536w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/04/250408_Cisco-Secure-AI-Factory-NVIDIA_Figure_5.png 2000w" sizes="(max-width: 768px) 100vw, 768px"/> Determine 5. MLPerf Benchmark – Coaching and Inference We targeted on AI/ML information heart options by executing coaching and inference benchmarks. To realize optimum outcomes, we extensively tuned throughout compute, storage, and networking parts utilizing congestion administration options of Cisco Silicon One. Determine 5 exhibits comparable efficiency throughout varied platform distributors. Cisco Silicon One with Ethernet performs like different vendor options for Ethernet.

Conclusion Our deep dive into Ethernet and InfiniBand inside AI/ML environments highlights the exceptional prowess of Cisco Silicon One in tackling congestion and boosting efficiency. These modern developments showcase the unwavering dedication of Cisco to supply strong, high-performance networking options that meet the rigorous calls for of immediately’s AI/ML functions.

Many due to Vijay Tapaskar, Will Eatherton, and Kevin Wollenweber for his or her assist on this benchmarking course of.

Discover safe AI infrastructure Uncover safe, scalable, and high-performance AI infrastructure it is advisable develop, deploy, and handle AI workloads securely whenever you select Cisco Safe AI Manufacturing unit with NVIDIA.