13.3b NVSwitch generations: 1.0 (Volta), 2.0 (Ampere), 3.0 (Hopper), 4.0 (Blackwell)¶

🧠 Context Introduction¶

When training large AI models, a single GPU is rarely enough. Engineers need to connect many GPUs together so they can share data quickly. NVSwitch is Nvidia's dedicated hardware solution that creates a high-speed, all-to-all connection fabric between GPUs inside a single server (or across multiple servers). Think of it as a super-fast, non-blocking switch that lets every GPU talk to every other GPU simultaneously without slowing down.

Each new generation of NVSwitch brings higher bandwidth, lower latency, and better scalability. This guide breaks down the four major generations: Volta (1.0), Ampere (2.0), Hopper (3.0), and Blackwell (4.0).

⚙️ Generation 1.0 — NVSwitch (Volta Architecture)¶

Release Year: 2017
Target GPU: V100

• Purpose: First dedicated NVSwitch silicon, designed to connect up to 16 V100 GPUs in a single node (DGX-1 and DGX-2 systems).
• Bandwidth per GPU: 300 GB/s (bidirectional) — each GPU had 6 NVLink 2.0 links.
• Topology: All-to-all, fully connected mesh within the switch.
• Key Limitation: Only supported intra-node (single server) connectivity. Could not connect across multiple nodes without additional networking.

🕵️ Why it mattered: Before NVSwitch, engineers had to rely on CPU-based PCIe switches which were much slower. NVSwitch 1.0 was the first step toward truly scalable multi-GPU training.

📊 Generation 2.0 — NVSwitch (Ampere Architecture)¶

Release Year: 2020
Target GPU: A100

• Purpose: Significantly improved bandwidth and introduced NVLink 3.0.
• Bandwidth per GPU: 600 GB/s (bidirectional) — each GPU had 12 NVLink 3.0 links.
• Topology: Still all-to-all, but now supported up to 8 NVSwitches per node (DGX A100).
• New Feature: NVSwitch 2.0 enabled GPU-to-GPU communication across multiple nodes using NVLink bridges, creating a single logical GPU cluster.
• Scalability: Up to 256 A100 GPUs in a single NVSwitch fabric (DGX SuperPOD).

🛠️ Key improvement for engineers: Doubled bandwidth per GPU compared to Volta, making it much easier to train large models like GPT-3.

🚀 Generation 3.0 — NVSwitch (Hopper Architecture)¶

Release Year: 2022
Target GPU: H100

• Purpose: Introduced NVLink 4.0 with even higher bandwidth and new NVLink Switch System for multi-node scaling.
• Bandwidth per GPU: 900 GB/s (bidirectional) — each GPU had 18 NVLink 4.0 links.
• Topology: Third-generation NVSwitch with support for up to 256 H100 GPUs in a single NVLink domain (DGX H100).
• New Feature: NVLink Switch System — a dedicated rack of NVSwitch chips that connects up to 576 H100 GPUs across multiple servers (DGX SuperPOD).
• Latency: Reduced by 50% compared to Ampere generation.

🕵️ Why it's important: The NVLink Switch System allowed engineers to build massive GPU clusters without traditional InfiniBand or Ethernet for GPU-to-GPU traffic. This dramatically simplified network topology.

📊 Visual Representation: NVSwitch ASIC Generational Progression¶

This flowchart tracks the routing bandwidth and port density of NVSwitch ASICs from NVSwitch 1 up to NVSwitch 4.

🔮 Generation 4.0 — NVSwitch (Blackwell Architecture)¶

Release Year: 2024
Target GPU: B200 / B100

• Purpose: Next-generation NVSwitch with NVLink 5.0 and co-packaged optics for even higher bandwidth.
• Bandwidth per GPU: 1.8 TB/s (bidirectional) — each GPU has 18 NVLink 5.0 links (doubled per-link speed).
• Topology: Fourth-generation NVSwitch supports up to 576 GPUs in a single NVLink domain (DGX B200).
• New Feature: NVLink Switch 4.0 uses co-packaged optics — fiber optic cables directly integrated into the switch silicon, reducing power and increasing density.
• Scalability: Up to 576 B200 GPUs in a single NVSwitch fabric, with NVLink bandwidth matching GPU memory bandwidth for the first time.

🛠️ Key improvement for engineers: With 1.8 TB/s per GPU, engineers can train models with trillions of parameters (e.g., GPT-4 scale) without GPU-to-GPU communication becoming a bottleneck.

📊 Comparison Table: NVSwitch Generations¶

🧩 How NVSwitch Generations Impact Engineers¶

• Model Size: Each generation allows training larger models without communication bottlenecks. For example, a 1 trillion parameter model requires ~2 TB of GPU memory. With NVSwitch 4.0, 576 GPUs can share data at 1.8 TB/s each — fast enough to keep all GPUs busy.
• Network Simplification: With NVSwitch 3.0 and 4.0, engineers can reduce or eliminate the need for InfiniBand for GPU-to-GPU traffic inside a cluster. This lowers cost and complexity.
• Job Scheduling: Higher bandwidth means shorter training times. Engineers can schedule more jobs per day, improving cluster utilization.

✅ Summary¶

• NVSwitch 1.0 (Volta) — First step, limited to 16 GPUs per node.
• NVSwitch 2.0 (Ampere) — Doubled bandwidth, enabled multi-node clusters up to 256 GPUs.
• NVSwitch 3.0 (Hopper) — Introduced NVLink Switch System, 900 GB/s per GPU, up to 576 GPUs.
• NVSwitch 4.0 (Blackwell) — 1.8 TB/s per GPU, co-packaged optics, bandwidth matches GPU memory speed.

Each generation makes it easier for engineers to scale AI training from a single GPU to thousands of GPUs working together as one giant virtual GPU.