12.1d Hopper variants: H100 SXM5, H100 PCIe, H800 (export-limited), H200 (HBM3e upgrade)

📦 Nvidia GPU Architecture 📖 NVIDIA GPU Generations & Form Factors

🧭 Context Introduction

The NVIDIA Hopper architecture (introduced in 2022) represents a major leap in AI and high-performance computing (HPC) capability. It powers the most demanding workloads, from training large language models (LLMs) to scientific simulations. However, not all Hopper GPUs are the same. This topic breaks down the four key variants you will encounter: H100 SXM5, H100 PCIe, H800, and H200. Understanding their differences is critical for selecting the right hardware for your AI infrastructure.

⚙️ The Hopper Architecture Family

Hopper GPUs share a common core architecture (e.g., Transformer Engine, NVLink Switch, DPX instructions), but they differ in form factor, memory capacity, memory bandwidth, and regulatory restrictions.

📊 Variant Overview Table

Variant Form Factor Memory Memory Bandwidth Key Feature Availability
H100 SXM5 SXM5 (module) 80 GB HBM2e 3.35 TB/s Full NVLink, highest performance Global
H100 PCIe PCIe 5.0 (card) 80 GB HBM2e 2.0 TB/s Standard server compatibility Global
H800 SXM5 or PCIe 80 GB HBM2e 3.35 TB/s (SXM) Export-limited variant for China China only
H200 SXM5 or PCIe 141 GB HBM3e 4.8 TB/s HBM3e memory upgrade Global (late 2023+)

📊 Visual Representation: NVIDIA H100 SXM vs. PCIe Architectural Differences

This diagram contrasts the power and bandwidth characteristics of the SXM and PCIe form factors of the Hopper H100 GPU.

flowchart LR SXM["H100 SXM5 Form Factor"] -->|Power: 700W| SXM_BW["NVLink Bandwidth: 900 GB/s"] PCIe["H100 PCIe Form Factor"] -->|Power: 350W| PCIe_BW["NVLink Bandwidth: 600 GB/s"] classDef cpu fill:#eafaf1,stroke:#76b900,stroke-width:2px,rx:6px,ry:6px; classDef memory fill:#f0f7ff,stroke:#3498db,stroke-width:1.5px,rx:4px,ry:4px; classDef system fill:#f1f5f9,stroke:#64748b,stroke-width:1.5px; class SXM,PCIe cpu; class SXM_BW,PCIe_BW memory;

🕵️ Detailed Breakdown of Each Variant

1️⃣ H100 SXM5 — The Flagship

  • Form Factor: SXM5 module (plugs directly into an NVIDIA HGX baseboard).
  • Memory: 80 GB HBM2e (High Bandwidth Memory, 2nd generation enhanced).
  • Memory Bandwidth: 3.35 TB/s — extremely fast for large model training.
  • Connectivity: Full NVLink 4.0 (900 GB/s per GPU) for multi-GPU scaling.
  • Use Case: Ideal for large-scale AI training clusters (e.g., 8-GPU HGX systems).
  • Power: 700W (requires liquid cooling or high-performance air cooling).

2️⃣ H100 PCIe — The Flexible Workhorse

  • Form Factor: Standard PCIe 5.0 add-in card (fits in most servers).
  • Memory: 80 GB HBM2e (same capacity as SXM5).
  • Memory Bandwidth: 2.0 TB/s (lower than SXM5 due to PCIe bus limits).
  • Connectivity: PCIe 5.0 (128 GB/s per GPU) — no NVLink support.
  • Use Case: Inference, smaller training jobs, or single-GPU workstations.
  • Power: 350W (air-cooled, easier to deploy).
  • Note: Engineers often choose PCIe when they need flexibility or already have PCIe-based servers.

3️⃣ H800 — The Export-Limited Variant

  • Background: Created to comply with U.S. export regulations for China.
  • Key Difference: Reduced NVLink bandwidth (400 GB/s vs. 900 GB/s on H100 SXM5).
  • Form Factor: Available as SXM5 or PCIe.
  • Memory: 80 GB HBM2e (same as H100).
  • Memory Bandwidth: 3.35 TB/s (SXM) or 2.0 TB/s (PCIe).
  • Use Case: AI workloads in China, but with limited multi-GPU scaling.
  • Important: You will only encounter this variant if you work with partners or data centers in China.

4️⃣ H200 — The Memory Upgrade

  • Form Factor: SXM5 or PCIe (same physical design as H100).
  • Memory: 141 GB HBM3e (3rd generation enhanced HBM).
  • Memory Bandwidth: 4.8 TB/s — 40% more bandwidth than H100.
  • Key Benefit: Can fit larger models (e.g., 70B+ parameter LLMs) in a single GPU.
  • Use Case: Inference for very large models, memory-bound HPC workloads.
  • Power: 700W (SXM) or 350W (PCIe) — same as H100.
  • Note: The H200 is a drop-in replacement for H100 in existing HGX systems.

🛠️ Practical Guidance for Engineers

When to Choose Which Variant

  • Choose H100 SXM5 if you are building a new, high-performance AI cluster and need maximum multi-GPU scaling.
  • Choose H100 PCIe if you are upgrading an existing server or need a single-GPU solution for inference.
  • Choose H800 only if you are deploying in China and must comply with export regulations.
  • Choose H200 if your workloads are memory-bound (e.g., large LLM inference) and you can afford the premium.

Key Considerations

  • NVLink vs. PCIe: For multi-GPU training, NVLink (SXM) is essential. For single-GPU inference, PCIe is sufficient.
  • Cooling: SXM modules often require liquid cooling in dense configurations. PCIe cards are easier to air-cool.
  • Power Budget: A single H100 SXM5 draws 700W. Plan your power distribution unit (PDU) and cooling accordingly.
  • Software Compatibility: All Hopper variants support CUDA 11.8+ and the same software stack (e.g., NVIDIA AI Enterprise, TensorRT).

🧪 Quick Reference: Command-Line Check (For Reference)

To identify which Hopper variant is installed in a system, use the NVIDIA System Management Interface (nvidia-smi).

For reference: 

nvidia-smi --query-gpu=name,memory.total,memory.bandwidth --format=csv

📤 Output:
H100 SXM5, 81920 MiB, 3350 GB/s
H100 PCIe, 81920 MiB, 2000 GB/s
H800, 81920 MiB, 3350 GB/s
H200, 144000 MiB, 4800 GB/s

✅ Summary

  • H100 SXM5 is the standard for high-performance AI clusters.
  • H100 PCIe offers flexibility and easier deployment.
  • H800 is a restricted variant for China with reduced NVLink.
  • H200 brings a major memory and bandwidth upgrade for large models.

Understanding these variants helps you make informed decisions about hardware procurement, workload placement, and system design. As you gain experience, you will learn to match the right GPU to the right task.