30.1e nvidia-smi topo -m: printing the GPU interconnect topology matrix¶

🔍 Context Introduction¶

When you work with multiple GPUs in a single server — which is very common in AI training and inference — the way those GPUs are physically connected to each other matters a lot. Some GPUs can talk to each other very fast (like through NVLink), while others might have to go through slower paths (like PCIe switches or even the CPU). The command nvidia-smi topo -m prints a matrix that shows you exactly how every GPU is connected to every other GPU. This helps engineers understand which GPU pairs are best for high-speed communication, which is critical for distributed training performance.

⚙️ What Does the Topology Matrix Show?¶

The topology matrix is a grid where each row and column represents a GPU (and sometimes the CPU or NICs). Each cell in the matrix tells you the type of connection between two devices.

• NVLink — The fastest connection, direct GPU-to-GPU links (used in NVIDIA DGX systems and high-end cards like A100, H100).
• PIX — GPUs connected through the same PCIe switch (very good, but slower than NVLink).
• PXB — GPUs connected through multiple PCIe switches (slower than PIX).
• PHB — GPUs connected through the PCIe host bridge (even slower, goes through the CPU).
• NODE — GPUs in different NUMA nodes or sockets (slowest, requires going through the CPU and memory fabric).
• SYS — GPUs in different systems (only relevant for multi-node setups).

📊 How to Read the Matrix¶

Let's say you have 4 GPUs (GPU0, GPU1, GPU2, GPU3). The matrix will look something like this (conceptually):

• The diagonal (GPU0 to GPU0) is always X (same device).
• If GPU0 and GPU1 show NVLink, they can communicate at very high speed.
• If GPU0 and GPU2 show PIX, they share a PCIe switch — still fast.
• If GPU0 and GPU3 show PHB, they are on different CPU sockets — slower.

Engineers use this to decide which GPUs to pair for data parallelism or model parallelism.

📊 Visual Representation: nvidia-smi topo -m Connection Matrix¶

This diagram displays the connection speeds (NVLink vs. PCIe Gen5 vs. NUMA socket) between GPU slots.

🛠️ When to Use This Command¶

You should run nvidia-smi topo -m in these situations:

• Before starting distributed training — to verify that GPUs are connected via NVLink as expected.
• After hardware changes — like adding or replacing GPUs, to confirm the topology is correct.
• When debugging slow training — if training is slower than expected, the matrix may reveal unexpected slow paths between GPUs.
• When planning GPU assignments — to assign workloads to GPU pairs that have the fastest connections.

🕵️ Interpreting Common Patterns¶

✅ Practical Tips for New Engineers¶

• Always check the matrix before running multi-GPU jobs. You might assume all GPUs are equally connected, but the matrix will show you the reality.
• NVLink pairs are your best friends. If you have 8 GPUs with NVLink, they form a high-speed mesh. If only some pairs have NVLink, prioritize those pairs for communication-heavy tasks.
• Watch out for PHB connections. If two GPUs show PHB, they are on different CPU sockets. This adds latency and reduces bandwidth — try to avoid using them together for tight communication.
• The matrix also shows CPU and NIC connections. Some versions of the command include CPU affinity and network interface topology, which helps with NUMA-aware pinning.
• Use this alongside nvidia-smi (without flags) to check GPU health, memory usage, and processes — the topology matrix is just one piece of the diagnostic puzzle.

📌 Summary¶

nvidia-smi topo -m is your go-to tool for understanding the physical interconnect layout of your GPUs. It reveals which GPUs can talk to each other at high speed and which are stuck on slower paths. For any engineer working with multi-GPU AI infrastructure, reading this matrix is a fundamental skill — it directly impacts how you design and optimize distributed training workflows.