15.2a Three-tier (Access-Aggregation-Core): the traditional enterprise design and why it fails AI

📦 AI Data Center Networking 📖 The AI Traffic Problem

🧭 Context Introduction

Before diving into AI clusters, most data centers used a three-tier network design — also called Access-Aggregation-Core. This design was built for traditional enterprise workloads like email, web servers, and databases. However, when engineers try to run AI training workloads on this design, it breaks down badly.

This section explains what the three-tier design looks like, how it works, and why it is fundamentally unsuitable for AI infrastructure.

🏗️ What Is the Three-Tier Design?

The three-tier design organizes network switches into three layers:

  • Access Layer — Connects servers directly (e.g., top-of-rack switches).
  • Aggregation Layer — Also called the distribution layer; connects multiple access switches together.
  • Core Layer — The backbone that connects aggregation switches and provides connectivity to the outside world.

📊 How Traffic Flows in a Three-Tier Network

Traffic Type Path Example
North-South Server → Access → Aggregation → Core → External User requests a webpage
East-West Server → Access → Aggregation → Access → Server Two servers talking inside the same rack or across racks

In traditional enterprise setups, most traffic is north-south (users to servers). East-west traffic is minimal.

⚙️ Why This Design Fails for AI

AI training (especially distributed deep learning) creates a completely different traffic pattern. Here is what happens when you try to run AI on a three-tier network:

🕵️ Problem 1: East-West Traffic Explosion

  • AI training requires all GPUs to synchronize gradients after every training step.
  • This creates massive east-west traffic — every server talks to every other server.
  • In a three-tier design, this traffic must go up to the aggregation layer and back down, creating bottlenecks.

🕵️ Problem 2: Oversubscription at the Aggregation Layer

  • Access switches connect to aggregation switches with limited uplink bandwidth.
  • Example: A 48-port access switch with 4 x 25G uplinks creates a 12:1 oversubscription ratio.
  • AI workloads need 1:1 bandwidth between all servers — oversubscription kills performance.

🕵️ Problem 3: High Latency for Collective Operations

  • AI uses collective communication patterns like All-Reduce and All-to-All.
  • Every hop through an aggregation switch adds latency.
  • With hundreds or thousands of GPUs, this latency compounds and slows down training time dramatically.

🕵️ Problem 4: No Support for GPU-to-GPU Direct Communication

  • Modern AI clusters use NVLink or InfiniBand for GPU direct memory access (GDM).
  • Three-tier Ethernet designs do not support the lossless, low-jitter transport required for RDMA (Remote Direct Memory Access).

📊 Visual Representation: Traditional Core-Aggregation-Access 3-Tier Network

This diagram displays traditional 3-tier networking, organizing data paths hierarchically from core switches down to server cabinets.

flowchart LR Core["Core Layer (Routing)"] --- Aggr["Aggregation Layer (Filtering)"] Aggr --- Access["Access Layer (Server connections)"] classDef cpu fill:#eafaf1,stroke:#76b900,stroke-width:2px,rx:6px,ry:6px; classDef memory fill:#f0f7ff,stroke:#3498db,stroke-width:1.5px,rx:4px,ry:4px; classDef system fill:#f1f5f9,stroke:#64748b,stroke-width:1.5px; class Access cpu; class Aggr memory; class Core system;

🛠️ Comparison: Three-Tier vs. AI-Optimized Design

Feature Three-Tier (Enterprise) AI-Optimized (e.g., Fat-Tree, Dragonfly)
Primary traffic pattern North-South East-West
Bandwidth ratio Oversubscribed (e.g., 10:1) 1:1 non-blocking
Latency High (multiple hops) Low (direct paths)
Congestion handling TCP backoff RoCEv2 / InfiniBand flow control
Scalability Limited by core ports Linear scaling with leaf-spine
GPU communication Not designed for it Native support (NVLink, RDMA)

🔍 Real-World Example of Failure

Imagine you have 64 GPUs in 8 servers (8 GPUs per server) connected through a three-tier network:

  • Each server connects to an access switch at 100 Gbps.
  • The access switch connects to the aggregation switch with 4 x 100 Gbps uplinks (shared by 8 servers).
  • During an All-Reduce operation, all 8 servers try to send 10 GB of gradient data simultaneously.
  • The uplink becomes saturated — training stalls while waiting for data.

Result: GPU utilization drops from 95% to 30%. Training time increases by 3x or more.

✅ Key Takeaway for New Engineers

The three-tier design is like a highway system built for local traffic — it fails when every car needs to talk to every other car at the same time.

For AI infrastructure, you need flat, non-blocking topologies (like leaf-spine or fat-tree) that provide full bisection bandwidth and low, predictable latency between all GPUs.

📚 Summary

  • Three-tier = Access → Aggregation → Core
  • Built for north-south traffic (users to servers)
  • AI needs massive east-west traffic (GPU to GPU)
  • Oversubscription and latency kill AI training performance
  • Modern AI clusters use leaf-spine or direct-connect topologies instead

Next step: Learn about leaf-spine topology — the foundation of AI data center networking.