5.1b vmstat: virtual memory, swap, and I/O statistics per second¶

🧠 Context Introduction¶

When managing AI infrastructure, understanding how your system's memory, swap space, and I/O are behaving is critical. AI workloads — especially training large models or processing massive datasets — can quickly consume memory and cause the system to start swapping data between RAM and disk. This slows everything down.

vmstat is a lightweight, built-in Linux tool that reports real-time statistics about processes, memory, paging, block I/O, traps, and CPU activity. For new engineers, it's one of the first commands you should learn to quickly diagnose performance bottlenecks.

⚙️ What vmstat Reports¶

vmstat gives you a snapshot of system health, broken into key areas:

• Procs (processes) — how many processes are running or waiting
• Memory — how much RAM is used, free, buffered, or cached
• Swap — how much swap space is in use and how much is being moved in/out
• I/O — blocks read from and written to disk per second
• System — interrupts and context switches per second
• CPU — percentages of user, system, idle, and wait time

📊 Key Fields Explained for AI Workloads¶

When you run vmstat, you'll see columns like these. Here's what each means for an AI operator:

🕵️ How to Interpret vmstat for AI Infrastructure¶

🟢 Healthy AI Server¶

• r is close to number of CPU cores
• free memory is stable (not dropping rapidly)
• si and so are near zero
• wa is low (under 5-10%)
• bi/bo are steady, not spiking

🔴 Warning Signs¶

• si or so consistently above 0 → system is swapping → performance will tank
• wa above 20% → storage subsystem cannot keep up with I/O demands
• r much higher than CPU cores → processes are fighting for CPU time
• free memory dropping while cache stays flat → memory leak or insufficient RAM

📊 Visual Representation: vmstat Resource Monitoring Points¶

This diagram illustrates the flow of memory demands, swapping activity (si/so), and block I/O (bi/bo) monitored by the vmstat utility.

🛠️ Practical Usage Pattern for AI Engineers¶

To monitor your system in real time, you typically run vmstat with a delay interval. For example, to see statistics every 2 seconds:

Command format:
vmstat 2

This will print a new line every 2 seconds, showing current system state. The first line is an average since boot — ignore it. Watch the second and subsequent lines.

📤 Output:
A table of columns (procs, memory, swap, io, system, cpu) updating every 2 seconds.

📈 What to Watch During AI Training¶

During model training, pay special attention to:

• Swap columns (si, so) — If these are non-zero, your model is using more memory than available RAM. Consider reducing batch size or adding more RAM.
• I/O columns (bi, bo) — High values during data loading phases are normal, but sustained high I/O during training means data pipeline is a bottleneck.
• CPU wait (wa) — If this is high, your storage (NVMe, SSD, or network storage) cannot feed data fast enough to the GPU.

🔍 Quick Troubleshooting Flow¶

1. Run vmstat 2 and watch for 10-15 seconds
2. If si or so > 0 → memory pressure → check RAM usage, reduce workload, or add swap
3. If wa > 20% → I/O bottleneck → check disk speed, data pipeline, or storage configuration
4. If r > CPU cores × 2 → CPU saturation → consider more CPU resources or optimizing code
5. If free memory is very low but cache is high → this is normal (Linux uses free RAM for cache)

✅ Summary for New Engineers¶

• vmstat is your first diagnostic tool for memory, swap, and I/O issues
• Focus on si/so for swap problems, wa for I/O problems, and r for CPU problems
• AI workloads are memory and I/O hungry — vmstat helps you catch problems before they crash your training job
• Always run with a delay (like vmstat 2) to see real-time behavior, not just averages

Remember: If you see swapping (si/so > 0), your AI workload is already suffering. Fix memory issues before they become training failures.

Next in this section: 5.1c iostat — detailed I/O statistics for storage devices