3.1d The /proc and /sys virtual filesystems: live kernel data in files¶

🧭 Context Introduction¶

When you work with AI infrastructure, you often need to understand what's happening inside the system at a very low level — how much memory is being used, which processes are running, how the CPU is performing, or what hardware devices are connected. In Linux, two special virtual filesystems — /proc and /sys — give you direct access to live kernel data, all presented as if they were ordinary files and directories.

Unlike regular files on your hard drive, these virtual filesystems exist only in memory. They are created by the kernel at boot time and updated in real time. Reading from them gives you a snapshot of the current system state. This makes them invaluable for monitoring, debugging, and understanding your AI workloads.

⚙️ What Are Virtual Filesystems?¶

• Virtual filesystems do not store data on a physical disk.
• They are created and managed entirely by the Linux kernel in memory.
• When you read a file in /proc or /sys, the kernel dynamically generates the content based on the current system state.
• When you write to certain files, you can change kernel parameters or hardware settings in real time.
• They provide a unified interface for accessing kernel data without needing special system calls or tools.

📂 The /proc Filesystem — Process and System Information¶

The /proc filesystem is primarily focused on process information and general system statistics. It was the first virtual filesystem introduced in Linux.

🧩 Key Characteristics¶

• Each running process has a subdirectory named after its Process ID (PID) — for example, /proc/1/ contains data about the init process.
• It provides system-wide information through top-level files like /proc/cpuinfo, /proc/meminfo, and /proc/uptime.
• Many files are read-only, but some (like /proc/sys/) can be written to for kernel tuning.

🕵️ What You Can Find in /proc¶

🛠️ Practical Use Cases for AI Infrastructure¶

• Memory monitoring: Read /proc/meminfo to check if your AI model training is exhausting system RAM.
• CPU analysis: Check /proc/cpuinfo to verify that CPU features like AVX-512 (important for some AI frameworks) are available.
• Process debugging: Look at /proc/[PID]/status to see if a GPU process is consuming too much memory.
• Load tracking: Use /proc/loadavg to understand overall system pressure during inference workloads.

🧰 The /sys Filesystem — Device and Kernel Objects¶

The /sys filesystem is a newer, more structured virtual filesystem introduced to expose kernel objects — particularly hardware devices, drivers, and kernel subsystems — in a hierarchical, organized way.

🧩 Key Characteristics¶

• It represents the kernel's device model as a tree of directories and files.
• Each physical or virtual device appears as a directory under /sys/devices/.
• It provides attribute files that let you read or write device properties.
• It is the primary interface for hardware discovery and driver configuration in modern Linux.

🕵️ What You Can Find in /sys¶

🛠️ Practical Use Cases for AI Infrastructure¶

• GPU discovery: Check /sys/class/drm/ to find NVIDIA GPU devices and their PCI addresses.
• Network tuning: Read /sys/class/net/eth0/speed to verify your network link speed for distributed training.
• Storage configuration: Examine /sys/block/nvme0n1/queue/scheduler to see or change the I/O scheduler for NVMe drives.
• Power management: Adjust /sys/power/state to control system sleep behavior during long training jobs.

🔄 Comparing /proc and /sys¶

📊 Visual Representation: Virtual Filesystem Hierarchy¶

This diagram maps how user space utilities query the /proc and /sys virtual filesystems, showing how the VFS layer dynamically retrieves state information directly from the kernel core and hardware drivers.

🧪 How to Read Data from /proc and /sys¶

Since these are virtual filesystems, you can use standard Linux tools to read them. The most common approach is to use a simple text-reading command.

For reference — reading a file in /proc:¶

cat /proc/meminfo

📤 Output: A block of text showing memory statistics like MemTotal, MemFree, Buffers, Cached, SwapTotal, and SwapFree.

For reference — reading a file in /sys:¶

cat /sys/class/net/eth0/speed

📤 Output: A single number representing the link speed in megabits per second (e.g., 10000 for 10 Gbps).

⚠️ Important Warnings for New Engineers¶

• Never write to files in /proc or /sys unless you fully understand the consequences — you can crash the system or damage hardware.
• These filesystems are dynamic — the data changes every time you read it. Two consecutive reads may give different results.
• Some files are very large — for example, /proc/kcore represents all of physical memory. Avoid reading it directly.
• Permissions matter — many files require root access to read or write.
• These are not real files — you cannot copy them to another system and expect them to work. They only make sense on the live kernel that created them.

✅ Summary¶

• /proc and /sys are virtual filesystems that expose live kernel data as files and directories.
• /proc focuses on processes, memory, CPU, and system statistics.
• /sys focuses on hardware devices, drivers, and kernel object hierarchy.
• Both are essential tools for monitoring and configuring AI infrastructure — from checking GPU availability to tuning network performance.
• Always read with caution and never write without understanding the impact.

These virtual filesystems are your window into the Linux kernel's real-time behavior, making them indispensable for any engineer working with AI infrastructure and operations.