33.2a vGPU architecture: host driver intercepts GPU commands from guest VMs¶

📖 Context Introduction¶

When you run AI workloads inside virtual machines (VMs), the guest operating system doesn't have direct access to the physical GPU hardware. Instead, NVIDIA vGPU technology creates a shared architecture where a host driver sits between the guest VMs and the physical GPU. This host driver acts as a traffic controller — it intercepts every GPU command coming from the guest VMs, validates it, and forwards it to the physical GPU. This interception is the core mechanism that enables multiple VMs to safely and efficiently share a single physical GPU.

⚙️ How the Interception Works¶

The interception process follows a clear, step-by-step flow:

• Step 1 — Guest VM issues a GPU command: The AI application inside the VM (e.g., a training script using PyTorch or TensorFlow) sends a GPU command through the guest vGPU driver.
• Step 2 — Command reaches the hypervisor layer: The guest driver forwards the command to the hypervisor (such as VMware vSphere or KVM), which passes it to the host-level vGPU manager.
• Step 3 — Host driver intercepts the command: The NVIDIA host driver (part of the vGPU software stack) catches the command before it reaches the physical GPU.
• Step 4 — Host driver validates and translates: The driver checks that the command is valid, applies any necessary memory address translations, and ensures the VM is only accessing its allocated portion of GPU resources.
• Step 5 — Command is forwarded to physical GPU: After validation, the host driver sends the command to the actual GPU hardware for execution.
• Step 6 — Results are returned to the guest VM: The GPU processes the command, and the host driver routes the results back to the requesting VM.

🕵️ Why Interception is Critical¶

The host driver interception provides three essential benefits for AI workloads in virtualized environments:

• 🔒 Security isolation: Prevents one VM from reading or modifying another VM's GPU memory or computations.
• 📊 Resource fairness: Ensures each VM gets its guaranteed share of GPU compute and memory, preventing a single VM from starving others.
• 🛡️ Error containment: If a guest VM sends a malformed or dangerous command, the host driver can block it before it reaches the physical GPU, protecting the entire system.

📊 Visual Representation: NVIDIA vGPU Software Architecture¶

This diagram displays the NVIDIA vGPU stack, illustrating how the vGPU Manager partitions physical GPUs for VM drivers.

📊 Comparison: With vs. Without Host Driver Interception¶

🛠️ Real-World Example of Interception in Action¶

Imagine a server with one NVIDIA A100 GPU running two VMs:

• VM 1 runs a model training job using 8 GB of GPU memory.
• VM 2 runs a model inference job using 4 GB of GPU memory.

When VM 1 sends a command to allocate 8 GB of memory, the host driver intercepts this request. It checks the allocation against VM 1's configured limit (8 GB) and the total available GPU memory (40 GB on an A100). Since the request is valid, the driver allocates the memory and returns a success response to VM 1.

Later, if VM 2 accidentally tries to allocate 20 GB (exceeding its 4 GB limit), the host driver intercepts this command and blocks it, returning an out-of-memory error to VM 2. The physical GPU and VM 1 remain unaffected.

🔍 Key Components in the Interception Path¶

• Guest vGPU Driver: Runs inside each VM; translates standard GPU APIs (CUDA, OpenGL) into commands the host driver understands.
• vGPU Manager: The host-level software that manages all vGPU instances and performs the actual interception.
• Physical GPU Driver: The low-level driver that communicates directly with the GPU hardware; only the host driver talks to this.
• Hypervisor Integration Layer: Connects the vGPU manager to the virtualization platform (e.g., VMware ESXi or KVM).

✅ Summary for New Engineers¶

• The host driver interception is the fundamental mechanism that makes GPU virtualization possible.
• Every GPU command from a guest VM passes through the host driver for validation, translation, and routing.
• This interception provides security, fairness, and stability — essential for running AI workloads in multi-tenant environments.
• The slight performance overhead from interception is a worthwhile trade-off for the ability to share expensive GPU hardware across multiple VMs.

Remember: Without this interception layer, you would not be able to safely run AI workloads in virtual machines — each VM would need its own dedicated physical GPU.