28.4e Creating, listing, and destroying MIG instances and compute instances¶

🧠 Context Introduction¶

Multi-Instance GPU (MIG) allows you to physically partition a single NVIDIA GPU (like the A100, A30, or H100) into multiple smaller, isolated GPU instances. Each instance has its own dedicated memory, cache, and compute units — making them behave like separate, smaller GPUs. This is extremely useful when you want to share a powerful GPU among multiple workloads without any interference.

As a new engineer, think of MIG like slicing a pizza into separate slices. Each slice is a complete, independent mini-pizza with its own toppings (memory and compute). You can create these slices, see which slices exist, and remove them when no longer needed.

⚙️ Creating MIG Instances — Partitioning the GPU¶

Creating a MIG instance means defining how you want to split the GPU's resources. You start by creating a GPU instance (the physical partition), then create a compute instance (the logical unit that workloads actually use).

• GPU Instance: Defines the physical boundaries — how much memory and how many compute units are allocated.
• Compute Instance: Runs inside a GPU instance and is what your applications (containers, VMs, etc.) actually see and use.

Key steps for creating MIG instances:

• First, you must enable MIG mode on the GPU. This tells the GPU driver that you intend to partition the GPU.
• Next, you choose a MIG profile (a predefined slice size). For example, on an A100 80GB GPU, a profile like 1g.10gb creates one slice with 10GB of memory and one compute unit.
• You then create the GPU instance using that profile.
• Finally, you create a compute instance inside that GPU instance.

Important note: You can create multiple MIG instances on a single GPU, as long as the total resources don't exceed what the GPU offers. For example, an A100 80GB can be split into seven 1g.10gb instances, or three 2g.20gb instances, or one large 7g.80gb instance.

📋 Listing MIG Instances — Seeing What You Have¶

Once you've created MIG instances, you'll want to verify they exist and see their details. Listing helps you:

• Confirm that your partitioning worked correctly.
• See which GPU instances and compute instances are active.
• Check how much memory and compute each instance has.
• Identify which instances are available for workloads.

What you'll typically see when listing:

• GPU Instance ID: A unique identifier for the physical partition.
• Profile Name: The slice size (e.g., 1g.10gb, 2g.20gb).
• Placement: Which physical GPU and which position on that GPU the instance occupies.
• Compute Instance ID: A unique identifier for the logical compute unit.
• Status: Whether the instance is active or idle.

Use case example: You might list instances to see if a 1g.10gb slice is free to run a small inference job, or if all slices are occupied by training workloads.

🗑️ Destroying MIG Instances — Cleaning Up¶

When a workload finishes, or if you need to reconfigure the GPU, you must destroy the MIG instances. Destruction is a two-step process that mirrors creation:

• Step 1: Destroy the compute instance first. You cannot destroy a GPU instance while a compute instance is still using it.
• Step 2: Destroy the GPU instance after the compute instance is gone.

Important rules to remember:

• Destroying a MIG instance is irreversible — all data in that instance's memory is lost.
• You must destroy instances in the correct order (compute first, then GPU).
• After destroying all instances, the GPU returns to its full, unpartitioned state.
• If you want to change the partition layout (e.g., from seven 1g slices to three 2g slices), you must destroy all existing instances first.

When to destroy instances:

• After a training job completes and you need the resources back.
• When reconfiguring the GPU for different workload sizes.
• During maintenance or when upgrading drivers.

📊 Visual Representation: MIG Instance Creation command loop¶

This flowchart maps out creating compute and GPU instances using target profile IDs.

🛠️ Practical Workflow Summary¶

Here's the typical lifecycle of a MIG instance from start to finish:

• Enable MIG mode on the target GPU.
• Create GPU instance(s) using your chosen profile(s).
• Create compute instance(s) inside each GPU instance.
• List instances to verify everything is set up correctly.
• Run your workload (container, VM, or bare-metal application) on the compute instance.
• List instances again to monitor usage.
• Destroy compute instance(s) when the workload finishes.
• Destroy GPU instance(s) to free the resources.
• Disable MIG mode if you want the GPU back to its full, unpartitioned state.

📊 Comparison: GPU Instance vs. Compute Instance¶

🕵️ Common Mistakes and Tips for New Engineers¶

• Forgetting to enable MIG mode: The GPU will not allow partitioning until MIG mode is active.
• Creating instances in the wrong order: Always create GPU instance first, then compute instance.
• Destroying in the wrong order: Always destroy compute instance first, then GPU instance.
• Over-allocating resources: The sum of all MIG profiles cannot exceed the GPU's total resources.
• Not listing before destroying: Always check which instances exist before destroying to avoid accidental removal.
• Mixing MIG and non-MIG workloads: A GPU in MIG mode cannot run non-MIG workloads, and vice versa.

✅ Key Takeaways¶

• MIG lets you split one physical GPU into multiple smaller, isolated GPUs.
• You create a GPU instance first, then a compute instance inside it.
• Listing instances helps you verify the partition layout and monitor usage.
• Destruction is the reverse of creation: destroy compute instance first, then GPU instance.
• Always check your current instances before making changes to avoid errors.
• MIG is ideal for sharing a powerful GPU among multiple smaller workloads without interference.

This guide gives you the foundational understanding to start working with MIG instances. As you gain experience, you'll learn to optimize partition sizes for different workloads and automate the creation/destruction process in your cluster orchestration workflows.