GPU Slicing & Multi-Instance GPU (MIG)

1. Concept Overview & Systemic Problem

Renting modern server GPUs (NVIDIA H100 or RTX 6000 Ada) incurs significant costs:

• Many specialized services (e.g., a model for generating vector embeddings or a micro-classification model) utilize only 5–10% of the flagship chip's computational power.
• Maintaining an entire GPU worth $35,000 for a lightweight model is a waste of company budget.
• Meanwhile, simply running multiple models on one card via standard Docker creates the risk of a "Noisy Neighbor": a heavy request from one model can consume all VRAM and crash all neighboring services.

GPU Slicing & MIG (Multi-Instance GPU) resolves this contradiction by allowing a single graphical giant to be sliced into several secure, reliable, and cost-effective virtual GPUs.

2. Architectural Taxonomy & Mental Model

┌─────────────────────────────────────────────────────────────┐
│                 NVIDIA HARDWARE MIG PARTITION               │
├─────────────────────────────────────────────────────────────┤
│ PHYSICAL NVIDIA H100 (80 GB HBM3 / 114 SMs)                 │
├─────────────────────────────────────────────────────────────┤
│ ┌──────────────────────┐ ┌───────────────────┐ ┌──────────┐ │
│ │ INSTANCE 1: 3g.40gb  │ │ INSTANCE 2: 2g.20gb│ │3: 1g.10gb│ │
│ │ • 40 GB VRAM         │ │ • 20 GB VRAM      │ │• 10 GB   │ │
│ │ • 42 SM Cores        │ │ • 28 SM Cores     │ │• 14 SM   │ │
│ │ • Runs 70B Coder LLM │ │ • Runs RAG Model  │ │• Embeds  │ │
│ └──────────────────────┘ └───────────────────┘ └──────────┘ │
│ • Isolated DMA channels, independent fault isolation        │
└─────────────────────────────────────────────────────────────┘

3. Technical Pipeline & Internal Mechanics

01. Configuring MIG Profiles on Ubuntu Server

Activate and slice the GPU with a single command using the NVIDIA utility:

sudo nvidia-smi -i 0 -mig 1
sudo nvidia-smi mig -cgi 9,19,19 -C

The server receives three independent devices /dev/nvidia0, /dev/nvidia1, /dev/nvidia2, which can be passed into three different Docker containers or virtual machines.

02. Utilizing vGPU in Kubernetes Clusters

A cloud provider slices a pool of GPUs into smaller vGPU fractions, allowing developers to request a pod with a limit of nvidia.com/gpu: 0.25, paying only a quarter of the GPU hourly cost.

4. Production Engineering Scenarios

01. Configuring MIG Profiles on Ubuntu Server

Activate and slice the GPU with a single command using the NVIDIA utility:

sudo nvidia-smi -i 0 -mig 1
sudo nvidia-smi mig -cgi 9,19,19 -C

The server receives three independent devices /dev/nvidia0, /dev/nvidia1, /dev/nvidia2, which can be passed into three different Docker containers or virtual machines.

02. Utilizing vGPU in Kubernetes Clusters

A cloud provider slices a pool of GPUs into smaller vGPU fractions, allowing developers to request a pod with a limit of nvidia.com/gpu: 0.25, paying only a quarter of the GPU hourly cost.

03. Deploying Multi-Instance Workloads

Designing workloads that leverage multiple MIG instances can maximize resource utilization. For instance, a single H100 can run a large language model alongside an embedding service and a testing environment, ensuring efficient use of VRAM and compute resources.

5. Pitfalls, Common Mistakes & Security

• Limitations of Consumer GPUs: The hardware MIG technology is only available on enterprise-grade server cards (A100, A30, H100, H200). Consumer-grade GeForce cards (RTX 4090 / 5090) only offer the less reliable software Time-Slicing.
• Inability to Dynamically Resize Without Stopping: Changing MIG profile configurations requires restarting the containers tied to those instances.