Slicing NVidia GPUs

GPUs Evolved from Virtualization to Hardware Partitioning

A decade ago, cloud providers started selling something that sounded almost impossible:

"Rent half a GPU."

You could launch a VM with part of an NVIDIA K80, M60, or similar accelerator and pay for less than a full card.

Today, the terminology has changed. Instead of "half a GPU", engineers talk about things like:

nvidia.com/mig-3g.40gb

or:

nvidia.com/mig-1g.10gb

A Kubernetes scheduler can now assign a fraction of an NVIDIA H100 with a defined amount of memory and compute capacity.

Both generations are often described as "fractional GPUs", but they represent completely different ideas.

The old world was:

GPU
 |
virtualization layer
 |
multiple users

The new world is:

GPU
 |
hardware partitions
 |
multiple independent GPU instances

A GPU Was Once a Single Unit

The original container model was simple.

A machine had:

GPU 0
GPU 1
GPU 2
GPU 3

A container requested one:

docker run \
  --gpus '"device=0"' \
  my-container

or several:

docker run \
  --gpus '"device=0,2"' \
  my-container

The container was not splitting anything. It was only controlling visibility.

The GPU was still a whole physical device.

Kubernetes worked similarly:

resources:
  limits:
    nvidia.com/gpu: 1

means:

"Give this pod one GPU."

But the AI industry created a problem:

What if a workload only needed 10GB of VRAM?

What if an inference service needed a small fraction of a large accelerator?

A $30,000 GPU sitting mostly idle was not attractive.

Cloud providers needed smaller GPU units.

The First Generation: Virtual GPUs

K60, K80, M60 Era

The first solution was not hardware partitioning.

It was virtualization.

Products such as NVIDIA GRID and NVIDIA vGPU allowed cloud providers and virtualization platforms to expose portions of GPUs to virtual machines.

The physical model looked something like:

Physical GPU
     |
     +---- VM A
     |
     +---- VM B
     |
     +---- VM C

The customer saw:

My VM has half a GPU

but internally:

Multiple workloads share the same GPU hardware.

The hypervisor and NVIDIA software stack managed access.

This approach enabled cloud products such as older Azure GPU VM families based on Tesla K-series and M-series GPUs.

Examples from that era included:

• NVIDIA Tesla K60
• NVIDIA Tesla K80
• NVIDIA Tesla M60

These GPUs were popular because they provided a way to sell smaller GPU allocations before hardware partitioning existed.

The Problem with Virtual GPU Sharing

Virtual GPU technology solved the business problem, but it had limitations.

1. Compute resources were still shared

Two workloads might both see a GPU slice, but the underlying CUDA cores were not physically separated.

A noisy workload could interfere and affect another workload.

2. Performance was less predictable

A customer buying "half a GPU" did not necessarily get:

• half the CUDA cores
• half the memory bandwidth
• half the cache

They got a virtualized share.

3. Kubernetes scheduling was difficult

Kubernetes understands resources like:

nvidia.com/gpu: 1

But what does:

half-a-k80: 1

actually mean?

• How much memory?
• How many CUDA cores?
• How much bandwidth?

There was no universal hardware abstraction.

Cloud Examples: Before MIG

Different cloud providers exposed GPU sharing differently.

A simplified historical view:

The important distinction:

Older fractional GPUs were usually virtual GPUs.

They were not physically carved/separated pieces of non-overlapping non-intersecting silicon.

Enter NVIDIA MIG

In 2020, NVIDIA introduced something fundamentally different:

Multi-Instance GPU (MIG)

MIG arrived with NVIDIA Ampere data center GPUs such as the A100 and continued with Hopper GPUs such as H100 and H200.

Instead of pretending a GPU was smaller, MIG actually partitions it.

A simplified view:

Before:

H100 80GB

+----------------+
|                |
|   One GPU      |
|                |
+----------------+

After MIG:

H100 80GB

+----------------+
| MIG instance   |
| 40GB           |
+----------------+

+----------------+
| MIG instance   |
| 40GB           |
+----------------+

Each MIG instance receives dedicated portions of:

• GPU compute resources
• memory
• cache resources
• GPU engines

The GPU becomes a collection of schedulable resources.

MIG Support: Which NVIDIA GPUs Can Actually Do This?

MIG is not available on every NVIDIA GPU.

The important distinction:

Data center GPUs with MIG support

Examples:

GPUs without MIG

Examples:

NVIDIA's MIG documentation specifically limits MIG support to supported data center products; many other workstation and virtualization-focused GPUs do not provide MIG.

Consumer GeForce cards

Consumer GPUs such as:

• RTX 4090
• RTX 4080
• RTX 3090
• GTX cards

do not support MIG.

You cannot take:

RTX 4090 24GB

and create:

12GB GPU
+
12GB GPU

using MIG.

Modern Cloud GPUs: Full GPU vs MIG GPU

Modern cloud providers now support both models.

For example:

Full GPU assignment

A VM receives:

1 x H100 80GB

The application sees:

CUDA device 0
80GB memory

MIG partitioning

The same physical GPU can become:

H100 80GB

+----------------+
|                |
|    H100        |
|    80GB        |
|                |
+----------------+

        |
        +--> 7 x 1g.10gb
        |       1g = 1 GPU slice (of 7)
        |       10gb = HBM memory allocated to each slice
        |
        +--> 2 x 3g.40gb
        |       3g = 3/7 of the GPU compute resources
        |       40gb = HBM memory allocated to each slice
        |
        +--> 1 x 7g.80gb
                7g = the entire GPU (all MIG compute resources)
                80gb = the full HBM memory capacity

g does not literally mean "number of slices"; it represents the number of GPU instances (GI) worth of compute resources out of the maximum 7 on an H100. So "3g" is best explained as "3/7 of the GPU compute partition" rather than simply "3 slices".

The number is different across GPU models. Examples:

So, an A30 does not have seven partitions. Its maximum is four GPU instances.

The reason is that MIG is not dividing only memory. It divides several physical GPU resources:

• GPU compute slices (SM partitions)
• memory controllers
• L2 cache portions
• copy engines
• media engines (where applicable)

The hardware layout determines how many independent partitions NVIDIA can create.

AWS documents MIG profiles for A100, H100, and H200 environments, including examples such as H100 1g.10gb, 2g.20gb, 3g.40gb, and 7g.80gb.

Azure also supports MIG-based GPU partitioning workflows on H100-based AKS deployments.

Enabling MIG on a GPU

MIG is not enabled by default. It is a GPU-level configuration that must be turned on before creating GPU partitions. The process happens on the host machine, before Docker or Kubernetes can consume the MIG devices.

First, verify that the GPU supports MIG:

nvidia-smi

Enable MIG mode on a specific GPU:

sudo nvidia-smi -i 0 -mig 1

or enable it on all supported GPUs:

sudo nvidia-smi -mig 1

Verify that MIG is enabled:

nvidia-smi -i 0 --query-gpu=name,mig.mode.current --format=csv

Example:

name, mig.mode.current
H100 80GB HBM3, Enabled

Once MIG mode is enabled, the GPU still needs to be partitioned into MIG instances. Enabling MIG only turns on the capability; it does not create the slices.

List the available MIG profiles:

sudo nvidia-smi mig -lgip

Example:

MIG 1g.10gb
MIG 2g.20gb
MIG 3g.40gb
MIG 7g.80gb

Create a partition layout:

sudo nvidia-smi mig -cgi 3g.40gb,3g.40gb -C

This creates two 40GB MIG instances on an H100 80GB:

H100 80GB

+----------------+
| 3g.40gb        |
+----------------+

+----------------+
| 3g.40gb        |
+----------------+

You can inspect the created MIG devices:

nvidia-smi -L

Example:

GPU 0: H100 80GB

  MIG 3g.40gb Device 0:
      MIG-GPU-xxxx/1/0

  MIG 3g.40gb Device 1:
      MIG-GPU-xxxx/2/0

Those MIG UUIDs are what Docker and Kubernetes ultimately expose to workloads.

A few operational notes:

• Enabling MIG generally requires exclusive access to the GPU because the driver may need to reset it. Stop running CUDA workloads before enabling it.
• On Ampere GPUs such as A100, enabling MIG may require a GPU reset; Hopper GPUs such as H100 have fewer restrictions around enabling MIG mode.
• In production environments, MIG layouts are usually automated at boot using tooling such as NVIDIA's MIG management tools rather than manually recreated each time.

Docker: Selecting GPUs

Before MIG, Docker selected physical devices.

One GPU:

docker run \
  --gpus '"device=0"' \
  pytorch-image

Two GPUs:

docker run \
  --gpus '"device=0,1"' \
  pytorch-image

Using environment variables:

docker run \
  -e NVIDIA_VISIBLE_DEVICES=0 \
  pytorch-image

or:

docker run \
  -e NVIDIA_VISIBLE_DEVICES=0,2 \
  pytorch-image

Docker Compose

A simple GPU assignment:

services:
  trainer:
    image: pytorch/pytorch
    environment:
      NVIDIA_VISIBLE_DEVICES: 0

Two GPUs:

services:
  trainer:
    image: pytorch/pytorch
    environment:
      NVIDIA_VISIBLE_DEVICES: 0,1

Docker With MIG

With MIG, the visible device is no longer:

GPU 0

It becomes a MIG UUID.

Find devices:

nvidia-smi -L

Example:

MIG-GPU-xxxx/1/0
MIG-GPU-xxxx/2/0

Run a container:

docker run \
  --gpus '"device=MIG-GPU-xxxx/1/0"' \
  my-container

The container now sees only that MIG instance.

Kubernetes: Full GPU Allocation

The simplest Kubernetes example:

resources:
  limits:
    nvidia.com/gpu: 1

Two GPUs:

resources:
  limits:
    nvidia.com/gpu: 2

This means:

"Give me physical GPUs."

It does not mean:

"Split a GPU."

Kubernetes With MIG

After enabling MIG and configuring the NVIDIA Kubernetes device plugin, MIG devices become schedulable resources.

Example:

resources:
  limits:
    nvidia.com/mig-3g.40gb: 1

The pod receives:

one 40GB MIG instance

Another pod can receive:

resources:
  limits:
    nvidia.com/mig-3g.40gb: 1

The scheduler places it on another available MIG slice.

Splitting an H100: Equal and Unequal Examples

An H100 80GB can be configured in different ways.

Two equal workloads

H100 80GB

+
+-- 3g.40gb
+
+-- 3g.40gb

Two pods:

resources:
  limits:
    nvidia.com/mig-3g.40gb: 1

Smaller inference workloads

H100 80GB

+-- 1g.10gb
+-- 1g.10gb
+-- 1g.10gb
+-- 1g.10gb
+-- 1g.10gb
+-- 1g.10gb
+-- 1g.10gb

Seven smaller workloads.

Mixed workloads

A cluster might choose:

H100 80GB

+-- 3g.40gb  (large model)
+-- 2g.20gb  (medium inference)
+-- 1g.10gb  (small service)
+-- 1g.10gb

The GPU is no longer a single scheduling unit.

What MIG Does Not Do

MIG does not magically split every GPU.

You cannot:

RTX 4090

into:

GPU A
GPU B

You cannot ask Kubernetes:

nvidia.com/half-gpu: 1

and expect it to work.

The partitioning happens at the GPU level before containers run.

The hierarchy is:

Physical GPU

      |
      v

MIG configuration

      |
      v

Kubernetes resources

      |
      v

Containers

The Mental Model

The useful unlock is that GPUs have become programmable infrastructure.

The old model:

GPU
 |
virtualization
 |
VMs

The new model:

GPU
 |
hardware partitions
 |
containers and Kubernetes pods

The story of GPU sharing is really the story of GPUs becoming infrastructure. What started as a software trick to make expensive accelerators easier to rent has evolved into hardware-level partitioning where a single H100 can safely become multiple independent GPU resources. Understanding the difference between vGPU, direct GPU assignment, and MIG is increasingly important as AI workloads move from individual machines into shared cloud and Kubernetes environments. The future of GPU computing is not simply about buying bigger GPUs: it is about making every unit of GPU capacity schedulable, measurable, and efficiently shared.