NVIDIA Multi-Instance GPU divides a supported physical GPU into isolated GPU instances with defined compute and memory resources. That changes the capacity model from “one pod gets one GPU” into a portfolio of instance profiles that can serve smaller workloads concurrently. MIG therefore belongs in the resource-governance layer of NVIDIA AI infrastructure, not merely as a hardware feature to turn on.
For NVIDIA NCA-AIIO candidates, the key design question is whether the workload benefits from deterministic partitioning more than it benefits from whole-GPU flexibility. MIG can improve utilization and create stronger performance isolation for appropriately sized jobs, but it also introduces profile planning, fragmentation, reconfiguration, monitoring, and compatibility constraints. A platform should enable MIG because it solves a defined workload problem, not because unused GPU memory looks inefficient on a dashboard.
MIG is also not a complete multi-tenant security architecture by itself. It creates hardware-backed resource separation inside a GPU, while the surrounding host, container runtime, kernel, storage, network, and control plane remain shared. The appropriate boundary must be evaluated alongside workload isolation and the sensitivity of the workloads sharing the node.
Instance profiles turn capacity planning into a geometry problem
A service catalog should publish the exact profiles application teams can request and the intended use cases for each one. Allowing arbitrary profile combinations can maximize theoretical flexibility while increasing fragmentation and support complexity. A small, measured profile set often produces higher effective utilization because scheduling behavior becomes predictable.
Capacity forecasts should include fragmentation as a first-class metric. Report physical GPUs, configured instance capacity, allocated instances, and unusable remainder separately. This prevents management dashboards from reporting “30% GPU free” when the remaining profile geometry cannot satisfy any queued production request.
Supported GPUs expose specific MIG profiles that allocate fractions of compute and device memory. The available combinations depend on the GPU generation and model. Capacity planning should therefore start from the actual profile table for the installed hardware rather than assuming a generic “half GPU” or “quarter GPU.” Model memory footprint, KV cache, batch size, framework overhead, and peak allocation all determine whether a profile is viable.
Packing efficiency matters because a set of requested profiles may not fit cleanly on the remaining GPU even when the unused memory total appears sufficient. Operators should think in supported profile geometry, not only aggregate gigabytes. A scheduler that repeatedly encounters unusable fragments may leave expensive capacity stranded. Standardizing a small set of service profiles can make placement and forecasting more predictable.
Isolation improves predictability but does not remove noisy neighbors everywhere
MIG instances receive isolated compute and memory resources within the GPU, which can make latency and throughput more predictable than unconstrained sharing. However, workloads on the same server may still share CPU, PCIe paths, NICs, storage, power, thermals, and host software. If those shared resources saturate, an application can experience interference even while its MIG slice remains healthy.
Service-level objectives should therefore include end-to-end measurements. If a small inference service receives a dedicated MIG profile but shares a congested network path or overloaded CPU socket, GPU isolation alone will not protect latency. Platform teams should baseline the whole node and decide which shared resources require quotas, topology-aware scheduling, or dedicated-node tiers.
Kubernetes needs a stable resource model for MIG instances
MIG Manager can reconcile configured layouts through Kubernetes-oriented control, but layout changes should still follow a maintenance workflow. A configuration label or custom resource is an instruction to reconfigure physical hardware; it should be protected by RBAC and change control appropriate to node-level infrastructure.
Cluster autoscaling also needs MIG-aware capacity groups. Adding a generic GPU node may not help a pending workload if the new node boots with the wrong MIG layout. Node templates should represent the intended profile geometry so the autoscaler adds the resource type that the scheduler actually needs.
GPU Operator and MIG Manager can configure MIG-enabled nodes and expose instance resources through the Kubernetes device stack. Kubernetes scheduling should then distinguish nodes and resource types so workloads request the intended profile rather than any available accelerator. A model that was qualified for one profile should not silently land on a materially different slice because the resource naming or admission policy is loose.
Changes to MIG geometry are infrastructure changes, not ordinary pod rescheduling. Reconfiguring a GPU can disrupt the instances already present, so operators should drain the node or otherwise coordinate workload removal before applying a new layout. Frequent on-demand reconfiguration can create more churn than value. Many platforms are easier to operate with node pools dedicated to a small number of stable MIG layouts.
Communication limitations affect which workloads belong on MIG
Time-slicing and MIG should not be treated as equivalent sharing models. Time-slicing multiplexes access to a GPU over time and can improve utilization, but it does not create the same hardware-isolated memory and compute partitions. A platform may offer both, with different performance predictability and tenancy guarantees, but the resource names and service expectations should make the distinction obvious.
For distributed workloads, qualification should test the exact communication pattern rather than relying on memory fit. A model that starts successfully on multiple MIG instances may still be unsupported or inefficient if it depends on peer access or collectives unavailable in that configuration. Admission policy can prevent known-incompatible distributed profiles from reaching production.
NVIDIA’s current deployment considerations document important restrictions around peer communication and collective behavior for MIG. NCCL is not supported with MIG in the cited deployment guidance, and CUDA IPC and peer-to-peer capabilities have constraints. Those limitations matter for distributed training or tightly coupled multi-GPU inference. A workload that depends on direct GPU-to-GPU communication may be a poor candidate for partitioning even if its memory footprint fits.
This is where model serving and training architecture must be considered separately. Small independent inference replicas can be natural MIG consumers because each instance serves requests largely on its own. A distributed service that expects tensor parallelism across devices has different interconnect requirements and may need whole GPUs or a different scale-up design.
Profile selection should be driven by measured workload envelopes
Model upgrades should trigger requalification of the selected profile. Parameter count is not the only driver of memory demand: tokenizer settings, context length, KV-cache growth, batching, quantization mode, and engine implementation can change the runtime envelope. A service that fit comfortably last quarter can begin failing after an application change with no infrastructure modification.
Admission controls can require a declared profile and validated model-service tier before deployment. That prevents a team from experimenting with arbitrary MIG shapes in the production pool and turning shared hardware into an uncontrolled compatibility lab.
Qualification should test more than whether a model starts. Measure steady-state memory, peak memory during loading and warm-up, tokens or samples per second, tail latency, batching behavior, and failure recovery. Leave headroom for framework changes and model revisions. A profile that operates at the edge of memory capacity may fail after an innocuous library upgrade or a larger request context.
Right-sizing also needs economic context. A profile that doubles theoretical density but causes enough latency degradation to require extra replicas may not improve cost. Compare useful work per physical GPU and per rack, not only the count of Kubernetes resources. The objective is sustainable service capacity.
Observability must preserve physical-GPU and instance context
Chargeback or capacity accounting should follow the same instance identity. A tenant consuming a small MIG profile should be measured against the physical capacity reserved by that profile, while shared host overhead is accounted for separately. Transparent accounting helps teams choose between denser partitioning and simpler whole-GPU service tiers.
Metrics should identify the physical GPU, MIG instance, workload, node, and profile. Aggregate GPU utilization can be misleading because several instances may have different load patterns. Health events at the physical GPU can affect every instance, while an application-level saturation issue may be isolated to one slice. Dashboards and alerts need both views.
Capacity reporting should also distinguish allocatable profiles from theoretical hardware totals. A cluster may have significant raw GPU memory free but no profile geometry that satisfies a pending workload. Exposing profile inventory and fragmentation makes scheduler failures understandable to application teams instead of appearing as mysterious “insufficient GPU” events.
MIG works best as a deliberate service tier
Review profile demand periodically; if one service tier is consistently saturated while another sits idle, rebalance node-pool geometry during planned maintenance instead of allowing permanent fragmentation to become normal capacity loss.
The NVIDIA platform should define where MIG is allowed, which profiles are supported, what workloads qualify, how nodes are reconfigured, and what evidence is required before a new profile enters production. That turns partitioning into a managed capacity product rather than an ad hoc optimization performed by individual teams.
Whole GPUs, MIG instances, and stronger virtual-machine isolation can coexist as different service classes. The platform succeeds when application owners choose among them based on workload behavior and risk rather than implementation convenience. Partitioning is valuable when it creates predictable capacity; it is counterproductive when it makes scheduling, communication, and incident ownership harder to reason about.