NVIDIA detailed on October 27 how CUDA 13.4 changes GPU sharing. The release adds a memory boundary that is transparent to applications: a workload requests an exact memory size and allocations above it fail as they would on a smaller GPU, or it can take a guaranteed minimum with a burstable maximum for when neighboring workloads are idle. Workloads can also choose sm-sharing, where they run concurrently, instead of the default time-slicing compute mode.
Enforcement needs something on each node, which is the job of a new open-source GPU Fractioning Operator hosted under the KAI Scheduler organization. It runs alongside the GPU Operator, starts and supervises Multi-Process Service on every GPU node, applies each container's memory limits and compute mode before the process starts, and reports memory and compute use per fraction rather than per whole GPU.
KAI Scheduler v0.18.0 adds an NvFractions mode that writes per-container pod annotations, covering the memory request, an optional upper limit, the compute mode and the assigned devices, which the operator then reads. NVIDIA says any scheduler able to write those annotations can use the mechanism, though the scheduler must still avoid mixing compute modes on one GPU. The post notes that sm-sharing places workloads in a single error domain, so it is meant for workloads trusted to run together, while time-slicing gives each job its own error domain at the cost of waiting its turn.
Our takeMemory enforcement closes a real gap: time-slicing could let one job's allocation starve its neighbors, and utilization was reported per device rather than per tenant. The design choice to watch is the error domain. Sharing SMs buys concurrency but couples failures across tenants; time-slicing isolates crashes but serializes compute. Neither reproduces the hardware isolation of MIG, so this makes oversubscription safer without making it equivalent to partitions.