Speakers
Description
Proposal
On modern multi-socket, multi-GPU systems, application performance is often limited not by compute availability but by poor CPU/GPU locality. Today, customers are frequently instructed to rely on strict node pinning (numactl) and disabling NUMA balancing in order to avoid costly cross-node memory accesses. While effective in some cases, this approach can be suboptimal, since it limits workloads to a subset of available resources and it also requires deep system knowledge from the user.
In this talk, we will explore how GPU-aware auto-affinitization can be implemented directly in the Linux scheduler using sched_ext, enabling dynamic and transparent placement of CPU tasks close to the GPUs they actively use. We discuss the challenges of integrating scheduler-driven task migration with NUMA balancing, handling mixed CPU/GPU thread groups, and avoiding resource over-concentration. Finally, we present design principles showing how scheduler-level techniques can outperform static affinity policies, while simplifying the user experience.
Key Challenges
Interaction with NUMA Balancing
Naively migrating a task closer to its GPU can backfire if the task's memory remains allocated on a remote NUMA node. In such cases, migration may increase memory access latency instead of reducing it.
Key questions explored:
How can sched_ext and NUMA balancing cooperate rather than conflict?
When should migration be deferred or paired with memory migration?
Thread Aggregation and Shared State
Many GPU-enabled applications use multi-threaded CPU components, where:
Only a subset of threads directly interacts with the GPU
Other threads share memory, locks, or cache lines with GPU-driving threads
Migrating only GPU-active threads may introduce new inefficiencies due to cross-node communication among threads of the same process.
We need to explore strategies for:
Selective thread aggregation, migrating related threads together
Avoiding overload of a single NUMA node or LLC
Balancing locality benefits against parallel resource contention
Resource Saturation and Fairness
Automatically clustering tasks near GPUs risks oversubscribing CPUs, LLCs, or memory bandwidth on specific NUMA nodes. The scheduler must therefore avoid overloading a certain LLC or NUMA node, make optimal use of system resources and maintain fairness.